Execute commands on EC2 using subprocess and aws ssm

Question

I am working with group of EC2 instances that are deployed in a subnet.

I usually use the aws ssm command

aws ssm start-session --region us-east-2 --target i-01234567abcdef --profile profile-one

to connect to that ec2 instance and run commands such as

ssh mirrora

to go into a different ec2 instance in the subnet from the one I am connected to

Then I can run ls on mirrora to look at the files and such.

Now I am trying to automate the same using python

I am new to python (and first time working with subprocess). I have researched a bit and learned that I can use Popen to open a subprocess and I set the stdin and stdout arguments to subprocess.PIPE

Here’s a sample of the code I am trying to execute

command = "aws ssm start-session --region us-east-2 --target i-0123456abcdef --profile profile-one"

ssh = subprocess.Popen(command.split(), stdin=subprocess.PIPE, stdout=subprocess.PIPE, bufsize=0, shell= True, universal_newlines=True)

ssh.stdin.write("ssh mirrora")
ssh.stdin.write("ls")
# output = ssh.stdout.read()
# output , err = ssh.communicate()
# print("output:" + output)
# print("err:" + err)
# output = ssh.stdout.read()
# print("Output of ls is:", output)

print("HERE")

If I use either stdout.read or communicate method, the program seems to get stuck and nothing works. I can’t even Ctrl + C the terminal.

If I run it without any stdout or communicate, it prints "HERE" in the terminal (almost instantaneously) and nothing actually seems to happen on the server side (as I tried to run the shutdown command to turn off the mirrora, but it is still running after program exits without stdout or communicate).

What am I doing wrong?

Ideally I want to run the commands I want inside the subprocess that aws ssm opens and run certain commands on one of the ec2 instances and get the output for one or more commands using stdout.read after certain stdin.write

Asked By: Narendra Babu

||

Source

Answer 1

It seems to me that you should use run-command instead start-session.

This is an SSM feature used to execute static commands and scripts, where a session is similar to just an ssh connection.

I think it should work like this:

import boto3

script = f"""
ssh mirrora 'ls'
ssh mirrora 'ls; ps -aux'
"""
target = "i-01234567abcdef "

session = boto3.Session(profile_name="profile-one")
ssm_client = session.client("ssm")


response = ssm_client.send_command(
    InstanceIds=[target],
    DocumentName="AWS-RunShellScript",  # https://eu-central-1.console.aws.amazon.com/systems-manager/documents/AWS-RunShellScript/description?region=eu-central-1
    TimeoutSeconds=20,
    Comment="Example of send command",
    Parameters={
        'commands': [
            script,
        ],
        'workingDirectory': "/home/user1",
    },
)

Here you can find the documentation for send-command API. https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ssm.html#SSM.Client.send_command

If you want to view the logs in the python script then you will need to extend it – probably easiest way is to set an s3 bucket in the send_command (it will redirect your stdout and stderr to provided s3 bucket), download saved in s3 files, and cat/print them.

Answered By: ZabielskiGabriel

Execute commands on EC2 using subprocess and aws ssm

Question:

Answers: