Python pexpect logfile option works correctly in Jupyter Lab but not in python script

Question:

Environment:

  • OS: macOS Monterey
  • Python: 3.9.7
  • Jipyterlab: 3.0.14

Code:

import pexpect
import sys
from getpass import getpass

selection = '0'

mypass = getpass('Enter your NT/AD user password: ')

child = pexpect.spawn ('adfs_auth --profile 123456789012')
child.expect (r'Enter your password:(.*?)')
child.sendline (mypass)

child.logfile = sys.stdout
child.expect('Pick an account from the above list:')
child.sendline (selection)

Behaviors:

  • Jupyterlab: Works as expected
  • Python script: Error TypeError: write() argument must be str, not bytes

Detailed Behaviors:

After entering my password, pexepect takes it over, and I get the following output on the screen as expected (20 is just an example):

Starting process...
Challenge code for your Authenticator app: 20
Waiting for MFA response...
...
...

Once I give the correct MFA response on my phone, I get something similar to the following output on the screen.

You have access to 3 accounts

+----+--------------------+---------------+
| #  |  ACCOUNT ALIAS     |  ACCOUNT ID   |
+----+--------------------+---------------+
| 0  | account-0          | 123456789012  |
| 1  | account-1          | 234567890123  |
| 2  | account-2          | 345678901234  |
+----+--------------------+---------------+
Pick an account from the above list:

The last line of the code gives the selection (= 0) as the input and it is done.

This works as expected in Jupyterlab but getting the following error when I run as a python script:

/usr/local/bin/python3 "/Users/testuser/testproject/create_aws_profile.py"
Enter your NT/AD user password: 
Traceback (most recent call last):
  File "/Users/testuser/testproject/create_aws_profile.py", line 14, in <module>
    child.expect('Pick an account from the above list:')
  File "/usr/local/lib/python3.9/site-packages/pexpect/spawnbase.py", line 343, in expect
    return self.expect_list(compiled_pattern_list,
  File "/usr/local/lib/python3.9/site-packages/pexpect/spawnbase.py", line 372, in expect_list
    return exp.expect_loop(timeout)
  File "/usr/local/lib/python3.9/site-packages/pexpect/expect.py", line 169, in expect_loop
    incoming = spawn.read_nonblocking(spawn.maxread, timeout)
  File "/usr/local/lib/python3.9/site-packages/pexpect/pty_spawn.py", line 460, in read_nonblocking
    incoming = super(spawn, self).read_nonblocking(size)
  File "/usr/local/lib/python3.9/site-packages/pexpect/spawnbase.py", line 182, in read_nonblocking
    self._log(s, 'read')
  File "/usr/local/lib/python3.9/site-packages/pexpect/spawnbase.py", line 129, in _log
    self.logfile.write(s)
TypeError: write() argument must be str, not bytes

I reviewed the following two posts:

  1. pexpect.interact(): TypeError: write() argument must be str, not bytes
  2. Double Characters while do interact() using pexpect

If I use child.logfile = sys.stdout.buffer ore child.logfile_read = sys.stdout.buffer everything works except the last child.sendline (selection) which just shows that it worked but not actually inserted the selection (= 0) into the spawned program.

If I use child.logfile = None or child.logfile_read = None, nothing is displayed on the screen after entering the password.

Asked By: Rafiq

||
Source

Answers:

Issue #1

According to pexpect doc:

By default, spawn is a bytes interface: its read methods return bytes, and its write/send and expect methods expect bytes. If you pass the encoding parameter to the constructor, it will instead act as a unicode interface: strings you send will be encoded using that encoding, and bytes received will be decoded before returning them to you.

So try like this:

child = pexpect.spawn ('...', encoding='utf8')
child.logfile_read = sys.stdout

Issue #2

After child.sendline(selection) you need to wait for the spawned program to finish before exiting the whole Python script. Otherwise the spawned program may be terminated prematurely.

child.sendline(selection)
child.expect(pexpect.EOF)
Answered By: pynexj