Connecting to Globalscape SFTP server with two-factor password and key authentication using Python Paramiko/pysftp

Question:

While working on a file upload project, using pysftp/paramiko I stumbled on an SFTP server connection issue:

  • The SFTP server requires the following authentication flow: username + private key -> ‘welcome the sftp server’ -> password
  • Using Linux sftp CLI command I can access the server with the flow above
  • When trying to access the server with the ssh CLI command, I get the password request and then the following error:
PTY allocation request failed on channel 0
shell request failed on channel 0

It appears the server has the shell disabled on the SFTP port. And I wonder how to make paramiko/pysftp work with such a configuration.

I tried the standard approach with

ssh = paramiko.SSHClient()
ssh.connect(hostname=host, username=user, port=port,password=password, pkey=pkey)

But this results in

AuthenticationException: Authentication failed.

I also tried the paramiko.SFTPClient.from_transport but this also results in an error

SSHException: SSH session not active

I will be thankful for any suggestions on how to approach this problem, avoiding cmd subprocess injections.

Additionally I am sharing:

Paramiko log:

DEB [20220828-13:34:39.668] thr=4   paramiko.transport: starting thread (client mode): 0x185a40d0
DEB [20220828-13:34:39.668] thr=4   paramiko.transport: Local version/idstring: SSH-2.0-paramiko_2.11.0
DEB [20220828-13:34:39.785] thr=4   paramiko.transport: Remote version/idstring: SSH-2.0-1.82_sshlib Globalscape
INF [20220828-13:34:39.785] thr=4   paramiko.transport: Connected (version 2.0, client 1.82_sshlib)
DEB [20220828-13:34:39.862] thr=4   paramiko.transport: === Key exchange possibilities ===
DEB [20220828-13:34:39.862] thr=4   paramiko.transport: kex algos: diffie-hellman-group16-sha512, diffie-hellman-group14-sha256, diffie-hellman-group-exchange-sha256, diffie-hellman-group14-sha1, diffie-hellman-group-exchange-sha1
DEB [20220828-13:34:39.862] thr=4   paramiko.transport: server key: ssh-rsa
DEB [20220828-13:34:39.862] thr=4   paramiko.transport: client encrypt: aes256-cbc, aes256-ctr, 3des-cbc, aes128-cbc, aes128-ctr
DEB [20220828-13:34:39.862] thr=4   paramiko.transport: server encrypt: aes256-cbc, aes256-ctr, 3des-cbc, aes128-cbc, aes128-ctr
DEB [20220828-13:34:39.862] thr=4   paramiko.transport: client mac: hmac-sha2-512, hmac-sha2-256, hmac-sha1, hmac-md5, hmac-sha1-96, hmac-md5-96
DEB [20220828-13:34:39.862] thr=4   paramiko.transport: server mac: hmac-sha2-512, hmac-sha2-256, hmac-sha1, hmac-md5, hmac-sha1-96, hmac-md5-96
DEB [20220828-13:34:39.862] thr=4   paramiko.transport: client compress: zlib, none
DEB [20220828-13:34:39.862] thr=4   paramiko.transport: server compress: zlib, none
DEB [20220828-13:34:39.862] thr=4   paramiko.transport: client lang: <none>
DEB [20220828-13:34:39.862] thr=4   paramiko.transport: server lang: <none>
DEB [20220828-13:34:39.862] thr=4   paramiko.transport: kex follows: False
DEB [20220828-13:34:39.862] thr=4   paramiko.transport: === Key exchange agreements ===
DEB [20220828-13:34:39.862] thr=4   paramiko.transport: Kex: diffie-hellman-group16-sha512
DEB [20220828-13:34:39.862] thr=4   paramiko.transport: HostKey: ssh-rsa
DEB [20220828-13:34:39.862] thr=4   paramiko.transport: Cipher: aes128-ctr
DEB [20220828-13:34:39.862] thr=4   paramiko.transport: MAC: hmac-sha2-256
DEB [20220828-13:34:39.862] thr=4   paramiko.transport: Compression: none
DEB [20220828-13:34:39.862] thr=4   paramiko.transport: === End of kex handshake ===
DEB [20220828-13:34:40.153] thr=4   paramiko.transport: kex engine KexGroup16SHA512 specified hash_algo <built-in function openssl_sha512>
DEB [20220828-13:34:40.153] thr=4   paramiko.transport: Switch to new keys ...
DEB [20220828-13:34:40.154] thr=2   paramiko.transport: Trying SSH key b'some_characters'
DEB [20220828-13:34:40.260] thr=4   paramiko.transport: userauth is OK
DEB [20220828-13:34:40.260] thr=4   paramiko.transport: Finalizing pubkey algorithm for key of type 'ssh-rsa'
DEB [20220828-13:34:40.260] thr=4   paramiko.transport: Our pubkey algorithm list: ['ssh-rsa']
DEB [20220828-13:34:40.260] thr=4   paramiko.transport: Server did not send a server-sig-algs list; defaulting to our first preferred algo ('ssh-rsa')
DEB [20220828-13:34:40.260] thr=4   paramiko.transport: NOTE: you may use the 'disabled_algorithms' SSHClient/Transport init kwarg to disable that or other algorithms if your server does not support them!
INF [20220828-13:34:40.265] thr=4   paramiko.transport: Auth banner: b'Welcome to TheServerService SFTP'
INF [20220828-13:34:40.364] thr=4   paramiko.transport: Authentication continues...
DEB [20220828-13:34:40.364] thr=4   paramiko.transport: Methods: ['password', 'keyboard-interactive']
INF [20220828-13:34:40.458] thr=4   paramiko.transport: Disconnect (code 2): unexpected service request

And verbose sftp log:

OpenSSH_8.6p1, LibreSSL 3.3.6
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files
debug1: /etc/ssh/ssh_config line 54: Applying options for *
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> 'mydesktop/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> 'mydesktop/.ssh/known_hosts2'
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug1: Connecting to the server port port.
debug1: Connection established.
debug1: identity file id_key.key type -1
debug1: identity file id_key.key-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.6
debug1: Remote protocol version 2.0, remote software version 1.82_sshlib Globalscape
debug1: compat_banner: no match: 1.82_sshlib Globalscape
debug3: fd 6 is O_NONBLOCK
debug1: Authenticating to the_server:port as 'user'
debug3: put_host_port: [the_server]:port
debug3: record_hostkey: found key type RSA in file mydesktop/.ssh/known_hosts:7
debug3: load_hostkeys_file: loaded 1 keys from [the_server]:port2
debug1: load_hostkeys: fopen my_desktop/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug3: order_hostkeyalgs: prefer hostkeyalgs: [email protected],[email protected],[email protected],rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c
debug2: host key algorithms: [email protected],[email protected],[email protected],rsa-sha2-512,rsa-sha2-256,ssh-rsa,[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ssh-ed25519,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],[email protected]
debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,[email protected],zlib
debug2: compression stoc: none,[email protected],zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: diffie-hellman-group16-sha512,diffie-hellman-group14-sha256,diffie-hellman-group-exchange-sha256,diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha1
debug2: host key algorithms: ssh-rsa
debug2: ciphers ctos: aes256-cbc,aes256-ctr,3des-cbc,aes128-cbc,aes128-ctr
debug2: ciphers stoc: aes256-cbc,aes256-ctr,3des-cbc,aes128-cbc,aes128-ctr
debug2: MACs ctos: hmac-sha2-512,hmac-sha2-256,hmac-sha1,hmac-md5,hmac-sha1-96,hmac-md5-96
debug2: MACs stoc: hmac-sha2-512,hmac-sha2-256,hmac-sha1,hmac-md5,hmac-sha1-96,hmac-md5-96
debug2: compression ctos: zlib,none
debug2: compression stoc: zlib,none
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug1: kex: algorithm: diffie-hellman-group-exchange-sha256
debug1: kex: host key algorithm: ssh-rsa
debug1: kex: server->client cipher: aes128-ctr MAC: hmac-sha2-256 compression: none
debug1: kex: client->server cipher: aes128-ctr MAC: hmac-sha2-256 compression: none
debug3: send packet: type 34
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(2048<8192<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug3: receive packet: type 31
debug1: SSH2_MSG_KEX_DH_GEX_GROUP received
debug2: bits set: 1016/2048
debug3: send packet: type 32
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug3: receive packet: type 33
debug1: SSH2_MSG_KEX_DH_GEX_REPLY received
debug1: Server host key: ssh-rsa SHA256:some_characters/some_characters
debug3: put_host_port: [the_server]:port
debug3: put_host_port: [the_server]:port
debug3: record_hostkey: found key type RSA in file mydesktop/.ssh/known_hosts:7
debug3: load_hostkeys_file: loaded 1 keys from [the_server]:port
debug1: load_hostkeys: fopen mydesktop/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: Host '[the_server]:port' is known and matches the RSA host key.
debug1: Found key in mydesktop/.ssh/known_hosts:7
debug2: bits set: 1038/2048
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey out after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey in after 4294967296 blocks
debug1: Will attempt key: id_key.key  explicit
debug2: pubkey_prepare: done
debug3: send packet: type 5
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 53
debug3: input_userauth_banner: entering
Welcome to TheServerService SFTPdebug3: receive packet: type 51
debug1: Authentications that can continue: password,publickey,keyboard-interactive
debug3: start over, passed a different list password,publickey,keyboard-interactive
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Trying private key: id_key.key
debug3: sign_and_send_pubkey: RSA SHA256:E2e/Cwlca+/some_characters
debug3: sign_and_send_pubkey: signing using ssh-rsa SHA256:E2e/Cwlca+/some_characters
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 51
Authenticated with partial success.
debug1: Authentications that can continue: password,keyboard-interactive
debug3: start over, passed a different list password,keyboard-interactive
debug3: preferred publickey,keyboard-interactive,password
debug3: authmethod_lookup keyboard-interactive
debug3: remaining preferred: password
debug3: authmethod_is_enabled keyboard-interactive
debug1: Next authentication method: keyboard-interactive
debug2: userauth_kbdint
debug3: send packet: type 50
debug2: we sent a keyboard-interactive packet, wait for reply
debug3: receive packet: type 60
debug2: input_userauth_info_req: entering
debug2: input_userauth_info_req: num_prompts 1
(user@the_server) Enter password: 
debug3: send packet: type 61
debug3: receive packet: type 52
debug1: Authentication succeeded (keyboard-interactive).
Authenticated to the_server ([the_server]:port).
debug2: fd 7 setting O_NONBLOCK
debug3: fd 8 is O_NONBLOCK
debug1: channel 0: new [client-session]
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug3: send packet: type 90
debug1: Entering interactive session.
debug1: pledge: filesystem full
debug3: receive packet: type 91
debug2: channel_input_open_confirmation: channel 0: callback start
debug2: fd 6 setting TCP_NODELAY
debug3: set_sock_tos: set socket 6 IP_TOS 0x20
debug2: client_session2_setup: id 0
debug1: Sending environment.
debug3: Ignored env USER
debug3: Ignored env MallocNanoZone
debug3: Ignored env __CFBundleIdentifier
debug3: Ignored env COMMAND_MODE
debug3: Ignored env LOGNAME
debug3: Ignored env PATH
debug3: Ignored env SSH_AUTH_SOCK
debug3: Ignored env SHELL
debug3: Ignored env HOME
debug3: Ignored env __CF_USER_TEXT_ENCODING
debug3: Ignored env TMPDIR
debug3: Ignored env XPC_SERVICE_NAME
debug3: Ignored env XPC_FLAGS
debug3: Ignored env ORIGINAL_XDG_CURRENT_DESKTOP
debug3: Ignored env APPLICATION_INSIGHTS_NO_DIAGNOSTIC_CHANNEL
debug3: Ignored env TERM_PROGRAM
debug3: Ignored env TERM_PROGRAM_VERSION
debug1: channel 0: setting env LANG = "en_US.UTF-8"
debug2: channel 0: request env confirm 0
debug3: send packet: type 98
debug3: Ignored env COLORTERM
debug3: Ignored env GIT_ASKPASS
debug3: Ignored env VSCODE_GIT_ASKPASS_NODE
debug3: Ignored env VSCODE_GIT_ASKPASS_EXTRA_ARGS
debug3: Ignored env VSCODE_GIT_ASKPASS_MAIN
debug3: Ignored env VSCODE_GIT_IPC_HANDLE
debug3: Ignored env VSCODE_INJECTION
debug3: Ignored env ZDOTDIR
debug3: Ignored env PWD
debug3: Ignored env TERM
debug3: Ignored env SHLVL
debug3: Ignored env OLDPWD
debug3: Ignored env HOMEBREW_PREFIX
debug3: Ignored env HOMEBREW_CELLAR
debug3: Ignored env HOMEBREW_REPOSITORY
debug3: Ignored env MANPATH
debug3: Ignored env INFOPATH
debug3: Ignored env PYENV_ROOT
debug3: Ignored env P9K_TTY
debug3: Ignored env _P9K_TTY
debug3: Ignored env ZSH
debug3: Ignored env PAGER
debug3: Ignored env LESS
debug3: Ignored env LSCOLORS
debug3: Ignored env PYENV_SHELL
debug3: Ignored env P9K_SSH
debug3: Ignored env VIRTUAL_ENV
debug3: Ignored env PS1
debug3: Ignored env _
debug1: Sending subsystem: sftp
debug2: channel 0: request subsystem confirm 1
debug3: send packet: type 98
debug2: channel_input_open_confirmation: channel 0: callback done
debug2: channel 0: open confirm rwindow 33554432 rmax 35840
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: subsystem request accepted on channel 0
debug2: Remote version: 3
Connected to the_server.
debug3: Sent message fd 3 T:16 I:1
debug3: SSH_FXP_REALPATH . -> / size 0
Asked By: RafalSiwek

||

Answers:

Your server is using a keyboard-interactive authentication, not a simple password authentication:

debug1: Next authentication method: keyboard-interactive
debug2: userauth_kbdint
debug2: we sent a keyboard-interactive packet, wait for reply
debug2: input_userauth_info_req: entering
debug2: input_userauth_info_req: num_prompts 1
(user@the_server) Enter password: 
debug1: Authentication succeeded (keyboard-interactive).

Normally Paramiko is smart enough to fallback to the keyboard-interactive authentication, when the password authentication fails and the keyboard-interactive prompt has one field only (likely a password).

The problem is probably the combination with key authentication that stops Paramiko from being "smart".

You can explicitly make Paramiko try the keyboard-interactive authentication (after the key authentication) using low-level Transport class with its Transport.auth_publickey and Transport.auth_interactive:

def handler(title, instructions, fields):
    if len(fields) > 1:
        raise SSHException("Expecting one field only.")
    return [password]

transport = paramiko.Transport('example.com') 
transport.connect(username='myuser', hostkey=hostkey)
transport.auth_publickey(username, key)
transport.auth_interactive(username, handler)

sftp = transport.open_sftp_client()

Similar question: Password authentication in Python Paramiko fails, but same credentials work in SSH/SFTP client


Though with your specific server (Globalscape), this does not seem to work. You had success with the code from Multi-factor authentication (password and key) with Paramiko.

If I understand the difference correctly, my version above does separate service request for each authentication method. While the code by @osekmedia does both authentications within one service request, what Paramiko API does not allow normally. The code fabricates fake service request response to make Paramiko believe the server is ready for the second authentication.

Answered By: Martin Prikryl

It looks like the SFTP server comes with kind of a multifactor authentication flow, as it stands in the paramiko.log:

INF [20220828-13:34:40.364] thr=4   paramiko.transport: Authentication continues...
DEB [20220828-13:34:40.364] thr=4   paramiko.transport: Methods: ['password', 'keyboard-interactive']

The code snippet suggested in here was helpful and easy to implement.

Many thanks to Martin Prikryl for pointing in the right direction and osekmedia for the response!

Answered By: RafalSiwek
Categories: questions Tags: , , , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.