Pysftp "cd" fails against AIX server with "OverflowError: mode out of range" error
Question:
Having trouble getting the pysftp.connection.cd() working – not sure what I am doing wrong or what the issue is. I can achieve what I need by passing the full remote path into the relevant methods. But if try to change the remote active directory I get error. Relevant code is below:
with ps.Connection('hostname or IP address', username='username', password='secret', cnopts=cnopts) as conn:
print("Connection successful....")
print(conn.listdir()) # this works
print(conn.listdir(remote_location)) # this works
with conn.cd(remote_location): # this throws an error
print(conn.listdir())
The error I get is below here:
File "C:Usersusername.condaenvsdata_wranglinglibsite-packagespysftp__init__.py", line 508, in cd
self.cwd(remotepath)
File "C:Usersusername.condaenvsdata_wranglinglibsite-packagespysftp__init__.py", line 524, in chdir
self._sftp.chdir(remotepath)
File "C:Usersusername.condaenvsdata_wranglinglibsite-packagesparamikosftp_client.py", line 659, in chdir
if not stat.S_ISDIR(self.stat(path).st_mode):
OverflowError: mode out of range
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c:Usersusernamemy_codepythonmears_to_er_interfacesftp_chdir_test.py", line 41, in <module>
with conn.cd("remote_location"):
File "C:Usersusername.condaenvsdata_wranglinglibcontextlib.py", line 112, in __enter__
return next(self.gen)
File "C:Usersusername.condaenvsdata_wranglinglibsite-packagespysftp__init__.py", line 511, in cd
self.cwd(original_path)
File "C:Usersusername.condaenvsdata_wranglinglibsite-packagespysftp__init__.py", line 524, in chdir
self._sftp.chdir(remotepath)
File "C:Usersusername.condaenvsdata_wranglinglibsite-packagesparamikosftp_client.py", line 659, in chdir
if not stat.S_ISDIR(self.stat(path).st_mode):
OverflowError: mode out of range
The output for conn.stat(remote_location).st_mode is 83448.
Answers:
Paramiko uses Python stat module in the SFTPClient.chdir implementation to check if the path is a folder (and not a file).
The problem is that the Python stat module can work with 16-bit permissions only. Your SFTP server returns permissions with 17th bit set. That results in the OverflowError.
There’s not much you can do about it. See this Python issue: stat.S_ISXXX can raise OverflowError for remote file modes.
Just avoid using the SFTPClient.chdir. Simply use absolute paths everywhere in the Paramiko API, instead of relying on paths relative to the working directory set using chdir.
The only other place where Paramiko currently uses the stat module is the SFTPAttributes.__str__. Otherwise you should be safe.
I’ve reported this to the Paramiko project: https://github.com/paramiko/paramiko/issues/1802
If you want to analyze the permission bits in your own code, just mask out the excess bits. See:
"OverflowError: mode out of range" in Python when working with file mode/attributes returned by Paramiko
Having trouble getting the pysftp.connection.cd() working – not sure what I am doing wrong or what the issue is. I can achieve what I need by passing the full remote path into the relevant methods. But if try to change the remote active directory I get error. Relevant code is below:
with ps.Connection('hostname or IP address', username='username', password='secret', cnopts=cnopts) as conn:
print("Connection successful....")
print(conn.listdir()) # this works
print(conn.listdir(remote_location)) # this works
with conn.cd(remote_location): # this throws an error
print(conn.listdir())
The error I get is below here:
File "C:Usersusername.condaenvsdata_wranglinglibsite-packagespysftp__init__.py", line 508, in cd
self.cwd(remotepath)
File "C:Usersusername.condaenvsdata_wranglinglibsite-packagespysftp__init__.py", line 524, in chdir
self._sftp.chdir(remotepath)
File "C:Usersusername.condaenvsdata_wranglinglibsite-packagesparamikosftp_client.py", line 659, in chdir
if not stat.S_ISDIR(self.stat(path).st_mode):
OverflowError: mode out of range
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "c:Usersusernamemy_codepythonmears_to_er_interfacesftp_chdir_test.py", line 41, in <module>
with conn.cd("remote_location"):
File "C:Usersusername.condaenvsdata_wranglinglibcontextlib.py", line 112, in __enter__
return next(self.gen)
File "C:Usersusername.condaenvsdata_wranglinglibsite-packagespysftp__init__.py", line 511, in cd
self.cwd(original_path)
File "C:Usersusername.condaenvsdata_wranglinglibsite-packagespysftp__init__.py", line 524, in chdir
self._sftp.chdir(remotepath)
File "C:Usersusername.condaenvsdata_wranglinglibsite-packagesparamikosftp_client.py", line 659, in chdir
if not stat.S_ISDIR(self.stat(path).st_mode):
OverflowError: mode out of range
The output for conn.stat(remote_location).st_mode is 83448.
Paramiko uses Python stat module in the SFTPClient.chdir implementation to check if the path is a folder (and not a file).
The problem is that the Python stat module can work with 16-bit permissions only. Your SFTP server returns permissions with 17th bit set. That results in the OverflowError.
There’s not much you can do about it. See this Python issue: stat.S_ISXXX can raise OverflowError for remote file modes.
Just avoid using the SFTPClient.chdir. Simply use absolute paths everywhere in the Paramiko API, instead of relying on paths relative to the working directory set using chdir.
The only other place where Paramiko currently uses the stat module is the SFTPAttributes.__str__. Otherwise you should be safe.
I’ve reported this to the Paramiko project: https://github.com/paramiko/paramiko/issues/1802
If you want to analyze the permission bits in your own code, just mask out the excess bits. See:
"OverflowError: mode out of range" in Python when working with file mode/attributes returned by Paramiko