Getting error while deleting files from a folder using python script
Question:
Im running a python script which deletes the files from a folder.im running the script in a virtual machine…but im getting error as follows:
Traceback (most recent call last):
File "C:UsersDesktophytapython_scriptspython_scrfile_delete.py", line 66, in <module
compare_and_delete_sql_files(folder_a, folder_b, output_csv, sql_filenames)
File "C:UsersDesktophytapython_scriptspython_scrfile_delete.py", line 53, in compare_and_delete_sql_files
os.remove(os.path.join(folder_b, filename))
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:UsersDesktophytapython_scriptspython_scr/folder_b\scrum_1.sql'
My code:
import os
import csv
def extract_sql_filenames_from_yaml(yaml_file):
sql_filenames = {}
with open(yaml_file, 'r') as file:
is_sql_section = False
for line in file:
line = line.strip()
if line == 'sql:':
is_sql_section = True
elif is_sql_section and line.startswith('- name:') and not line.startswith('#'):
filename = line.split(':')[-1].strip()
if filename.endswith('.sql'):
sql_filenames[filename] = sql_filenames.get(filename, 0) + 1
elif line.startswith('-') and not line.startswith('#'):
is_sql_section = False
return sql_filenames
def compare_and_delete_sql_files(folder_a, folder_b, output_csv, sql_filenames):
processed_filenames = set() # To keep track of processed filenames
# Initialize CSV file writer
with open(output_csv, 'w', newline='') as csvfile:
fieldnames = ['SQL File Name', 'Occurrences', 'Match Status', 'Delete Status']
writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
writer.writeheader()
# Iterate through each SQL filename extracted from YAML
for filename, occurrences in sql_filenames.items():
# Skip processing if the filename has already been processed
if filename in processed_filenames:
continue
# Mark the filename as processed
processed_filenames.add(filename)
sql_file_a = os.path.join(folder_a, filename)
match_status = 'Not Matched'
delete_status = 'Not Deleted'
# Check if the file exists in folder_a
if os.path.exists(sql_file_a):
# Compare with folder_b if necessary
if os.path.exists(os.path.join(folder_b, filename)):
with open(sql_file_a, 'r') as f_a, open(os.path.join(folder_b, filename), 'r') as f_b:
lines_a = f_a.readlines()
lines_b = f_b.readlines()
if lines_a == lines_b:
match_status = 'Matched'
delete_status = 'Deleted'
# Delete file from folder_b
os.remove(os.path.join(folder_b, filename))
# Write to CSV
writer.writerow({'SQL File Name': filename, 'Occurrences': occurrences,
'Match Status': match_status, 'Delete Status': delete_status})
# Example usage
folder_a = '/path/to/folder/a'
folder_b = '/path/to/folder/b'
output_csv = 'comparison_results.csv'
yaml_file = 'example.yaml'
sql_filenames = extract_sql_filenames_from_yaml(yaml_file)
compare_and_delete_sql_files(folder_a, folder_b, output_csv, sql_filenames)
Answers:
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process:
The error indicated that file is open, so it can’t be deleted.
It is happening because you are deleting the file inside with open block of your code.
Execute this line os.remove(os.path.join(folder_b, filename)) outside of the open block.
Make sure folder_b and filename variables are available outside of the open block.
Some initial Questions:
- Should the file path have a forward slash in it (see file path reference below)?
- Do you have the file opened via a different thread or process (going to assume you don’t have it open on your PC)?
- Is the error on the first or last sql script that you are processing or was it able to process several sql scripts successfully but fail on that particular script file?
Let me know if you have additional info or logging built into the script operations.
File Path Reference:
C:UsersDesktophytapython_scriptspython_scr**/**folder_bscrum_1.sql
On windows, you cannot delete a file if it is open in any other process hence [WinError 32].[microsoft.com]
Even if you did not open the file in your own process, it might have been opened in another process hence os.remove() is not guaranteed to succeed on windows.
I’m your case, the file is open in your own process and you are attempting to delete the open file within the same context [python.org] hence the [WinError 32]. Close the file and then delete immediately to ensure that you are guaranteed to still be the permitted file holder and you are not violating any sharing policy. [1]
This is another one of the many reasons why windows is respectfully such a ”great” OS. Funny enough, there’s no such headache on Unix based OSes.
[1] I’ve had one such experience where another process takes hold of the same file right before you want to use it again. This led to a mind blowing bug.
Im running a python script which deletes the files from a folder.im running the script in a virtual machine…but im getting error as follows:
Traceback (most recent call last):
File "C:UsersDesktophytapython_scriptspython_scrfile_delete.py", line 66, in <module
compare_and_delete_sql_files(folder_a, folder_b, output_csv, sql_filenames)
File "C:UsersDesktophytapython_scriptspython_scrfile_delete.py", line 53, in compare_and_delete_sql_files
os.remove(os.path.join(folder_b, filename))
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:UsersDesktophytapython_scriptspython_scr/folder_b\scrum_1.sql'
My code:
import os
import csv
def extract_sql_filenames_from_yaml(yaml_file):
sql_filenames = {}
with open(yaml_file, 'r') as file:
is_sql_section = False
for line in file:
line = line.strip()
if line == 'sql:':
is_sql_section = True
elif is_sql_section and line.startswith('- name:') and not line.startswith('#'):
filename = line.split(':')[-1].strip()
if filename.endswith('.sql'):
sql_filenames[filename] = sql_filenames.get(filename, 0) + 1
elif line.startswith('-') and not line.startswith('#'):
is_sql_section = False
return sql_filenames
def compare_and_delete_sql_files(folder_a, folder_b, output_csv, sql_filenames):
processed_filenames = set() # To keep track of processed filenames
# Initialize CSV file writer
with open(output_csv, 'w', newline='') as csvfile:
fieldnames = ['SQL File Name', 'Occurrences', 'Match Status', 'Delete Status']
writer = csv.DictWriter(csvfile, fieldnames=fieldnames)
writer.writeheader()
# Iterate through each SQL filename extracted from YAML
for filename, occurrences in sql_filenames.items():
# Skip processing if the filename has already been processed
if filename in processed_filenames:
continue
# Mark the filename as processed
processed_filenames.add(filename)
sql_file_a = os.path.join(folder_a, filename)
match_status = 'Not Matched'
delete_status = 'Not Deleted'
# Check if the file exists in folder_a
if os.path.exists(sql_file_a):
# Compare with folder_b if necessary
if os.path.exists(os.path.join(folder_b, filename)):
with open(sql_file_a, 'r') as f_a, open(os.path.join(folder_b, filename), 'r') as f_b:
lines_a = f_a.readlines()
lines_b = f_b.readlines()
if lines_a == lines_b:
match_status = 'Matched'
delete_status = 'Deleted'
# Delete file from folder_b
os.remove(os.path.join(folder_b, filename))
# Write to CSV
writer.writerow({'SQL File Name': filename, 'Occurrences': occurrences,
'Match Status': match_status, 'Delete Status': delete_status})
# Example usage
folder_a = '/path/to/folder/a'
folder_b = '/path/to/folder/b'
output_csv = 'comparison_results.csv'
yaml_file = 'example.yaml'
sql_filenames = extract_sql_filenames_from_yaml(yaml_file)
compare_and_delete_sql_files(folder_a, folder_b, output_csv, sql_filenames)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process:
The error indicated that file is open, so it can’t be deleted.
It is happening because you are deleting the file inside with open block of your code.
Execute this line os.remove(os.path.join(folder_b, filename)) outside of the open block.
Make sure folder_b and filename variables are available outside of the open block.
Some initial Questions:
- Should the file path have a forward slash in it (see file path reference below)?
- Do you have the file opened via a different thread or process (going to assume you don’t have it open on your PC)?
- Is the error on the first or last sql script that you are processing or was it able to process several sql scripts successfully but fail on that particular script file?
Let me know if you have additional info or logging built into the script operations.
File Path Reference:
C:UsersDesktophytapython_scriptspython_scr**/**folder_bscrum_1.sql
On windows, you cannot delete a file if it is open in any other process hence [WinError 32].[microsoft.com]
Even if you did not open the file in your own process, it might have been opened in another process hence os.remove() is not guaranteed to succeed on windows.
I’m your case, the file is open in your own process and you are attempting to delete the open file within the same context [python.org] hence the [WinError 32]. Close the file and then delete immediately to ensure that you are guaranteed to still be the permitted file holder and you are not violating any sharing policy. [1]
This is another one of the many reasons why windows is respectfully such a ”great” OS. Funny enough, there’s no such headache on Unix based OSes.
[1] I’ve had one such experience where another process takes hold of the same file right before you want to use it again. This led to a mind blowing bug.