How can I ignore or remove ".ipynb_checkpoints" in colab?
Question:
My code in tf.keras is given below.
I want to retrieve a file(Xscale.npy) in each sub_directory(component_0, component_1) of model_cnn folder.
root_dir = '/content/drive/My Drive/DeepCID/model_cnn'
i=0
for (root, dirs, files) in os.walk(root_dir):
for d in dirs:
print(dirs)
os.chdir(os.path.join(root, d))
print(os.getcwd())
datafile3 = './Xscale.npy'
Xscale = np.load(datafile3)
The error message is,
['.ipynb_checkpoints', 'component_0', 'component_1']
/content/drive/My Drive/DeepCID/model_cnn/.ipynb_checkpoints
---------------------------------------------------------------------------
FileNotFoundError Traceback (most recent call last)
<ipython-input-1-862f78aebef9> in <module>()
57 print(os.getcwd())
58 datafile3 = './Xscale.npy'
---> 59 Xscale = np.load(datafile3)
60 Xtest = (Xtest0 - Xscale[0])/Xscale[1]
61
/usr/local/lib/python3.6/dist-packages/numpy/lib/npyio.py in load(file, mmap_mode, allow_pickle, fix_imports, encoding)
426 own_fid = False
427 else:
--> 428 fid = open(os_fspath(file), "rb")
429 own_fid = True
430
FileNotFoundError: [Errno 2] No such file or directory: './Xscale.npy'
I recognize that ‘.ipynb_checkpoints’ is the problem.
But, when I look into the folder, there is no .ipynb_checkpoints file or folder.
My questions are
1) How can I ignore .ipynb_checkpoints when accessing a file in sub_directories?
2) Why is .ipynb_checkpoints file not visible in colab disk?
Thanks in advance,
D.-H.
Answers:
Change your code to the following.
1) Check if it is a hidden file
2) Not use os.chdir
as it is not necessary.
root_dir = '/content/drive/My Drive/DeepCID/model_cnn'
datafile3 = 'Xscale.npy'
i=0
for (root, dirs, files) in os.walk(root_dir):
for d in dirs:
if not d.startswith('.'):
dir_path = os.path.join(root, d)
file_path = os.path.join(dir_path, datafile3)
Xscale = np.load(file_path)
There are more elegant ways to do this in terms of getting absolute file paths but I wanted to minimize the amount of code changed.
An alternative method uses pathlib
.
from pathlib import Path
root_dir = '/content/drive/My Drive/DeepCID/model_cnn'
datafile3 = 'Xscale.npy'
i=0
for (root, dirs, files) in os.walk(root_dir):
for d in dirs:
if not d.startswith('.'):
fp = Path(root) / d / datafile3
Xscale = np.load(str(fp))
You can use pathlib
with rglob
for a cleaner code.
from pathlib import Path
root_dir = '/content/drive/My Drive/DeepCID/model_cnn'
root = Path(root_dir)
# parent dir must not start with dot
for datafile in root.rglob('[!.]*/Xscale.npy'):
print(datafile) # or np.load(datafile)
I was having the same issue. This might be happening because during the upload, it is also uploading the .ipynb_checkpoint
which is a hidden folder. A fix for this is that you can manually delete the folder using
rmdir /content/drive/My Drive/DeepCID/model_cnn/.ipynb_checkpoints
in a cell just before executing the code you mention.
You can try the following solution in this link.
Use the command –
rm -rf `find -type d -name .ipynb_checkpoints`
My code in tf.keras is given below.
I want to retrieve a file(Xscale.npy) in each sub_directory(component_0, component_1) of model_cnn folder.
root_dir = '/content/drive/My Drive/DeepCID/model_cnn'
i=0
for (root, dirs, files) in os.walk(root_dir):
for d in dirs:
print(dirs)
os.chdir(os.path.join(root, d))
print(os.getcwd())
datafile3 = './Xscale.npy'
Xscale = np.load(datafile3)
The error message is,
['.ipynb_checkpoints', 'component_0', 'component_1']
/content/drive/My Drive/DeepCID/model_cnn/.ipynb_checkpoints
---------------------------------------------------------------------------
FileNotFoundError Traceback (most recent call last)
<ipython-input-1-862f78aebef9> in <module>()
57 print(os.getcwd())
58 datafile3 = './Xscale.npy'
---> 59 Xscale = np.load(datafile3)
60 Xtest = (Xtest0 - Xscale[0])/Xscale[1]
61
/usr/local/lib/python3.6/dist-packages/numpy/lib/npyio.py in load(file, mmap_mode, allow_pickle, fix_imports, encoding)
426 own_fid = False
427 else:
--> 428 fid = open(os_fspath(file), "rb")
429 own_fid = True
430
FileNotFoundError: [Errno 2] No such file or directory: './Xscale.npy'
I recognize that ‘.ipynb_checkpoints’ is the problem.
But, when I look into the folder, there is no .ipynb_checkpoints file or folder.
My questions are
1) How can I ignore .ipynb_checkpoints when accessing a file in sub_directories?
2) Why is .ipynb_checkpoints file not visible in colab disk?
Thanks in advance,
D.-H.
Change your code to the following.
1) Check if it is a hidden file
2) Not use os.chdir
as it is not necessary.
root_dir = '/content/drive/My Drive/DeepCID/model_cnn'
datafile3 = 'Xscale.npy'
i=0
for (root, dirs, files) in os.walk(root_dir):
for d in dirs:
if not d.startswith('.'):
dir_path = os.path.join(root, d)
file_path = os.path.join(dir_path, datafile3)
Xscale = np.load(file_path)
There are more elegant ways to do this in terms of getting absolute file paths but I wanted to minimize the amount of code changed.
An alternative method uses pathlib
.
from pathlib import Path
root_dir = '/content/drive/My Drive/DeepCID/model_cnn'
datafile3 = 'Xscale.npy'
i=0
for (root, dirs, files) in os.walk(root_dir):
for d in dirs:
if not d.startswith('.'):
fp = Path(root) / d / datafile3
Xscale = np.load(str(fp))
You can use pathlib
with rglob
for a cleaner code.
from pathlib import Path
root_dir = '/content/drive/My Drive/DeepCID/model_cnn'
root = Path(root_dir)
# parent dir must not start with dot
for datafile in root.rglob('[!.]*/Xscale.npy'):
print(datafile) # or np.load(datafile)
I was having the same issue. This might be happening because during the upload, it is also uploading the .ipynb_checkpoint
which is a hidden folder. A fix for this is that you can manually delete the folder using
rmdir /content/drive/My Drive/DeepCID/model_cnn/.ipynb_checkpoints
in a cell just before executing the code you mention.
You can try the following solution in this link.
Use the command –
rm -rf `find -type d -name .ipynb_checkpoints`