lmdb.Error: There is not enough space on the disk
Question:
I just encountered below error in python3 that seems related in lmdb package and my disk or ram space.
My ram is 8GB and HDD is 900GB. it seemed that 1099511627776 byte is about 1TB.
It is ram error or disk error? And I need 1099511627776 byte space in my HDD to solve below error?
env = lmdb.open(outputPath, map_size=1099511627776)
lmdb.Error: D:ocr_kor-masterocr_kor-masterdeep-text-recognition-benchmarkdata_lmdb_releasetraining;: There is not enough space on the disk.
Answers:
Changing:
env = lmdb.open(outputPath, map_size=1099511627776)
to
env = lmdb.open(outputPath, map_size=1073741824)
worked for me
I had this same problem because I ran out of open file descriptors.
You can increase the number of allowed file descriptors though it is not typically a good solution to your problem as noted by @osbor in this answer.
To view and then increase the limit you can use ulimit -a and set the limit with ulimit -u <number>
ulimit -a # show limits
ulimit -u 2048 # update file descriptor limit
See this question for more information: Is there a fix for the "Too many open files in system" error on OS X 10.7.1?
Finally, I recommend finding out whether your app using lmdb is properly closing and releasing file descriptors. That would be the complete solution to your issue. Maybe you need to do graceful shutdown with SIGTERM rather than killing a process. This was my issue. I was killing processes without releasing file descriptors and eventually my quantity of available file descriptors was exhausted.
I just encountered below error in python3 that seems related in lmdb package and my disk or ram space.
My ram is 8GB and HDD is 900GB. it seemed that 1099511627776 byte is about 1TB.
It is ram error or disk error? And I need 1099511627776 byte space in my HDD to solve below error?
env = lmdb.open(outputPath, map_size=1099511627776)
lmdb.Error: D:ocr_kor-masterocr_kor-masterdeep-text-recognition-benchmarkdata_lmdb_releasetraining;: There is not enough space on the disk.
Changing:
env = lmdb.open(outputPath, map_size=1099511627776)
to
env = lmdb.open(outputPath, map_size=1073741824)
worked for me
I had this same problem because I ran out of open file descriptors.
You can increase the number of allowed file descriptors though it is not typically a good solution to your problem as noted by @osbor in this answer.
To view and then increase the limit you can use ulimit -a and set the limit with ulimit -u <number>
ulimit -a # show limits
ulimit -u 2048 # update file descriptor limit
See this question for more information: Is there a fix for the "Too many open files in system" error on OS X 10.7.1?
Finally, I recommend finding out whether your app using lmdb is properly closing and releasing file descriptors. That would be the complete solution to your issue. Maybe you need to do graceful shutdown with SIGTERM rather than killing a process. This was my issue. I was killing processes without releasing file descriptors and eventually my quantity of available file descriptors was exhausted.