Find size and free space of the filesystem containing a given file
Question:
I’m using Python 2.6 on Linux. What is the fastest way:
-
to determine which partition contains a given directory or file?
For example, suppose that /dev/sda2 is mounted on /home, and /dev/mapper/foo is mounted on /home/foo. From the string "/home/foo/bar/baz" I would like to recover the pair ("/dev/mapper/foo", "home/foo").
-
and then, to get usage statistics of the given partition? For example, given /dev/mapper/foo I would like to obtain the size of the partition and the free space available (either in bytes or approximately in megabytes).
Answers:
Usually the /proc directory contains such information in Linux, it is a virtual filesystem. For example, /proc/mounts gives information about current mounted disks; and you can parse it directly. Utilities like top, df all make use of /proc.
I haven’t used it, but this might help too, if you want a wrapper: http://bitbucket.org/chrismiles/psi/wiki/Home
If you just need the free space on a device, see the answer using os.statvfs() below.
If you also need the device name and mount point associated with the file, you should call an external program to get this information. df will provide all the information you need — when called as df filename it prints a line about the partition that contains the file.
To give an example:
import subprocess
df = subprocess.Popen(["df", "filename"], stdout=subprocess.PIPE)
output = df.communicate()[0]
device, size, used, available, percent, mountpoint =
output.split("n")[1].split()
Note that this is rather brittle, since it depends on the exact format of the df output, but I’m not aware of a more robust solution. (There are a few solutions relying on the /proc filesystem below that are even less portable than this one.)
For the first point, you can try using os.path.realpath to get a canonical path, check it against /etc/mtab (I’d actually suggest calling getmntent, but I can’t find a normal way to access it) to find the longest match. (to be sure, you should probably stat both the file and the presumed mountpoint to verify that they are in fact on the same device)
For the second point, use os.statvfs to get block size and usage information.
(Disclaimer: I have tested none of this, most of what I know came from the coreutils sources)
import os
def get_mount_point(pathname):
"Get the mount point of the filesystem containing pathname"
pathname= os.path.normcase(os.path.realpath(pathname))
parent_device= path_device= os.stat(pathname).st_dev
while parent_device == path_device:
mount_point= pathname
pathname= os.path.dirname(pathname)
if pathname == mount_point: break
parent_device= os.stat(pathname).st_dev
return mount_point
def get_mounted_device(pathname):
"Get the device mounted at pathname"
# uses "/proc/mounts"
pathname= os.path.normcase(pathname) # might be unnecessary here
try:
with open("/proc/mounts", "r") as ifp:
for line in ifp:
fields= line.rstrip('n').split()
# note that line above assumes that
# no mount points contain whitespace
if fields[1] == pathname:
return fields[0]
except EnvironmentError:
pass
return None # explicit
def get_fs_freespace(pathname):
"Get the free space of the filesystem containing pathname"
stat= os.statvfs(pathname)
# use f_bfree for superuser, or f_bavail if filesystem
# has reserved space for superuser
return stat.f_bfree*stat.f_bsize
Some sample pathnames on my computer:
path 'trash':
mp /home /dev/sda4
free 6413754368
path 'smov':
mp /mnt/S /dev/sde
free 86761562112
path '/usr/local/lib':
mp / rootfs
free 2184364032
path '/proc/self/cmdline':
mp /proc proc
free 0
PS
if on Python ≥3.3, there’s shutil.disk_usage(path) which returns a named tuple of (total, used, free) expressed in bytes.
This should make everything you asked:
import os
from collections import namedtuple
disk_ntuple = namedtuple('partition', 'device mountpoint fstype')
usage_ntuple = namedtuple('usage', 'total used free percent')
def disk_partitions(all=False):
"""Return all mountd partitions as a nameduple.
If all == False return phyisical partitions only.
"""
phydevs = []
f = open("/proc/filesystems", "r")
for line in f:
if not line.startswith("nodev"):
phydevs.append(line.strip())
retlist = []
f = open('/etc/mtab', "r")
for line in f:
if not all and line.startswith('none'):
continue
fields = line.split()
device = fields[0]
mountpoint = fields[1]
fstype = fields[2]
if not all and fstype not in phydevs:
continue
if device == 'none':
device = ''
ntuple = disk_ntuple(device, mountpoint, fstype)
retlist.append(ntuple)
return retlist
def disk_usage(path):
"""Return disk usage associated with path."""
st = os.statvfs(path)
free = (st.f_bavail * st.f_frsize)
total = (st.f_blocks * st.f_frsize)
used = (st.f_blocks - st.f_bfree) * st.f_frsize
try:
percent = ret = (float(used) / total) * 100
except ZeroDivisionError:
percent = 0
# NB: the percentage is -5% than what shown by df due to
# reserved blocks that we are currently not considering:
# http://goo.gl/sWGbH
return usage_ntuple(total, used, free, round(percent, 1))
if __name__ == '__main__':
for part in disk_partitions():
print part
print " %sn" % str(disk_usage(part.mountpoint))
On my box the code above prints:
giampaolo@ubuntu:~/dev$ python foo.py
partition(device='/dev/sda3', mountpoint='/', fstype='ext4')
usage(total=21378641920, used=4886749184, free=15405903872, percent=22.9)
partition(device='/dev/sda7', mountpoint='/home', fstype='ext4')
usage(total=30227386368, used=12137168896, free=16554737664, percent=40.2)
partition(device='/dev/sdb1', mountpoint='/media/1CA0-065B', fstype='vfat')
usage(total=7952400384, used=32768, free=7952367616, percent=0.0)
partition(device='/dev/sr0', mountpoint='/media/WB2PFRE_IT', fstype='iso9660')
usage(total=695730176, used=695730176, free=0, percent=100.0)
partition(device='/dev/sda6', mountpoint='/media/Dati', fstype='fuseblk')
usage(total=914217758720, used=614345637888, free=299872120832, percent=67.2)
This doesn’t give the name of the partition, but you can get the filesystem statistics directly using the statvfs Unix system call. To call it from Python, use os.statvfs('/home/foo/bar/baz').
The relevant fields in the result, according to POSIX:
unsigned long f_frsize Fundamental file system block size.
fsblkcnt_t f_blocks Total number of blocks on file system in units of f_frsize.
fsblkcnt_t f_bfree Total number of free blocks.
fsblkcnt_t f_bavail Number of free blocks available to
non-privileged process.
So to make sense of the values, multiply by f_frsize:
import os
statvfs = os.statvfs('/home/foo/bar/baz')
statvfs.f_frsize * statvfs.f_blocks # Size of filesystem in bytes
statvfs.f_frsize * statvfs.f_bfree # Actual number of free bytes
statvfs.f_frsize * statvfs.f_bavail # Number of free bytes that ordinary users
# are allowed to use (excl. reserved space)
The simplest way to find out it.
import os
from collections import namedtuple
DiskUsage = namedtuple('DiskUsage', 'total used free')
def disk_usage(path):
"""Return disk usage statistics about the given path.
Will return the namedtuple with attributes: 'total', 'used' and 'free',
which are the amount of total, used and free space, in bytes.
"""
st = os.statvfs(path)
free = st.f_bavail * st.f_frsize
total = st.f_blocks * st.f_frsize
used = (st.f_blocks - st.f_bfree) * st.f_frsize
return DiskUsage(total, used, free)
As of Python 3.3, there an easy and direct way to do this with the standard library:
$ cat free_space.py
#!/usr/bin/env python3
import shutil
total, used, free = shutil.disk_usage(__file__)
print(total, used, free)
$ ./free_space.py
1007870246912 460794834944 495854989312
These numbers are in bytes. See the documentation for more info.
import os
def disk_stat(path):
disk = os.statvfs(path)
percent = (disk.f_blocks - disk.f_bfree) * 100 / (disk.f_blocks -disk.f_bfree + disk.f_bavail) + 1
return percent
print disk_stat('/')
print disk_stat('/data')
For the second part of your question, “get usage statistics of the given partition”, psutil makes this easy with the disk_usage(path) function. Given a path, disk_usage() returns a named tuple including total, used, and free space expressed in bytes, plus the percentage usage.
Simple example from documentation:
>>> import psutil
>>> psutil.disk_usage('/')
sdiskusage(total=21378641920, used=4809781248, free=15482871808, percent=22.5)
Psutil works with Python versions from 2.6 to 3.6 and on Linux, Windows, and OSX among other platforms.
Checking the disk usage on your Windows PC can be done as follows:
import psutil
fan = psutil.disk_usage(path="C:/")
print("Available: ", fan.total/1000000000)
print("Used: ", fan.used/1000000000)
print("Free: ", fan.free/1000000000)
print("Percentage Used: ", fan.percent, "%")
11 years later but expanding on others answers.
import psutil
#File systems
value=psutil.disk_partitions()
for i in value:
va=i[1]
value2=psutil.disk_usage(va).percent
print(value2)
fs_space[va]=value2
This is adding it to a dictionary, only grabbing percent as that is what I need, but you can grab all values or select the one you want from the total, used, free, or percent.
I’m using Python 2.6 on Linux. What is the fastest way:
-
to determine which partition contains a given directory or file?
For example, suppose that
/dev/sda2is mounted on/home, and/dev/mapper/foois mounted on/home/foo. From the string"/home/foo/bar/baz"I would like to recover the pair("/dev/mapper/foo", "home/foo"). -
and then, to get usage statistics of the given partition? For example, given
/dev/mapper/fooI would like to obtain the size of the partition and the free space available (either in bytes or approximately in megabytes).
Usually the /proc directory contains such information in Linux, it is a virtual filesystem. For example, /proc/mounts gives information about current mounted disks; and you can parse it directly. Utilities like top, df all make use of /proc.
I haven’t used it, but this might help too, if you want a wrapper: http://bitbucket.org/chrismiles/psi/wiki/Home
If you just need the free space on a device, see the answer using os.statvfs() below.
If you also need the device name and mount point associated with the file, you should call an external program to get this information. df will provide all the information you need — when called as df filename it prints a line about the partition that contains the file.
To give an example:
import subprocess
df = subprocess.Popen(["df", "filename"], stdout=subprocess.PIPE)
output = df.communicate()[0]
device, size, used, available, percent, mountpoint =
output.split("n")[1].split()
Note that this is rather brittle, since it depends on the exact format of the df output, but I’m not aware of a more robust solution. (There are a few solutions relying on the /proc filesystem below that are even less portable than this one.)
For the first point, you can try using os.path.realpath to get a canonical path, check it against /etc/mtab (I’d actually suggest calling getmntent, but I can’t find a normal way to access it) to find the longest match. (to be sure, you should probably stat both the file and the presumed mountpoint to verify that they are in fact on the same device)
For the second point, use os.statvfs to get block size and usage information.
(Disclaimer: I have tested none of this, most of what I know came from the coreutils sources)
import os
def get_mount_point(pathname):
"Get the mount point of the filesystem containing pathname"
pathname= os.path.normcase(os.path.realpath(pathname))
parent_device= path_device= os.stat(pathname).st_dev
while parent_device == path_device:
mount_point= pathname
pathname= os.path.dirname(pathname)
if pathname == mount_point: break
parent_device= os.stat(pathname).st_dev
return mount_point
def get_mounted_device(pathname):
"Get the device mounted at pathname"
# uses "/proc/mounts"
pathname= os.path.normcase(pathname) # might be unnecessary here
try:
with open("/proc/mounts", "r") as ifp:
for line in ifp:
fields= line.rstrip('n').split()
# note that line above assumes that
# no mount points contain whitespace
if fields[1] == pathname:
return fields[0]
except EnvironmentError:
pass
return None # explicit
def get_fs_freespace(pathname):
"Get the free space of the filesystem containing pathname"
stat= os.statvfs(pathname)
# use f_bfree for superuser, or f_bavail if filesystem
# has reserved space for superuser
return stat.f_bfree*stat.f_bsize
Some sample pathnames on my computer:
path 'trash':
mp /home /dev/sda4
free 6413754368
path 'smov':
mp /mnt/S /dev/sde
free 86761562112
path '/usr/local/lib':
mp / rootfs
free 2184364032
path '/proc/self/cmdline':
mp /proc proc
free 0
PS
if on Python ≥3.3, there’s shutil.disk_usage(path) which returns a named tuple of (total, used, free) expressed in bytes.
This should make everything you asked:
import os
from collections import namedtuple
disk_ntuple = namedtuple('partition', 'device mountpoint fstype')
usage_ntuple = namedtuple('usage', 'total used free percent')
def disk_partitions(all=False):
"""Return all mountd partitions as a nameduple.
If all == False return phyisical partitions only.
"""
phydevs = []
f = open("/proc/filesystems", "r")
for line in f:
if not line.startswith("nodev"):
phydevs.append(line.strip())
retlist = []
f = open('/etc/mtab', "r")
for line in f:
if not all and line.startswith('none'):
continue
fields = line.split()
device = fields[0]
mountpoint = fields[1]
fstype = fields[2]
if not all and fstype not in phydevs:
continue
if device == 'none':
device = ''
ntuple = disk_ntuple(device, mountpoint, fstype)
retlist.append(ntuple)
return retlist
def disk_usage(path):
"""Return disk usage associated with path."""
st = os.statvfs(path)
free = (st.f_bavail * st.f_frsize)
total = (st.f_blocks * st.f_frsize)
used = (st.f_blocks - st.f_bfree) * st.f_frsize
try:
percent = ret = (float(used) / total) * 100
except ZeroDivisionError:
percent = 0
# NB: the percentage is -5% than what shown by df due to
# reserved blocks that we are currently not considering:
# http://goo.gl/sWGbH
return usage_ntuple(total, used, free, round(percent, 1))
if __name__ == '__main__':
for part in disk_partitions():
print part
print " %sn" % str(disk_usage(part.mountpoint))
On my box the code above prints:
giampaolo@ubuntu:~/dev$ python foo.py
partition(device='/dev/sda3', mountpoint='/', fstype='ext4')
usage(total=21378641920, used=4886749184, free=15405903872, percent=22.9)
partition(device='/dev/sda7', mountpoint='/home', fstype='ext4')
usage(total=30227386368, used=12137168896, free=16554737664, percent=40.2)
partition(device='/dev/sdb1', mountpoint='/media/1CA0-065B', fstype='vfat')
usage(total=7952400384, used=32768, free=7952367616, percent=0.0)
partition(device='/dev/sr0', mountpoint='/media/WB2PFRE_IT', fstype='iso9660')
usage(total=695730176, used=695730176, free=0, percent=100.0)
partition(device='/dev/sda6', mountpoint='/media/Dati', fstype='fuseblk')
usage(total=914217758720, used=614345637888, free=299872120832, percent=67.2)
This doesn’t give the name of the partition, but you can get the filesystem statistics directly using the statvfs Unix system call. To call it from Python, use os.statvfs('/home/foo/bar/baz').
The relevant fields in the result, according to POSIX:
unsigned long f_frsize Fundamental file system block size. fsblkcnt_t f_blocks Total number of blocks on file system in units of f_frsize. fsblkcnt_t f_bfree Total number of free blocks. fsblkcnt_t f_bavail Number of free blocks available to non-privileged process.
So to make sense of the values, multiply by f_frsize:
import os
statvfs = os.statvfs('/home/foo/bar/baz')
statvfs.f_frsize * statvfs.f_blocks # Size of filesystem in bytes
statvfs.f_frsize * statvfs.f_bfree # Actual number of free bytes
statvfs.f_frsize * statvfs.f_bavail # Number of free bytes that ordinary users
# are allowed to use (excl. reserved space)
The simplest way to find out it.
import os
from collections import namedtuple
DiskUsage = namedtuple('DiskUsage', 'total used free')
def disk_usage(path):
"""Return disk usage statistics about the given path.
Will return the namedtuple with attributes: 'total', 'used' and 'free',
which are the amount of total, used and free space, in bytes.
"""
st = os.statvfs(path)
free = st.f_bavail * st.f_frsize
total = st.f_blocks * st.f_frsize
used = (st.f_blocks - st.f_bfree) * st.f_frsize
return DiskUsage(total, used, free)
As of Python 3.3, there an easy and direct way to do this with the standard library:
$ cat free_space.py
#!/usr/bin/env python3
import shutil
total, used, free = shutil.disk_usage(__file__)
print(total, used, free)
$ ./free_space.py
1007870246912 460794834944 495854989312
These numbers are in bytes. See the documentation for more info.
import os
def disk_stat(path):
disk = os.statvfs(path)
percent = (disk.f_blocks - disk.f_bfree) * 100 / (disk.f_blocks -disk.f_bfree + disk.f_bavail) + 1
return percent
print disk_stat('/')
print disk_stat('/data')
For the second part of your question, “get usage statistics of the given partition”, psutil makes this easy with the disk_usage(path) function. Given a path, disk_usage() returns a named tuple including total, used, and free space expressed in bytes, plus the percentage usage.
Simple example from documentation:
>>> import psutil
>>> psutil.disk_usage('/')
sdiskusage(total=21378641920, used=4809781248, free=15482871808, percent=22.5)
Psutil works with Python versions from 2.6 to 3.6 and on Linux, Windows, and OSX among other platforms.
Checking the disk usage on your Windows PC can be done as follows:
import psutil
fan = psutil.disk_usage(path="C:/")
print("Available: ", fan.total/1000000000)
print("Used: ", fan.used/1000000000)
print("Free: ", fan.free/1000000000)
print("Percentage Used: ", fan.percent, "%")
11 years later but expanding on others answers.
import psutil
#File systems
value=psutil.disk_partitions()
for i in value:
va=i[1]
value2=psutil.disk_usage(va).percent
print(value2)
fs_space[va]=value2
This is adding it to a dictionary, only grabbing percent as that is what I need, but you can grab all values or select the one you want from the total, used, free, or percent.