taskset – python
Question:
I have a dual quad core machine. so, the cpu list for me 0-7.
I am trying to run the taskset from python
mapping = [2,2,2,2,2]
for i in range(0,len(mapping)):
cmd = "taskset -c" + str(mapping[r]) + "python <path>/run-apps.py" + thr[r] + "&"
os.system(cmd)
and it says:
taskset: invalid option -- '2'
taskset (util-linux-ng 2.17.2)
usage: taskset [options] [mask | cpu-list] [pid | cmd [args...]]
set or get the affinity of a process
-p, --pid operate on existing given pid
-c, --cpu-list display and specify cpus in list format
-h, --help display this help
-V, --version output version information
The default behavior is to run a new command:
taskset 03 sshd -b 1024
You can retrieve the mask of an existing task:
taskset -p 700
Or set it:
taskset -p 03 700
List format uses a comma-separated list instead of a mask:
taskset -pc 0,3,7-11 700
Ranges in list format can take a stride argument:
e.g. 0-31:2 is equivalent to mask 0x55555555
But the core 2 is available and I am to run the same thing from commandline
.
taskset -c 2 python <path>/run-apps.py lbm &
not a clue what the issue is..
any hints?
Answers:
compared to the commandline you posted, you’re missing a couple spaces … e.g.:
cmd = "taskset -c " + str(mapping[r]) + " python <path>/run-apps.py " + thr[r] + " &"
In your code, when parsing the “commandline”, taskset
is seeing the string -c2
which according to many commandline parsing libraries is the same thing as -c -2
which would explain the error that you’re seeing.
Sometimes these things are easier to read if you use string interpolation instead:
cmd = "taskset -c %s python <path>/run-apps.py %s &" % (mapping[r],thr[r])
Or new style .format
:
cmd = "taskset -c {0} python <path>/run-apps.py {1} &".format(mapping[r],thr[r])
And finally, no solution using os.system
should go by without at least a mention of the python subprocess
module.
process = subprocess.Popen(['taskset',
'-c',
str(mapping[r]),
'python',
'<path>/run-apps.py',
str(thr[r]) ] )
It’ll avoid the shell altogether which is slightly more efficient and makes you safer from shell injection types of attacks.
You are missing spaces. To help yourself try the following:
- Comment out the os.system() line
-
Below the os.system() line, add:
print cmd
You will see what your command line looks like. Add spaces where needed. Then uncomment the os.system() line.
You can avoid calling taskset and use psutil instead:
https://pythonhosted.org/psutil/#psutil.Process.cpu_affinity
>>> import psutil
>>> psutil.cpu_count()
4
>>> p = psutil.Process()
>>> p.cpu_affinity() # get
[0, 1, 2, 3]
>>> p.cpu_affinity([0]) # set; from now on, process will run on CPU #0 only
>>> p.cpu_affinity()
[0]
>>>
>>> # reset affinity against all CPUs
>>> all_cpus = list(range(psutil.cpu_count()))
>>> p.cpu_affinity(all_cpus)
>>>
As of Python 3.3, sched_setaffinity
and friends are part of the os
module.
I have a dual quad core machine. so, the cpu list for me 0-7.
I am trying to run the taskset from python
mapping = [2,2,2,2,2]
for i in range(0,len(mapping)):
cmd = "taskset -c" + str(mapping[r]) + "python <path>/run-apps.py" + thr[r] + "&"
os.system(cmd)
and it says:
taskset: invalid option -- '2'
taskset (util-linux-ng 2.17.2)
usage: taskset [options] [mask | cpu-list] [pid | cmd [args...]]
set or get the affinity of a process
-p, --pid operate on existing given pid
-c, --cpu-list display and specify cpus in list format
-h, --help display this help
-V, --version output version information
The default behavior is to run a new command:
taskset 03 sshd -b 1024
You can retrieve the mask of an existing task:
taskset -p 700
Or set it:
taskset -p 03 700
List format uses a comma-separated list instead of a mask:
taskset -pc 0,3,7-11 700
Ranges in list format can take a stride argument:
e.g. 0-31:2 is equivalent to mask 0x55555555
But the core 2 is available and I am to run the same thing from commandline
.
taskset -c 2 python <path>/run-apps.py lbm &
not a clue what the issue is..
any hints?
compared to the commandline you posted, you’re missing a couple spaces … e.g.:
cmd = "taskset -c " + str(mapping[r]) + " python <path>/run-apps.py " + thr[r] + " &"
In your code, when parsing the “commandline”, taskset
is seeing the string -c2
which according to many commandline parsing libraries is the same thing as -c -2
which would explain the error that you’re seeing.
Sometimes these things are easier to read if you use string interpolation instead:
cmd = "taskset -c %s python <path>/run-apps.py %s &" % (mapping[r],thr[r])
Or new style .format
:
cmd = "taskset -c {0} python <path>/run-apps.py {1} &".format(mapping[r],thr[r])
And finally, no solution using os.system
should go by without at least a mention of the python subprocess
module.
process = subprocess.Popen(['taskset',
'-c',
str(mapping[r]),
'python',
'<path>/run-apps.py',
str(thr[r]) ] )
It’ll avoid the shell altogether which is slightly more efficient and makes you safer from shell injection types of attacks.
You are missing spaces. To help yourself try the following:
- Comment out the os.system() line
-
Below the os.system() line, add:
print cmd
You will see what your command line looks like. Add spaces where needed. Then uncomment the os.system() line.
You can avoid calling taskset and use psutil instead:
https://pythonhosted.org/psutil/#psutil.Process.cpu_affinity
>>> import psutil
>>> psutil.cpu_count()
4
>>> p = psutil.Process()
>>> p.cpu_affinity() # get
[0, 1, 2, 3]
>>> p.cpu_affinity([0]) # set; from now on, process will run on CPU #0 only
>>> p.cpu_affinity()
[0]
>>>
>>> # reset affinity against all CPUs
>>> all_cpus = list(range(psutil.cpu_count()))
>>> p.cpu_affinity(all_cpus)
>>>
As of Python 3.3, sched_setaffinity
and friends are part of the os
module.