String interpolation failing inside os.system call
Question:
Editing to skip to the answer: The problem was that the string I was interpolating had a non-ascii character in it that was invisible to the naked eye.
I’m trying to run this ImageMagick Command:
convert -verbose -size 732x142 xc:transparent -fill black -stroke black -strokewidth 2 -draw "roundrectangle 0,0 732,142 18,18" "Sample Card Title.png"
And it runs without error if I run it directly from the command line.
Only I’m trying to run it inside of a Python script, with most of those values being generated by the script. And when I launch it from there, with the same values, ImageMagick errors, reporting:
convert: non-conforming drawing primitive definition `roundrectangle' @ error/draw.c/RenderMVGContent/4506.
When it’s run as:
rectangle_coordinates = f""roundrectangle 0,0 {mm_to_px(62)},{mm_to_px(box_height)} {mm_to_px(1.5)},{mm_to_px(1.5)}""
command = f"convert -verbose -size {dimensions} xc:transparent -fill black -stroke black -strokewidth 2 -draw {rectangle_coordinates} "{filename}""
print(command)
os.system(command)
If I remove the variable "rectangle_coordinates" and replace it with hardcode, then it works:
command = f"convert -size {dimensions} xc:transparent -fill black -stroke black -strokewidth 2 -draw "roundrectangle 0,0 732,142 18,18" "{filename}""
print(command)
os.system(command)
If I literally copy-paste the results of that print(command) line from the output to the prompt and run it, it runs without error. But when run with os.system, the command it’s calling errors.
Anyone see why the command would fail with that one part interpolated, but run fine when it’s hardcoded?
P.S. Yes, I know that subprocess.run() is preferred over os.system(). I’ve tried so many variations of this command during the debugging, splitting up the arguments into a list for each different thing I tried was just adding one more source of (human) errors. I went back to os.system while I hunt the bug. Once I get the bug hammered out, I’ll be happy to go back to subprocess.run().
Running the script call as…
subprocess.run(["convert", "-size", dimensions, "xc:transparent", "-fill", "black", "-stroke", "black", "-strokewidth", "2", "-draw", rectangle_coordinates, ""{resisted_consequence_filename}""])
…produces the same (or more) error(s) than the os.system version.
P.P.S. Yes, I’m also aware there are multiple ImageMagick wrappers for Python. I prefer to not add that additional layer of abstraction, especially when bug-hunting.
Answers:
In the subprocess.call() arguments you shouldn’t add literal quotes in the last argument. Quoting is only needed in the shell to combine multiple words into a single word, but that’s already accomplished by making them separate list elements.
subprocess.run(["convert", "-size", dimensions, "xc:transparent", "-fill", "black", "-stroke", "black", "-strokewidth", "2", "-draw", rectangle_coordinates, resisted_consequence_filename])
I found the answer. I had to walk away for a couple days and come back.
In fact, you can’t see it on Stack Overflow for the same reason I had such trouble tracking it down:
My code had a non-breaking space in the definition of the problem variable, before it was interpolated.
That is, in the line:
rectangle_coordinates = f""roundrectangle 0,0 {mm_to_px(62)},{mm_to_px(box_height)} {mm_to_px(1.5)},{mm_to_px(1.5)}""
The final space wasn’t a normal space, ascii code 0x20.
It was a unicode non-breaking space, codepoint C2A0.
You literally can’t see the bug here on Stack Overflow because the same as printing to the terminal, when I copy-pasted it here, it was converted to a standard space, too. If I had literally back-copied the sample code I posted here to my original code, it would have overwritten the error.
10 Headdesk
20 Goto 10
I finally found the error in desperation by thinking, "What if I write the command to a file, then read it back from a file? Surely that will digest the string interpolation, forcing it to resolve the line the same as when I hardcode it." And when I first hardcoded a sample text file, of course that worked. But when I had the program write the text file and read it back, it failed again. But now I had my hardcoded version and the program written version that looked identical to the naked eye, but diffed as not being the same. And so I opened them in a hex editor and there it was: One character different, even though they print the same on screen.
Editing to skip to the answer: The problem was that the string I was interpolating had a non-ascii character in it that was invisible to the naked eye.
I’m trying to run this ImageMagick Command:
convert -verbose -size 732x142 xc:transparent -fill black -stroke black -strokewidth 2 -draw "roundrectangle 0,0 732,142 18,18" "Sample Card Title.png"
And it runs without error if I run it directly from the command line.
Only I’m trying to run it inside of a Python script, with most of those values being generated by the script. And when I launch it from there, with the same values, ImageMagick errors, reporting:
convert: non-conforming drawing primitive definition `roundrectangle' @ error/draw.c/RenderMVGContent/4506.
When it’s run as:
rectangle_coordinates = f""roundrectangle 0,0 {mm_to_px(62)},{mm_to_px(box_height)} {mm_to_px(1.5)},{mm_to_px(1.5)}""
command = f"convert -verbose -size {dimensions} xc:transparent -fill black -stroke black -strokewidth 2 -draw {rectangle_coordinates} "{filename}""
print(command)
os.system(command)
If I remove the variable "rectangle_coordinates" and replace it with hardcode, then it works:
command = f"convert -size {dimensions} xc:transparent -fill black -stroke black -strokewidth 2 -draw "roundrectangle 0,0 732,142 18,18" "{filename}""
print(command)
os.system(command)
If I literally copy-paste the results of that print(command) line from the output to the prompt and run it, it runs without error. But when run with os.system, the command it’s calling errors.
Anyone see why the command would fail with that one part interpolated, but run fine when it’s hardcoded?
P.S. Yes, I know that subprocess.run() is preferred over os.system(). I’ve tried so many variations of this command during the debugging, splitting up the arguments into a list for each different thing I tried was just adding one more source of (human) errors. I went back to os.system while I hunt the bug. Once I get the bug hammered out, I’ll be happy to go back to subprocess.run().
Running the script call as…
subprocess.run(["convert", "-size", dimensions, "xc:transparent", "-fill", "black", "-stroke", "black", "-strokewidth", "2", "-draw", rectangle_coordinates, ""{resisted_consequence_filename}""])
…produces the same (or more) error(s) than the os.system version.
P.P.S. Yes, I’m also aware there are multiple ImageMagick wrappers for Python. I prefer to not add that additional layer of abstraction, especially when bug-hunting.
In the subprocess.call() arguments you shouldn’t add literal quotes in the last argument. Quoting is only needed in the shell to combine multiple words into a single word, but that’s already accomplished by making them separate list elements.
subprocess.run(["convert", "-size", dimensions, "xc:transparent", "-fill", "black", "-stroke", "black", "-strokewidth", "2", "-draw", rectangle_coordinates, resisted_consequence_filename])
I found the answer. I had to walk away for a couple days and come back.
In fact, you can’t see it on Stack Overflow for the same reason I had such trouble tracking it down:
My code had a non-breaking space in the definition of the problem variable, before it was interpolated.
That is, in the line:
rectangle_coordinates = f""roundrectangle 0,0 {mm_to_px(62)},{mm_to_px(box_height)} {mm_to_px(1.5)},{mm_to_px(1.5)}""
The final space wasn’t a normal space, ascii code 0x20.
It was a unicode non-breaking space, codepoint C2A0.
You literally can’t see the bug here on Stack Overflow because the same as printing to the terminal, when I copy-pasted it here, it was converted to a standard space, too. If I had literally back-copied the sample code I posted here to my original code, it would have overwritten the error.
10 Headdesk
20 Goto 10
I finally found the error in desperation by thinking, "What if I write the command to a file, then read it back from a file? Surely that will digest the string interpolation, forcing it to resolve the line the same as when I hardcode it." And when I first hardcoded a sample text file, of course that worked. But when I had the program write the text file and read it back, it failed again. But now I had my hardcoded version and the program written version that looked identical to the naked eye, but diffed as not being the same. And so I opened them in a hex editor and there it was: One character different, even though they print the same on screen.