How to postpone/defer the evaluation of f-strings?
Question:
I am using template strings to generate some files and I love the conciseness of the new f-strings for this purpose, for reducing my previous template code from something like this:
template_a = "The current name is {name}"
names = ["foo", "bar"]
for name in names:
print (template_a.format(**locals()))
Now I can do this, directly replacing variables:
names = ["foo", "bar"]
for name in names:
print (f"The current name is {name}")
However, sometimes it makes sense to have the template defined elsewhere — higher up in the code, or imported from a file or something. This means the template is a static string with formatting tags in it. Something would have to happen to the string to tell the interpreter to interpret the string as a new f-string, but I don’t know if there is such a thing.
Is there any way to bring in a string and have it interpreted as an f-string to avoid using the .format(**locals()) call?
Ideally I want to be able to code like this… (where magic_fstring_function is where the part I don’t understand comes in):
template_a = f"The current name is {name}"
# OR [Ideal2] template_a = magic_fstring_function(open('template.txt').read())
names = ["foo", "bar"]
for name in names:
print (template_a)
…with this desired output (without reading the file twice):
The current name is foo
The current name is bar
…but the actual output I get is:
The current name is {name}
The current name is {name}
Answers:
Here’s a complete “Ideal 2”.
It’s not an f-string—it doesn’t even use f-strings—but it does as requested. Syntax exactly as specified. No security headaches since we are not using eval().
It uses a little class and implements __str__ which is automatically called by print. To escape the limited scope of the class we use the inspect module to hop one frame up and see the variables the caller has access to.
import inspect
class magic_fstring_function:
def __init__(self, payload):
self.payload = payload
def __str__(self):
vars = inspect.currentframe().f_back.f_globals.copy()
vars.update(inspect.currentframe().f_back.f_locals)
return self.payload.format(**vars)
template = "The current name is {name}"
template_a = magic_fstring_function(template)
# use it inside a function to demonstrate it gets the scoping right
def new_scope():
names = ["foo", "bar"]
for name in names:
print(template_a)
new_scope()
# The current name is foo
# The current name is bar
An f-string is simply a more concise way of creating a formatted string, replacing .format(**names) with f. If you don’t want a string to be immediately evaluated in such a manner, don’t make it an f-string. Save it as an ordinary string literal, and then call format on it later when you want to perform the interpolation, as you have been doing.
Of course, there is an alternative with eval.
template.txt:
f’The current name is {name}’
Code:
>>> template_a = open('template.txt').read()
>>> names = 'foo', 'bar'
>>> for name in names:
... print(eval(template_a))
...
The current name is foo
The current name is bar
But then all you’ve managed to do is replace str.format with eval, which is surely not worth it. Just keep using regular strings with a format call.
This means the template is a static string with formatting tags in it
Yes, that’s exactly why we have literals with replacement fields and .format, so we can replace the fields whenever we like by calling format on it.
Something would have to happen to the string to tell the interpreter to interpret the string as a new f-string
That’s the prefix f/F. You could wrap it in a function and postpone the evaluation during call time but of course that incurs extra overhead:
def template_a():
return f"The current name is {name}"
names = ["foo", "bar"]
for name in names:
print(template_a())
Which prints out:
The current name is foo
The current name is bar
but feels wrong and is limited by the fact that you can only peek at the global namespace in your replacements. Trying to use it in a situation which requires local names will fail miserably unless passed to the string as arguments (which totally beats the point).
Is there any way to bring in a string and have it interpreted as an f-string to avoid using the .format(**locals()) call?
Other than a function (limitations included), nope, so might as well stick with .format.
Or maybe do not use f-strings, just format:
fun = "The curent name is {name}".format
names = ["foo", "bar"]
for name in names:
print(fun(name=name))
In version without names:
fun = "The curent name is {}".format
names = ["foo", "bar"]
for name in names:
print(fun(name))
Using .format is not a correct answer to this question. Python f-strings are very different from str.format() templates … they can contain code or other expensive operations – hence the need for deferral.
Here’s an example of a deferred logger. This uses the normal preamble of logging.getLogger, but then adds new functions that interpret the f-string only if the log level is correct.
log = logging.getLogger(__name__)
def __deferred_flog(log, fstr, level, *args):
if log.isEnabledFor(level):
import inspect
frame = inspect.currentframe().f_back.f_back
try:
fstr = 'f"' + fstr + '"'
log.log(level, eval(fstr, frame.f_globals, frame.f_locals))
finally:
del frame
log.fdebug = lambda fstr, *args: __deferred_flog(log, fstr, logging.DEBUG, *args)
log.finfo = lambda fstr, *args: __deferred_flog(log, fstr, logging.INFO, *args)
This has the advantage of being able to do things like: log.fdebug("{obj.dump()}") …. without dumping the object unless debugging is enabled.
IMHO: This should have been the default operation of f-strings, however now it’s too late. F-string evaluation can have massive and unintended side-effects, and having that happen in a deferred manner will change program execution.
In order to make f-strings properly deferred, python would need some way of explicitly switching behavior. Maybe use the letter ‘g’? 😉
It has been pointed out that deferred logging shouldn’t crash if there’s a bug in the string converter. The above solution can do this as well, change the finally: to except:, and stick a log.exception in there.
A suggestion that uses f-strings. Do your evaluation on the
logical level where the templating is occurring and pass it as a generator.
You can unwind it at whatever point you choose, using f-strings
In [46]: names = (i for i in ('The CIO, Reed', 'The homeless guy, Arnot', 'The security guard Spencer'))
In [47]: po = (f'Strangely, {next(names)} has a nice {i}' for i in (" nice house", " fast car", " big boat"))
In [48]: while True:
...: try:
...: print(next(po))
...: except StopIteration:
...: break
...:
Strangely, The CIO, Reed has a nice nice house
Strangely, The homeless guy, Arnot has a nice fast car
Strangely, The security guard Spencer has a nice big boat
A concise way to have a string evaluated as an f-string (with its full capabilities) is using following function:
def fstr(template):
return eval(f"f'{template}'")
Then you can do:
template_a = "The current name is {name}"
names = ["foo", "bar"]
for name in names:
print(fstr(template_a))
# The current name is foo
# The current name is bar
And, in contrast to many other proposed solutions, you can also do:
template_b = "The current name is {name.upper() * 2}"
for name in names:
print(fstr(template_b))
# The current name is FOOFOO
# The current name is BARBAR
inspired by the answer by kadee, the following can be used to define a deferred-f-string class.
class FStr:
def __init__(self, s):
self._s = s
def __repr__(self):
return eval(f"f'{self._s}'")
...
template_a = FStr('The current name is {name}')
names = ["foo", "bar"]
for name in names:
print (template_a)
which is exactly what the question asked for
What you want appears to be being considered as a Python enhancement.
Meanwhile — from the linked discussion — the following seems like it would be a reasonable workaround that doesn’t require using eval():
class FL:
def __init__(self, func):
self.func = func
def __str__(self):
return self.func()
template_a = FL(lambda: f"The current name, number is {name!r}, {number+1}")
names = "foo", "bar"
numbers = 40, 41
for name, number in zip(names, numbers):
print(template_a)
Output:
The current name, number is 'foo', 41
The current name, number is 'bar', 42
How about:
s = 'Hi, {foo}!'
s
> 'Hi, {foo}!'
s.format(foo='Bar')
> 'Hi, Bar!'
Most of these answers will get you something that behaves sort of like f-strings some of the time, but they will all go wrong in some cases.
There is a package on pypi f-yeah that does all this, only costing you two extra characters! (full disclosure, I am the author)
from fyeah import f
print(f("""'{'"all" the quotes'}'"""))
There are a lot of differences between f-strings and format calls, here is a probably incomplete list
- f-strings allow for arbitrary eval of python code
- f-strings cannot contain a backslash in the expression (since formatted strings don’t have an expression, so I suppose you could say this isn’t a difference, but it does differ from what a raw eval() can do)
- dict lookups in formatted strings must not be quoted. dict lookups in f-strings can be quoted, and so non-string keys can also be looked up
- f-strings have a debug format that format() does not:
f"The argument is {spam=}"
- f-string expressions cannot be empty
The suggestions to use eval will get you full f-string format support, but they don’t work on all string types.
def f_template(the_string):
return eval(f"f'{the_string}'")
print(f_template('some "quoted" string'))
print(f_template("some 'quoted' string"))
some "quoted" string
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in f_template
File "<string>", line 1
f'some 'quoted' string'
^
SyntaxError: invalid syntax
This example will also get variable scoping wrong in some cases.
There’s a lot of talk about using str.format(), but as noted it doesn’t allow most of the expressions that are allowed in f-strings such as arithmetic or slices. Using eval() obviously also has it’s downsides.
I’d recommend looking into a templating language such as Jinja. For my use-case it works quite well. See the example below where I have overridden the variable annotation syntax with a single curly brace to match the f-string syntax. I didn’t fully review the differences between f-strings and Jinja invoked like this.
from jinja2 import Environment, BaseLoader
a, b, c = 1, 2, "345"
templ = "{a or b}{c[1:]}"
env = Environment(loader=BaseLoader, variable_start_string="{", variable_end_string="}")
env.from_string(templ).render(**locals())
results in
'145'
to do that I prefer to use fstring inside a lambda function like:
s = lambda x: f'this is your string template to embed {x} in it.'
n = ['a' , 'b' , 'c']
for i in n:
print( s(i) )
You could use a .format styled replacement and explicitly define the replaced variable name:
template_a = "The current name is {name}"
names = ["foo", "bar"]
for name in names:
print (template_a.format(name=name))
Output
The current name is foo
The current name is bar
I am using template strings to generate some files and I love the conciseness of the new f-strings for this purpose, for reducing my previous template code from something like this:
template_a = "The current name is {name}"
names = ["foo", "bar"]
for name in names:
print (template_a.format(**locals()))
Now I can do this, directly replacing variables:
names = ["foo", "bar"]
for name in names:
print (f"The current name is {name}")
However, sometimes it makes sense to have the template defined elsewhere — higher up in the code, or imported from a file or something. This means the template is a static string with formatting tags in it. Something would have to happen to the string to tell the interpreter to interpret the string as a new f-string, but I don’t know if there is such a thing.
Is there any way to bring in a string and have it interpreted as an f-string to avoid using the .format(**locals()) call?
Ideally I want to be able to code like this… (where magic_fstring_function is where the part I don’t understand comes in):
template_a = f"The current name is {name}"
# OR [Ideal2] template_a = magic_fstring_function(open('template.txt').read())
names = ["foo", "bar"]
for name in names:
print (template_a)
…with this desired output (without reading the file twice):
The current name is foo
The current name is bar
…but the actual output I get is:
The current name is {name}
The current name is {name}
Here’s a complete “Ideal 2”.
It’s not an f-string—it doesn’t even use f-strings—but it does as requested. Syntax exactly as specified. No security headaches since we are not using eval().
It uses a little class and implements __str__ which is automatically called by print. To escape the limited scope of the class we use the inspect module to hop one frame up and see the variables the caller has access to.
import inspect
class magic_fstring_function:
def __init__(self, payload):
self.payload = payload
def __str__(self):
vars = inspect.currentframe().f_back.f_globals.copy()
vars.update(inspect.currentframe().f_back.f_locals)
return self.payload.format(**vars)
template = "The current name is {name}"
template_a = magic_fstring_function(template)
# use it inside a function to demonstrate it gets the scoping right
def new_scope():
names = ["foo", "bar"]
for name in names:
print(template_a)
new_scope()
# The current name is foo
# The current name is bar
An f-string is simply a more concise way of creating a formatted string, replacing .format(**names) with f. If you don’t want a string to be immediately evaluated in such a manner, don’t make it an f-string. Save it as an ordinary string literal, and then call format on it later when you want to perform the interpolation, as you have been doing.
Of course, there is an alternative with eval.
template.txt:
f’The current name is {name}’
Code:
>>> template_a = open('template.txt').read()
>>> names = 'foo', 'bar'
>>> for name in names:
... print(eval(template_a))
...
The current name is foo
The current name is bar
But then all you’ve managed to do is replace str.format with eval, which is surely not worth it. Just keep using regular strings with a format call.
This means the template is a static string with formatting tags in it
Yes, that’s exactly why we have literals with replacement fields and .format, so we can replace the fields whenever we like by calling format on it.
Something would have to happen to the string to tell the interpreter to interpret the string as a new f-string
That’s the prefix f/F. You could wrap it in a function and postpone the evaluation during call time but of course that incurs extra overhead:
def template_a():
return f"The current name is {name}"
names = ["foo", "bar"]
for name in names:
print(template_a())
Which prints out:
The current name is foo
The current name is bar
but feels wrong and is limited by the fact that you can only peek at the global namespace in your replacements. Trying to use it in a situation which requires local names will fail miserably unless passed to the string as arguments (which totally beats the point).
Is there any way to bring in a string and have it interpreted as an f-string to avoid using the
.format(**locals())call?
Other than a function (limitations included), nope, so might as well stick with .format.
Or maybe do not use f-strings, just format:
fun = "The curent name is {name}".format
names = ["foo", "bar"]
for name in names:
print(fun(name=name))
In version without names:
fun = "The curent name is {}".format
names = ["foo", "bar"]
for name in names:
print(fun(name))
Using .format is not a correct answer to this question. Python f-strings are very different from str.format() templates … they can contain code or other expensive operations – hence the need for deferral.
Here’s an example of a deferred logger. This uses the normal preamble of logging.getLogger, but then adds new functions that interpret the f-string only if the log level is correct.
log = logging.getLogger(__name__)
def __deferred_flog(log, fstr, level, *args):
if log.isEnabledFor(level):
import inspect
frame = inspect.currentframe().f_back.f_back
try:
fstr = 'f"' + fstr + '"'
log.log(level, eval(fstr, frame.f_globals, frame.f_locals))
finally:
del frame
log.fdebug = lambda fstr, *args: __deferred_flog(log, fstr, logging.DEBUG, *args)
log.finfo = lambda fstr, *args: __deferred_flog(log, fstr, logging.INFO, *args)
This has the advantage of being able to do things like: log.fdebug("{obj.dump()}") …. without dumping the object unless debugging is enabled.
IMHO: This should have been the default operation of f-strings, however now it’s too late. F-string evaluation can have massive and unintended side-effects, and having that happen in a deferred manner will change program execution.
In order to make f-strings properly deferred, python would need some way of explicitly switching behavior. Maybe use the letter ‘g’? 😉
It has been pointed out that deferred logging shouldn’t crash if there’s a bug in the string converter. The above solution can do this as well, change the finally: to except:, and stick a log.exception in there.
A suggestion that uses f-strings. Do your evaluation on the
logical level where the templating is occurring and pass it as a generator.
You can unwind it at whatever point you choose, using f-strings
In [46]: names = (i for i in ('The CIO, Reed', 'The homeless guy, Arnot', 'The security guard Spencer'))
In [47]: po = (f'Strangely, {next(names)} has a nice {i}' for i in (" nice house", " fast car", " big boat"))
In [48]: while True:
...: try:
...: print(next(po))
...: except StopIteration:
...: break
...:
Strangely, The CIO, Reed has a nice nice house
Strangely, The homeless guy, Arnot has a nice fast car
Strangely, The security guard Spencer has a nice big boat
A concise way to have a string evaluated as an f-string (with its full capabilities) is using following function:
def fstr(template):
return eval(f"f'{template}'")
Then you can do:
template_a = "The current name is {name}"
names = ["foo", "bar"]
for name in names:
print(fstr(template_a))
# The current name is foo
# The current name is bar
And, in contrast to many other proposed solutions, you can also do:
template_b = "The current name is {name.upper() * 2}"
for name in names:
print(fstr(template_b))
# The current name is FOOFOO
# The current name is BARBAR
inspired by the answer by kadee, the following can be used to define a deferred-f-string class.
class FStr:
def __init__(self, s):
self._s = s
def __repr__(self):
return eval(f"f'{self._s}'")
...
template_a = FStr('The current name is {name}')
names = ["foo", "bar"]
for name in names:
print (template_a)
which is exactly what the question asked for
What you want appears to be being considered as a Python enhancement.
Meanwhile — from the linked discussion — the following seems like it would be a reasonable workaround that doesn’t require using eval():
class FL:
def __init__(self, func):
self.func = func
def __str__(self):
return self.func()
template_a = FL(lambda: f"The current name, number is {name!r}, {number+1}")
names = "foo", "bar"
numbers = 40, 41
for name, number in zip(names, numbers):
print(template_a)
Output:
The current name, number is 'foo', 41
The current name, number is 'bar', 42
How about:
s = 'Hi, {foo}!'
s
> 'Hi, {foo}!'
s.format(foo='Bar')
> 'Hi, Bar!'
Most of these answers will get you something that behaves sort of like f-strings some of the time, but they will all go wrong in some cases.
There is a package on pypi f-yeah that does all this, only costing you two extra characters! (full disclosure, I am the author)
from fyeah import f
print(f("""'{'"all" the quotes'}'"""))
There are a lot of differences between f-strings and format calls, here is a probably incomplete list
- f-strings allow for arbitrary eval of python code
- f-strings cannot contain a backslash in the expression (since formatted strings don’t have an expression, so I suppose you could say this isn’t a difference, but it does differ from what a raw eval() can do)
- dict lookups in formatted strings must not be quoted. dict lookups in f-strings can be quoted, and so non-string keys can also be looked up
- f-strings have a debug format that format() does not:
f"The argument is {spam=}" - f-string expressions cannot be empty
The suggestions to use eval will get you full f-string format support, but they don’t work on all string types.
def f_template(the_string):
return eval(f"f'{the_string}'")
print(f_template('some "quoted" string'))
print(f_template("some 'quoted' string"))
some "quoted" string
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 2, in f_template
File "<string>", line 1
f'some 'quoted' string'
^
SyntaxError: invalid syntax
This example will also get variable scoping wrong in some cases.
There’s a lot of talk about using str.format(), but as noted it doesn’t allow most of the expressions that are allowed in f-strings such as arithmetic or slices. Using eval() obviously also has it’s downsides.
I’d recommend looking into a templating language such as Jinja. For my use-case it works quite well. See the example below where I have overridden the variable annotation syntax with a single curly brace to match the f-string syntax. I didn’t fully review the differences between f-strings and Jinja invoked like this.
from jinja2 import Environment, BaseLoader
a, b, c = 1, 2, "345"
templ = "{a or b}{c[1:]}"
env = Environment(loader=BaseLoader, variable_start_string="{", variable_end_string="}")
env.from_string(templ).render(**locals())
results in
'145'
to do that I prefer to use fstring inside a lambda function like:
s = lambda x: f'this is your string template to embed {x} in it.'
n = ['a' , 'b' , 'c']
for i in n:
print( s(i) )
You could use a .format styled replacement and explicitly define the replaced variable name:
template_a = "The current name is {name}"
names = ["foo", "bar"]
for name in names:
print (template_a.format(name=name))
Output
The current name is foo
The current name is bar