using variable (f)-string stored in json

Question

I have a json config file where I store my path to data there

The data is bucketed in month and days, so without the json I would use an f-string like:

spark.read.parquet(f"home/data/month={MONTH}/day={DAY}")

Now I want to extract that from json. However, I run into problems with the Month and day variable. I do not want to split the path in the json.
But writing it like this:

{
    "path":"home/data/month={MONTH}/day={DAY}"
}

and loading with:

DAY="1"
MONTH="12"
conf_path=pandas.read_json("...")
path=conf_path["path"]
data=spark.read_parquet(f"{path}")

does not really work.

Could you hint me a solution to retrieving a path with variable elements and filling them after reading? How would you store the path or retrieve it without splitting the path?

Asked By: Florida Man

||

Source

Answer 1

you should use string.format() instead of f-strings

Still if you want to use f-strings then you should use eval like this, its unsafe

DAY="1"
MONTH="12"
df = pd.DataFrame(
    [{
    "path":"home/data/month={MONTH}/day={DAY}"
 },
 {
    "path":"home/data/month={MONTH}/day={DAY}"
 }
]
)
a = df['path'][0]
print(eval(f"f'{a}'"))
#home/data/month=12/day=1

Answered By: Deepak Tripathi

Answer 2

Thanks to Deepak Tripathi answer, the answer is to use string format. Like this:

day="1"
month="12"
conf_path=pandas.read_json("...")
path=conf_path["path"]
data=spark.read_parquet(path.format(MONTH=month, DAY=day))

_{This answer was posted as an edit to the question using variable (f)-string stored in json by the OP Florida Man under CC BY-SA 4.0.}

Answered By: vvvvv

using variable (f)-string stored in json

Question:

Answers: