.Parquet to .Hyper file conversion for any schema
Question:
I want to convert parquet file to hyper file format using python. There is the following git for this – https://github.com/tableau/hyper-api-samples/blob/main/Community-Supported/parquet-to-hyper/create_hyper_file_from_parquet.py.
But in this case the parquet format /schema is known beforehand. What should I do if I want it to work for any parquet file, irrespective of the schema.
About me, I mostly work in analytics and data science with python but wanted to work on this project to make some files accessible to tableau. Thank you in advance and please let me know if you want any more information.
Answers:
From Tableau’s official Github: https://github.com/tableau/hyper-api-samples/blob/main/Community-Supported/parquet-to-hyper/create_hyper_file_from_parquet.py
If you do not wish to define a schema when creating a .hyper file from a parquet file you can use the CREATE TABLE command instead of the COPY command.
To use the CREATE TABLE command you can skip the schema and table definition like this:
# Start the Hyper process.
with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU) as hyper:
# Open a connection to the Hyper process. This will also create the new Hyper file.
# The `CREATE_AND_REPLACE` mode causes the file to be replaced if it
# already exists.
with Connection(endpoint=hyper.endpoint,
database=hyper_database_path,
create_mode=CreateMode.CREATE_AND_REPLACE) as connection:
connection.execute_command("CREATE TABLE products AS (SELECT * FROM external('products.parquet'))")
I want to convert parquet file to hyper file format using python. There is the following git for this – https://github.com/tableau/hyper-api-samples/blob/main/Community-Supported/parquet-to-hyper/create_hyper_file_from_parquet.py.
But in this case the parquet format /schema is known beforehand. What should I do if I want it to work for any parquet file, irrespective of the schema.
About me, I mostly work in analytics and data science with python but wanted to work on this project to make some files accessible to tableau. Thank you in advance and please let me know if you want any more information.
From Tableau’s official Github: https://github.com/tableau/hyper-api-samples/blob/main/Community-Supported/parquet-to-hyper/create_hyper_file_from_parquet.py
If you do not wish to define a schema when creating a .hyper file from a parquet file you can use the CREATE TABLE command instead of the COPY command.
To use the CREATE TABLE command you can skip the schema and table definition like this:
# Start the Hyper process.
with HyperProcess(telemetry=Telemetry.SEND_USAGE_DATA_TO_TABLEAU) as hyper:
# Open a connection to the Hyper process. This will also create the new Hyper file.
# The `CREATE_AND_REPLACE` mode causes the file to be replaced if it
# already exists.
with Connection(endpoint=hyper.endpoint,
database=hyper_database_path,
create_mode=CreateMode.CREATE_AND_REPLACE) as connection:
connection.execute_command("CREATE TABLE products AS (SELECT * FROM external('products.parquet'))")