Truncate delta table in Databricks using python
Question:
Answers:
Not everything is exposed as a function for Python or Java/Scala. Some operations are SQL-only, like OPTIMIZE
for example. If you want to truncate table, you have two choices:
- Use
spark.sql("TRUNCATE TABLE <name>")
or
spark.sql("TRUNCATE TABLE delta.`<path>`")
- Emulate truncate with read + write empty dataframe in overwrite mode:
df = spark.read.format("delta").load("<path>")
df.limit(0).write.mode("overwrite").format("delta").save("<path>")
Not everything is exposed as a function for Python or Java/Scala. Some operations are SQL-only, like OPTIMIZE
for example. If you want to truncate table, you have two choices:
- Use
spark.sql("TRUNCATE TABLE <name>")
or
spark.sql("TRUNCATE TABLE delta.`<path>`")
- Emulate truncate with read + write empty dataframe in overwrite mode:
df = spark.read.format("delta").load("<path>")
df.limit(0).write.mode("overwrite").format("delta").save("<path>")