How can I use databricks utils functions in PyCharm? I can't find appropriate pip package
Question:
PyCharm IDE. I want to use dbutils.widgets.get() in a module and than to import this module to databricks. I already tried with pip install databricks-client pip install databricks-utils and pip install DBUtils
Answers:
The dbutils is available only as a part of the databricks-connect package. Its documentation contains detailed description on how to setup PyCharm to work with it. Ut also covers on how to use the dbutils.
You may need to define following wrapper to be able to use dbutils locally and on Databricks:
def get_dbutils(spark):
from pyspark.dbutils import DBUtils
return DBUtils(spark)
get_dbutils().fs.cp('file:/home/user/data.csv', 'dbfs:/uploads')
PyCharm IDE. I want to use dbutils.widgets.get() in a module and than to import this module to databricks. I already tried with pip install databricks-client pip install databricks-utils and pip install DBUtils
The dbutils is available only as a part of the databricks-connect package. Its documentation contains detailed description on how to setup PyCharm to work with it. Ut also covers on how to use the dbutils.
You may need to define following wrapper to be able to use dbutils locally and on Databricks:
def get_dbutils(spark):
from pyspark.dbutils import DBUtils
return DBUtils(spark)
get_dbutils().fs.cp('file:/home/user/data.csv', 'dbfs:/uploads')