How to Create HDFS file
Question:
I know it is possible to create directory HDFS with python using snakebite
But I am looking to create a file on HDFS directory
Answers:
You can use touchz to create an empty file on HDFS…
I see rename command in the docs, which might accept local source with a remote HDFS path for uploading files, but there is no put or copyFromLocal CLI command, so I don’t think Snakebite can do what you want.
I do see put for the minicluster.
Overall, you may have better luck with pyspark or hdfs depending on your needs.
using subprocess
from subprocess import Popen, PIPE
(ret, out, err)= run_cmd(['hdfs', 'dfs', '-touchz', '/directory/filename'])
I know it is possible to create directory HDFS with python using snakebite
But I am looking to create a file on HDFS directory
You can use touchz to create an empty file on HDFS…
I see rename command in the docs, which might accept local source with a remote HDFS path for uploading files, but there is no put or copyFromLocal CLI command, so I don’t think Snakebite can do what you want.
I do see put for the minicluster.
Overall, you may have better luck with pyspark or hdfs depending on your needs.
using subprocess
from subprocess import Popen, PIPE
(ret, out, err)= run_cmd(['hdfs', 'dfs', '-touchz', '/directory/filename'])