Copy PySpark Dataframe to MySQL

Question:

I am trying to load a dataframe created with PySpark in DataBricks to MySql but it tells me:

com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure

df_videojuegos.select("Nombre", "Plataforma", "Año", "Genero", "Editorial", "Ventas NA", "Ventas EU", "Ventas JP", "Ventas Otros", "Ventas Global") 
    .write 
    .format("jdbc") 
    .option("url", "jdbc:mysql://127.0.0.1:3306/dezyre_db&useUnicode=true&characterEncoding=UTF-8&useSSL=false") 
    .option("driver", "com.mysql.jdbc.Driver").option("dbtable", "reportes") 
    .option("user", "root").option("password", "root") 
    .save()

I already have the connector installed in the cluster for MySql and these imported libraries:

from pyspark.sql import SparkSession
from pyspark.sql import Row

spark = SparkSession.builder 
    .config("spark.jars", "/home/hduser/mysql-connector-java-5.1.47/mysql-connector-java-5.1.47.jar") 
    .master("local") 
    .appName("PySpark_MySQL_test2") 
    .getOrCreate()

I don’t know if the url that I have declared is really correct, or if I have the syntax wrong.

Install the connectors, create the connection with MySQL but when loading the data it gives me an error

Asked By: Linexthe

||

Answers:

Ok, the error was a newbie, I was trying to connect from DataBricks to a localhost! :C

Answered By: Linexthe
Categories: questions Tags: , ,
Answers are sorted by their score. The answer accepted by the question owner as the best is marked with
at the top-right corner.