PySpark logging from the executor Question: What is the correct way to access the log4j logger of Spark using pyspark on an executor? It’s easy to do so in the driver but I cannot seem to understand how to access the logging functionalities on the executor so that I can log locally and let YARN …
log4j with timestamp per log entry Question: this is my log output INFO main digestemails – process inbox INFO main digestemails – checking for emails in c:developmentDCMailemailKN-Source INFO main digestemails – digesting [email protected] INFO main digestemails – extracting attachments INFO main digestemails – no attachments or no attachments supported INFO main digestemails – updating database …
