isin

Including null inside PySpark isin

Including null inside PySpark isin Question: This is my dataframe: from pyspark.sql import SparkSession from pyspark.sql import functions as F spark = SparkSession.builder.getOrCreate() dCols = [‘c1’, ‘c2’] dData = [(‘a’, ‘b’), (‘c’, ‘d’), (‘e’, None)] df = spark.createDataFrame(dData, dCols) Is there a syntax to include null inside .isin()? Something like df = df.withColumn( ‘newCol’, F.when(F.col(‘c2’).isin({‘d’, …

Total answers: 4