Convert a simple one line string to RDD in Spark

Question:

I have a simple line:

line = "Hello, world"

I would like to convert it to an RDD with only one element.
I have tried

sc.parallelize(line)

But it get:

sc.parallelize(line).collect()
['H', 'e', 'l', 'l', 'o', ',', ' ', 'w', 'o', 'r', 'l', 'd']

Any ideas?

Asked By: poiuytrez

||

Answers:

try using List as parameter:

sc.parallelize(List(line)).collect()

it returns

res1: Array[String] = Array(hello,world)
Answered By: michaeltang

The below code works fine in Python

sc.parallelize([line]).collect()
['Hello, world']

Here we are passing the parameter "line" as a list.

Answered By: Dhruv

use the following code :

sc.parallelize(Seq(line))
Answered By: vivek