scala - How to extract RDD map from a line of space seperated values when the number of values are unknown? -
my input file has space separated values first 1 being index. converting rdd map. did fixed number of values. can please suggest me how if don't know how many values there in 1 line.
val vectors = inputfile.map(line => { val = line.split(" "); (a(0))->(a(1), a(2), a(3))})
in above line able first 4 values array of form (1,(4,5,6))
. if have 1 4 5 6 38 24
in input file. how handle this?
use head
, tail
methods:
val s = sc.parallelize(seq("1 2 3 4 5", "2 1 3 4")) s.map(line => { val fields = line.split(" ") map(fields.head -> fields.tail) }).collect // res16: array[scala.collection.immutable.map[string,array[string]]] = // array(map(1 -> array(2, 3, 4, 5)), map(2 -> array(1, 3, 4)))
Comments
Post a Comment