Apache Spark Interview Questions

In this blog post, i am going to cover most frequently asked Apache Spark Interview Questions. Hope it will help you.

1. What is the difference between map() and flatmap() in Apache Spark?

Ans: This is one of the most frequently asked Interview Questions for Spark Developers.

For answer this one Question, first i will explain similarities  between these two and then we will see the differences between these two.

Similarities:

Similarity between map() and flatmap() is they take a input line from input RDD and and apply a function on it.

Differences:

map transforms an RDD of length N into another RDD of length N.

But flatMap transforms an RDD of length N into a collection of N collections, then flattens these into a single RDD of results.

Example:

Here is my input RDD called, input:

val input = sc.parallelize(Seq("this is spark blogpost", "regarding spark interview questions"))

map():

input.map(_.length).collect

    res1: Array[Int] = Array(23, 32)

flatmap():

rdd.flatMap(_.split(" ")).collect

    res2: Array[String] = Array("this", "is", "spark", "blogpost", "regarding", "spark","interview","questions")