Key Concepts

Review core concepts you need to learn to master this subject

Lambdas in Spark Operations

tiny_rdd = spark.SparkContent.parallelize([1,2,3,4,5]) rdd.map(lambda x: x*2).take(5) # output: [2,4,6,8,10]

Lambdas expressions allow us to apply a simple operation to an object without needing to define it as a function. This improves readability by condensing what could be a few lines of code into a single line. Utilizing lambdas in Spark operations allow us to apply any arbitrary function to all RDD elements specified by the transformation or action.

What you'll create

Portfolio projects that showcase your new skills

Pro Logo