Apache Spark Structured Streaming

Structured Streaming is a scalable and fault-tolerant stream processing engine built on the Spark SQL engine. inshort Structure streaming rebuild streaming functionalities on top of Catalyst and Data Frames. Structured Streaming is the first API to build stream processing on top of SQL engine,in...

Read More

Apache Spark Broadcast vs Accumulators

Broadcast variable : Broadcast variable is a read-only variable that is made available from the driver program that runs the SparkContext object to the nodes that will execute the computation. This is very useful in applications that need to make the same data available to the worker nodes in an...

Read More