Faster Spark jobs with less wasted memory.
What it does
Guides improvements to Apache Spark jobs to reduce shuffle, data skew, and memory pressure.
When to use it
Use when optimizing slow Spark jobs, tuning memory, improving partitioning, debugging performance, or scaling pipelines.
How to use it
It changes how the agent optimizes Spark jobs through execution settings, data formats, joins, and partitioning.
What you provide
- An existing project
Latest review
What you need · 1
The Python examples import pyspark for Spark sessions and DataFrame functions.
Access · 0
No additional actions listed.