Describe the feature you'd like
SDK v3 has EMR Serverless support. It'd be good to use that instead of the PySpark processor stuck on Spark 3.3
How would this feature be used? Please describe.
Deprecate and replace PySparkProcessor
Describe alternatives you've considered
Keeping with PySparkProcessor but that would require migrating code with upgrading to SDK v3
Additional context
Add any other context or screenshots about the feature request here.
Describe the feature you'd like
SDK v3 has EMR Serverless support. It'd be good to use that instead of the PySpark processor stuck on Spark 3.3
How would this feature be used? Please describe.
Deprecate and replace PySparkProcessor
Describe alternatives you've considered
Keeping with PySparkProcessor but that would require migrating code with upgrading to SDK v3
Additional context
Add any other context or screenshots about the feature request here.