Skip to content

Backport EMRServless step and job to V2 #5857

@mjost5v

Description

@mjost5v

Describe the feature you'd like
SDK v3 has EMR Serverless support. It'd be good to use that instead of the PySpark processor stuck on Spark 3.3

How would this feature be used? Please describe.
Deprecate and replace PySparkProcessor

Describe alternatives you've considered
Keeping with PySparkProcessor but that would require migrating code with upgrading to SDK v3

Additional context
Add any other context or screenshots about the feature request here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions