feat: Stream Processor CloudFormation Resource

[sivaram-mongodb]

Proposed changes

Added new resource Stream Processor:

• Manages Atlas Stream Processing Processors within Stream Workspaces
• Supports configurable stream processing pipelines
• Provides state management (CREATED, STARTED, STOPPED)
• Includes Dead Letter Queue (DLQ) configuration for error handling
• Implements callback-based asynchronous operations for long-running processor creation and state transitions
• Supports both WorkspaceName and InstanceName for backward compatibility

Resource Configuration:

The Stream Processor resource enables you to create and manage stream processing pipelines that transform and route data from source connections to sink destinations. The resource supports the following key features:

Required Properties:

• ProjectId: Atlas project identifier (24-hexadecimal characters)
• ProcessorName: Human-readable label for the stream processor
• Pipeline: JSON-encoded array of stream aggregation pipeline stages

Optional Properties:

• WorkspaceName or InstanceName: Stream workspace identifier (WorkspaceName is preferred)
• DesiredState: Target state for the processor (CREATED, STARTED, STOPPED)
• Options.Dlq: Dead Letter Queue configuration for failed messages
  • ConnectionName: Atlas cluster connection for DLQ storage
  • Db: Database name for DLQ messages
  • Coll: Collection name for DLQ messages
• Timeouts.Create: Timeout duration for creation operations (default: 20 minutes)
• DeleteOnCreateTimeout: Whether to delete the resource on timeout (default: true)

Read-Only Properties:

• Id: Unique 24-hexadecimal identifier
• State: Current processor state from Atlas API
• Stats: Processing statistics (available when processor is STARTED)

cfn testing:

stack testing:

result:

Atlas Stream Processor Created:

Jira ticket: CLOUDP-368428

Please include a summary of the fix/feature/change, including any relevant motivation and context.

Link to any related issue(s):

Type of change:

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as
  expected)
• This change requires a documentation update
• If changes include removal or addition of 3rd party GitHub actions, I updated our internal document. Reach out to the APIx Integration slack channel to get access to the internal document.

Manual QA performed:

• cfn invoke for each of CRUDL/cfn test
• Updated resource in example
• Published to AWS private registry
• Used the template in example to create and update a stack in AWS
• Deleted stack to ensure resources are deleted
• Created multiple resources in same stack
• Validated in Atlas UI
• Included screenshots

Required Checklist:

• I have signed the MongoDB CLA
• I have added tests that prove my fix is effective or that my feature works
• I have checked that this change does not generate any credentials and that they are NOT accidentally logged anywhere.
• I have added any necessary documentation (if appropriate)
• I have run make fmt and formatted my code
• For CFN Resources: I have released by changes in the private registry and proved by change
  works in Atlas

Further comments

[sivaram-mongodb]

Closing PR - Stream processor with Kafka examples need to be added and require testing.