feat: Stream Connection CloudFormation Resource

[sivaram-mongodb]

Proposed changes

• Manages connections for Atlas Stream Processing workspaces to data sources and sinks
• Supports five connection types: Cluster, Kafka, Sample, AWSLambda, and HTTPS
• Provides comprehensive authentication mechanisms for Kafka (PLAIN, SCRAM-256, SCRAM-512, OAUTHBEARER)
• Supports AWS PrivateLink networking for secure Kafka connectivity
• Backward compatible with both WorkspaceName (preferred) and InstanceName fields.

Resource Configuration:

The Stream Connection resource enables you to connect your Stream Processing workspaces to various data sources and destinations. Connections serve as the bridge between your streaming pipelines and external systems like Atlas clusters, Kafka brokers, sample data sources, AWS Lambda functions, and HTTP endpoints.

Required Properties:

• ProjectId: Atlas project identifier (24-hexadecimal characters)
• ConnectionName: Human-readable label for the connection
• Type: Connection type (Cluster, Kafka, Sample, AWSLambda, or Https)

Optional Properties (Type-Specific):

• WorkspaceName or InstanceName: Stream workspace identifier (WorkspaceName is preferred)
• Profile: AWS Secrets Manager profile (default: "default")

Cluster Connection:

• ClusterName: Name of the Atlas cluster
• ClusterProjectId: Project ID for cross-project connections
• DbRoleToExecute: Database role configuration
  • Role: Role name (built-in or custom)
  • Type: BUILT_IN or CUSTOM

Kafka Connection:

• BootstrapServers: Comma-separated list of Kafka broker addresses
• Authentication: Kafka authentication configuration
  • Mechanism: PLAIN, SCRAM-256, SCRAM-512, or OAUTHBEARER
  • Username: Kafka username
  • Password: Kafka password (write-only)
  • Method, TokenEndpointUrl, ClientId, ClientSecret, Scope, SaslOauthbearerExtensions: OAuth fields
• Security: Kafka security configuration
  • Protocol: PLAINTEXT or SSL
  • BrokerPublicCertificate: x509 certificate for SSL
• Config: Additional Kafka configuration (key-value map)
• Networking: AWS PrivateLink configuration
  • Access.Type: PRIVATE_ENDPOINT or PUBLIC
  • Access.ConnectionId: PrivateLink connection ID

AWS Lambda Connection:

• Aws.RoleArn: IAM role ARN for Lambda invocation

HTTPS Connection:

• Url: HTTP endpoint URL
• Headers: HTTP headers (key-value map)

Create-Only Properties:

• ProjectId, ConnectionName, Type, WorkspaceName, InstanceName, Profile: Cannot be changed after creation

Write-Only Properties:

• Authentication.Password, Authentication.ClientSecret: Not returned in Read operations for security

Configuration Examples:

Cluster Connection:

{
  "ProjectId": "64d2a9b8f1a2c3e4d5e6f7a8",
  "WorkspaceName": "my-stream-workspace",
  "ConnectionName": "cluster-connection",
  "Type": "Cluster",
  "ClusterName": "my-cluster",
  "DbRoleToExecute": {
    "Role": "atlasAdmin",
    "Type": "BUILT_IN"
  }
}

cfn testing:

stack testing:

Atlas UI - Before Creation

Atlas UI - Before Creation

Jira ticket: CLOUDP-369806

Please include a summary of the fix/feature/change, including any relevant motivation and context.

Link to any related issue(s):

Type of change:

• [] Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as
  expected)
• This change requires a documentation update
• If changes include removal or addition of 3rd party GitHub actions, I updated our internal document. Reach out to the APIx Integration slack channel to get access to the internal document.

Manual QA performed:

• cfn invoke for each of CRUDL/cfn test
• Updated resource in example
• Published to AWS private registry
• Used the template in example to create and update a stack in AWS
• Deleted stack to ensure resources are deleted
• Created multiple resources in same stack
• Validated in Atlas UI
• Included screenshots

Required Checklist:

• I have signed the MongoDB CLA
• I have added tests that prove my fix is effective or that my feature works
• I have checked that this change does not generate any credentials and that they are NOT accidentally logged anywhere.
• I have added any necessary documentation (if appropriate)
• I have run make fmt and formatted my code
• For CFN Resources: I have released by changes in the private registry and proved by change
  works in Atlas

Further comments

[sivaram-mongodb]

Closing PR - AWS Lambda stream connection examples need to be added and require testing.