Skip to content

Log streaming from Kestrel Operator to Kestrel Cloud via S2#7

Open
ramanv0 wants to merge 21 commits into
mainfrom
log-streaming-s2
Open

Log streaming from Kestrel Operator to Kestrel Cloud via S2#7
ramanv0 wants to merge 21 commits into
mainfrom
log-streaming-s2

Conversation

@ramanv0
Copy link
Copy Markdown
Collaborator

@ramanv0 ramanv0 commented Oct 8, 2025

This PR introduces S2 log streaming support in the Kestrel Operator. Includes configuration updates, Helm changes, and core implementation for streaming runtime logs to Kestrel Cloud.

Summary of changes:

  • Add S2 streaming config to values.yaml and deployment.yaml
  • Add conditional secret for S2 credentials
  • Implement S2 streaming client, log manager, and S2Core (zapcore.Core)
  • Initialize log streaming via s2_streaming.InitializeLogStreaming
  • Move Helm template for cluster ID initialization
  • Update Go module and sum files with new S2 SDK
  • Create tests for new S2 client implementation
  • Minor cleanups and comment updates

…ingWithLogger; set logger to wrapperLogger, which streams to s2
… env var) to the deployment.yaml in the operator helm chart
@ramanv0 ramanv0 changed the title Log streaming s2 Log streaming from Kestrel Operator to Kestrel Cloud via S2 Oct 8, 2025
"go.uber.org/zap/zapcore"
)

// S2Core is a zapcore.Core that streams logs to S2
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nicee

Comment on lines +235 to +245
// Create timestamp (S2 expects milliseconds since epoch)
timestamp := uint64(entry.Timestamp)

// Create headers for metadata (can be used for filtering/indexing)
headers := []s2.Header{
{Name: []byte("level"), Value: []byte(entry.Level)},
{Name: []byte("component"), Value: []byte(entry.Component)},
{Name: []byte("cluster_id"), Value: []byte(entry.ClusterID)},
{Name: []byte("pod_name"), Value: []byte(entry.PodName)},
{Name: []byte("namespace"), Value: []byte(entry.Namespace)},
}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like that you are making use of native ways to model this metadata :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants