Skip to content

[aws] elb_metrics: collect Application ELB HTTPCode_Target_* metrics - #20609

Open
shmsr wants to merge 7 commits into
elastic:mainfrom
shmsr:fix/aws-elb-target-http-metrics
Open

[aws] elb_metrics: collect Application ELB HTTPCode_Target_* metrics#20609
shmsr wants to merge 7 commits into
elastic:mainfrom
shmsr:fix/aws-elb-target-http-metrics

Conversation

@shmsr

@shmsr shmsr commented Aug 7, 2026

Copy link
Copy Markdown
Member

Proposed commit message

Application ELB elb_metrics previously collected HTTPCode_ELB_* response-code metrics but not the target-generated HTTPCode_Target_{2,3,4,5}XX_Count CloudWatch metrics, so target HTTP status counts were missing from ingested documents.

Add those metrics to the Application ELB CloudWatch metric list and field mappings, map HealthyHostCount/UnHealthyHostCount under aws.applicationelb.metrics, bump the package to 7.1.2, regenerate ELB docs, and add an elb_metrics system-test Terraform stack (internal ALB + target + in-VPC traffic client) so the metrics can be validated end-to-end.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

Author's Checklist

  • Confirm HTTPCode_Target_* appear under aws.applicationelb.metrics when Application ELB traffic reaches healthy targets
  • Confirm system-test Terraform applies cleanly in CI/AWS test account

How to test this PR locally

cd packages/aws
elastic-package build
elastic-package check
# Requires AWS credentials with permission to create ALB/EC2 in the default VPC:
elastic-package test system --data-streams elb_metrics -v --defer-cleanup 15m

Verification notes

A fresh local system test passed for aws/elb_metrics (PASS, exit 0). During the run, Elasticsearch documents contained non-zero values for:

  • aws.applicationelb.metrics.HTTPCode_Target_2XX_Count.sum
  • aws.applicationelb.metrics.HTTPCode_Target_3XX_Count.sum
  • aws.applicationelb.metrics.HTTPCode_Target_4XX_Count.sum
  • aws.applicationelb.metrics.HTTPCode_Target_5XX_Count.sum

The generated ELB docs also include the new Target HTTP code metrics and the HealthyHostCount / UnHealthyHostCount Application ELB mappings.

Application ELB was only collecting HTTPCode_ELB_* metrics. Add
HTTPCode_Target_{2,3,4,5}XX_Count to the CloudWatch metric list and
field mappings, and map HealthyHostCount/UnHealthyHostCount that were
already collected. Add a system-test stack that generates in-VPC ALB
traffic so these metrics can be validated.
@shmsr
shmsr requested review from a team as code owners August 7, 2026 17:55
@shmsr shmsr added the bug Something isn't working, use only for issues label Aug 7, 2026
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

✅ Elastic Docs Style Checker (Vale)

No issues found on modified lines!


The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

shmsr added 4 commits August 10, 2026 12:25
Generate 2xx/3xx/4xx/5xx responses in the system-test fixture so all
HTTPCode_Target_* metrics are exercised, and discover t3.micro AZs via
instance-type offerings instead of hardcoding us-east-1 names.
Quote the systemd unit heredoc and inject the ALB DNS via sed so \$p
is not expanded away at cloud-init time (which forced all traffic to /).
@shmsr
shmsr requested a balanced review from Copilot August 10, 2026 09:24

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds missing Application ELB target response-code metrics, field mappings, documentation, and end-to-end test infrastructure.

Changes:

  • Collects and maps HTTPCode_Target_{2,3,4,5}XX_Count and host-health metrics.
  • Adds an ALB traffic-generation system test.
  • Bumps AWS integration to 7.1.2.

Contributor tooling: /plugin install integration-skills@elastic-integration-skills

Reviewed changes

Copilot reviewed 9 out of 10 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
packages/aws/manifest.yml Bumps package version.
packages/aws/changelog.yml Records the metric additions.
packages/aws/docs/elb.md Documents new fields.
packages/aws/data_stream/elb_metrics/fields/fields.yml Defines metric mappings.
packages/aws/data_stream/elb_metrics/agent/stream/stream.yml.hbs Requests target-code metrics.
packages/aws/data_stream/elb_metrics/_dev/test/system/test-default-config.yml Configures the system test.
packages/aws/data_stream/elb_metrics/_dev/deploy/tf/main.tf Provisions the ALB test environment.
packages/aws/data_stream/elb_metrics/_dev/deploy/tf/variables.tf Defines deployment variables.
packages/aws/data_stream/elb_metrics/_dev/deploy/tf/env.yml Passes AWS credentials and region.
packages/aws/data_stream/elb_metrics/_dev/deploy/tf/.terraform.lock.hcl Locks the AWS provider.
Files not reviewed (1)
  • packages/aws/data_stream/elb_metrics/_dev/deploy/tf/.terraform.lock.hcl: Generated file

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

internal = true
load_balancer_type = "application"
security_groups = [aws_security_group.alb.id]
subnets = slice(sort(data.aws_subnets.default.ids), 0, min(2, length(data.aws_subnets.default.ids)))
@infra-vault-gh-plugin-prod

Copy link
Copy Markdown

💚 Build Succeeded

History

Align the ALB subnet selection with the AZ-qualified target subnet set so the test topology always routes through a target-enabled AZ, and require all four Target HTTP code fields in the system test so missing metrics fail verification.
@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

✅ All changelog entries have the correct PR link.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working, use only for issues

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants