Update grafana/mimir Docker tag to v3.2.0 - #347
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR contains the following updates:
3.1.4→3.2.0Warning
Some dependencies could not be looked up. Check the Dependency Dashboard for more information.
Release Notes
grafana/mimir (grafana/mimir)
v3.2.0Grafana Mimir
-query-frontend.log-query-request-headersnow rejects headers that carry credentials or session material (e.g.Authorization,Cookie,X-Api-Key) at startup, and any such headers that reach the slow-query/query-stats log paths are redacted as defense in depth. Operators that previously allow-listed such headers must remove them from the flag. #15487prometheus/alertmanagerlibrary to v0.33.0. The per-tenant alertmanager web UI and the/-/healthy,/-/ready,/-/reload,/script.js, and/favicon.icoendpoints under the alertmanager prefix are no longer served because upstream removed the embeddable UI package. The v2 API endpoints and request/response shapes are unchanged, but a few error message strings from the upstream parser have been reformatted (for example, matcher-validation errors now includein set N). #15144, #15733cortex_alertmanager_dispatcher_aggregation_group_limit_reached_totalcounter is now best-effort under concurrent alert ingest. The upstream alertmanager v0.32.0 ingests alerts via worker goroutines and the group-limit check is a racy check-then-act, so the configured limit can be exceeded under burst load. #15144min(...)andmax(...)used inside[...]range/subquery brackets andoffsetclauses have been renamed tomin_of(...)andmax_of(...). #15593 #15603querier.mimir-query-engine.enable-projection-pushdown. #15618tests.write-read-series-test.num-seriesandtests.write-read-series-test.max-query-ageto match the values being set in jsonnet. #15705gcr.io/distroless/static-debian13; race images usebase-nossl-debian13). #15629component="querier"label are now emitted with aengine="querier"label instead, mirroring the similar metrics emitted by the query-frontend and other querier metrics. #15787-querier.max-concurrent, to 8. #15984-query-frontend.query-sharding-total-shardsand for the per-query computed value. #15807-compactor.split-and-merge-shardsand-compactor.ooo-split-and-merge-shardsare now rounded up to the next power of two, so compactor and query shards always mesh (one is a divisor or multiple of the other). #15807-querier.minimize-ingester-requests-hedging-delay=3s. #15976-querier.mimir-query-engine.enable-prune-toggles. Pruning of toggled query expressions such as... and on() (vector(0) == 1)are now handled by the remove-statically-empty-expressions optimization pass, enabled with-querier.mimir-query-engine.enable-remove-statically-empty-expressions=true. #16037-query-frontend.enable-multiple-node-remote-execution-requestsCLI flag and associated config file option has been removed. #16187separator/replacement(upstream #18653). The/runtime_config(and?mode=diff) output for tenants that set an emptyseparator/replacementinmetric_relabel_configswill now show those fields explicitly. No effect on relabeling behavior. #16198-query-frontend.parallelize-shardable-queries=false. #16212-ingest-storage.kafka.producer-compressionflag to configure the Kafka producer compression codec. Supported values arenone,gzip,snappy,lz4, andzstd. Set it tononeto target Azure Event Hub's Kafka-compatible endpoint, which does not support compressed produce requests. #15235-ingester.compute-workers(default 0 = GOMAXPROCS) and-ingester.label-values-count-chunk-size(default 32). #15493-ingester.float-chunk-encodingflag (per-tenantfloat_chunk_encodinglimit) to select the float chunk encoding (xororxor2). The overrides-exporter can export it as a numeric value (4forxor,7forxor2) whenfloat_chunk_encodingis added to-overrides-exporter.enabled-metrics. #15831-auth.label-access-control-enabled, Mimir enforces label selectors from theX-Prom-Label-PolicyHTTP header at query time, filtering series and exemplars per tenant policy. Cache key isolation is applied automatically in the query-frontend. #15554mimirtool rules,mimirtool alertmanager,mimirtool alerts,mimirtool backfill, andmimirtool analyze ruler. #14959additional_cost_attribution_trackersconfig field. #15302cortex_querier_inflight_query_max_age_secondsmetric reporting the age of the oldest in-flight query memory consumption tracker. #15300present_over_timeover range vectors in instant queries. #15386cortex_query_scheduler_queue_max_wait_secondsmetric reporting how long the oldest request still waiting in the queue has been waiting since it was enqueued. Requests already dispatched to a querier for execution are excluded, so the metric reflects only queue wait time and acts as a queue-starvation detector. Reports 0 when no requests are waiting. Enabled by default; can be disabled with-query-scheduler.queue-max-wait-metric-enabled=false. #15419 #15960cortex_query_scheduler_max_queue_lengthmetric reporting the per-tenant peak queue length observed since the last scrape. Enable with-query-scheduler.max-queue-length-metric-enabled=true. #15906min_ofandmax_of. #15597histogram_quantiles, which computes multiple quantiles from classic or native histograms in a single call. #15710</(trim upper) and>/(trim lower), including query sharding support. #15708 #15711-ingester.early-head-compaction-owned-series-thresholdand the-ingester.early-compaction-non-owned-series-min-grace-periodhas elapsed, or (b) the-ingester.early-compaction-non-owned-series-max-grace-periodhas elapsed, regardless of the threshold. Disabled by default; enable with-ingester.early-compaction-non-owned-series-enabled. Requires-ingester.track-ingester-owned-seriesor-ingester.use-ingester-owned-series-for-limitsto be enabled. Adds thecortex_ingester_tsdb_early_compaction_non_owned_series_triggered_totalcounter. #15314 #15653 #15657 #15661cortex_ingester_tsdb_head_chunks_max_mmappedgauge reporting the maximum, across all per-tenant TSDBs, of the maximum number of head chunks memory-mapped for any individual series during the last memory-mapping pass. Temporary measurement metric; will be removed once we have collected enough data. #15616dump find-duplicatescommand to scan an exported dump and report float samples re-sent with the same timestamp and value as the previous sample for a series. These exact duplicates are silently dropped by the ingester but still count toward received-samples metrics. An optional--tenantflag restricts the scan to a single tenant. #15506-ingest-storage.kafka.backend=warpstream. It hedges and reroutes produce requests across WarpStream agents to reduce write tail latency, and is tuned via the experimental-ingest-storage.kafka.warpstream-*flags. #15236 #15809-query-frontend.use-mimir-query-engine-for-splitting-and-caching-results=true. #15348 #15393 #15397 #15650 #15720 #15750 #15769 #15783 #15787 #15795 #15804 #15822 #15823 #15846 #15827 #15884 #15887 #15912 #15928 #15930 #15954 #15955 #15978 #15999 #16015 #16213-query-frontend.query-sharding-max-sharded-querieslimit applies per time-split interval and spun-off subquery, rather than to the entire interval (or entire spun-off subquery).-ruler.distributor.addresssupport for pushing rule evaluation results to distributors over native gRPC instead of using an internal distributor. Configure the per-request timeout with-ruler.distributor.remote-timeout. #15891-querier.mimir-query-engine.enable-scalar-common-subexpression-elimination=true. #15991mimirtool partition-ring remove-all-owners-and-partitionsto forcefully remove all owners and partitions from a partition ring in a single operation. #16081-ingest-storage.ingester-partition-metric-label-enabledflag. When set together with-ingest-storage.enabled, every metric emitted by the ingester (and the ingest-storage reader / partition-ring lifecycler it owns) carries aningester_partitionlabel identifying the Kafka partition the ingester consumes. Planned to default to enabled in Mimir 3.2 and to be removed in Mimir 3.5. #15130/api/v1/search/{metric_names,label_names,label_values}. Gated by-querier.experimental-search-api-enabled(default false). #15233, #15349, #15301, #15347, #15364exclude_tenantsoption to the-backend.config-fileconfiguration. Requests whoseX-Scope-OrgIDtenants are all excluded are not sent to that backend, while the preferred backend always receives all traffic. #16075-query-frontend.active-series-max-shard-concurrencyto bound how many sharded active series (and active native histogram metrics) sub-requests are dispatched and merged concurrently within a single request, limiting the resource usage caused by fanning out to a large number of shards, both on queriers and on the query-frontend. 0 (the default) keeps the previous unbounded behavior. #15970-query-frontend.active-series-framed-responsesto request active series responses from queriers in a length-delimited framed format that the query-frontend can merge using significantly less CPU (roughly 5-6x faster merges for high shard counts). Queriers that don't support the format fall back to JSON transparently. #15971-ingest-storage.kafka.producer-max-buffered-byteslimit is reached, instead of rejecting individual records. #15227unlessandoroperations where one side can be proven to be empty by inspecting the expression. #15198cortex_ingest_storage_writer_serialize_duration_secondsnative histogram metric tracking the time spent serializing an incoming request to Kafka records. #15527-memberlist.compression-algorithmflag to select the algorithm used to compress outgoing messages. Supported values:lzw(default) andsnappy. The flag is ignored when-memberlist.compression-enabledis false. Before reconfiguring any node to emit a new algorithm, upgrade every cluster member to a build that can decode it, otherwise messages are dropped. #15357memberlist.processed-messages-queue-sizeflag to set the size of the per-key internal queue for processing messages received from other nodes. Increasing this value may help to avoid dropping per-key updates when the node is processing many updates for the same key. #15536Cache-Control: no-storerequest header when caching intermediate results for range vector splitting. #15148-ingest-storage.kafka.backend=warpstream) now traces produce requests, emitting the same producer spans andtraceparentpropagation as the default Kafka backend. #16039-runtime-config.http-client-disable-keep-alivesis enabled by default, an can be set tofalsein-order to re-enable keep-alives. #15695smoothedandanchoredextended range selector modifiers. #15398-usage-tracker.min-time-between-shards-cleanup, to avoid blocking latency-sensitive series-tracking calls on large single-tenant instances. #15871-distributor.usage-tracker-client.use-sync-batched-trackingis enabled, synchronous series-tracking calls linger for up to-distributor.usage-tracker-client.sync-batch-delayand are sent together in a single batch RPC, reducing the number of network calls while still returning rejected series to each caller. By default all partitions flush together on a shared timer so the usage-tracker can coalesce the packets; set-distributor.usage-tracker-client.sync-batch-independent-partition-timeoutsto make each partition linger independently instead. #15805ingest_storagefeature flag in the/api/v1/status/buildinfoendpoint, reflecting whether Mimir runs with ingest storage architecture. #15743-blocks-storage.tsdb.bigger-out-of-order-blocks-for-old-samplesto produce 24h blocks for out-of-order data belonging to previous days. #15892retriesfield to the "query stats" log line reporting the number of times requests were retried while processing the query. The value is 0 when all requests succeeded on their first attempt. #15929query-frontend.cardinality-sharding-max-sharded-queriesto optionally limit sharding forcardinality/active_seriesandcardinality/active_native_histogram_metricsendpoints separately fromquery-frontend.query-sharding-max-sharded-queries. #15922-ingest-storage.kafka.write-timeout-overheadto configure the overhead added on top of the Kafka write timeout (default 2s, unchanged). #16023ignoringor noon/ignoringclause. #15178-query-frontend.subquery-spin-off-simple-subqueries=truespins off subqueries whose inner expression was previously considered too simple to spin off and-query-frontend.subquery-spin-off-with-excess-downstream-queries=truespins off subqueries when the rewritten query contains more downstream queries than spun-off subqueries. All are disabled by default and require subquery spin-off to be enabled with-query-frontend.subquery-spin-off-enabled=true. #16211-<prefix>.keepalive-time(default20s)-<prefix>.keepalive-timeout(default10s)last_over_time(...[1s]), matching the existing float query behavior. #16163cardinality_analysis_max_resultsbeing ignored when set higher than the default of 500. #15581KafkaProducer.ProduceSync()returning a single result with a nil record when the context is canceled, instead of one result per input record (with the record set) as the underlying franz-go client does. #15199cortex_ingest_storage_reader_receive_delay_secondsinflation by no longer setting the Kafka recordTimestampon the distributor side; the Kafka client now sets it at produce time. #15572too_far_in_past). #15253MimirAutoscalerKedaFailingwhen queriers have no traffic becausecortex_querier_request_duration_seconds_sumis not published until the first request is received. #15106cortex_ingest_storage_writer_produce_records_enqueued_totalnot being incremented whenKafkaProducer.ProduceSync()rejects a batch because a record has itsTimestampset by the caller. #15610label=~"|foo|bar") incorrectly excluding series that don't have the label, potentially resulting in incomplete query results. #15767evaluation statslog, instead of logging it as successful. The querier still re-panics afterwards, crash behaviour is unchanged. #15753namelabel. #15794GatherBlockHealthStatspostings walk error check to prevent swallowing errors. #15895info()function incorrectly handling negated name matchers. #15168this indicates something has been returned to a pool more than oncepanic when asum()oravg()group contains, at the same output step, a float sample and native histograms that cannot be added together (e.g. exponential and custom bucket schemas). #16059env="foo"andenv="foobar"). #16036/api/v1/search/metric_names?include_metadata=truealmost never returning metric metadata. #16062, #16214-querier.query-ingesters-within, preventing query failures when the partition is still registered but has no available ingesters to serve the queries. #15721queue_time_secondsin the query stats log always reporting 0 when a query is cancelled while still waiting in the query-scheduler queue. #16094on()/on(...) group_left/group_rightjoin boundary, which could cause some queries to unexpectedly evaluate as an empty result. #16155mimir,mimirtool,metaconvert, andquery-teebinaries without the executable bit set, which causedmimir.serviceto fail to start. #16166Mixin
MimirIngesterKafkaReadFailednow fires aswarningafter 5m, and escalates tocriticalif it persists for 30m.MimirStrongConsistencyEnforcementFailedseverity changed fromcriticaltowarning, since queriers retry on a different ingester and fire their own alerts if that retry fails. #16019MimirInconsistentRuntimeConfigalert less flaky when performing multiple configuration changes in a row in a large Kubernetes cluster. #15257MimirBlockBuilderPersistentJobFailurelookback window to 20m to prevent the alert from flapping. #15332MimirRequestLatencyalert, distinguished by thehistogramlabel (classicornative). #15413MimirBucketIndexNotUpdatedto avoid false-positives when a compactor restarted near the end of the previous update cycle. #15935SearchLabelNames/SearchLabelValuesgRPC routes in the ingester panels of the "Reads", "Queries", and "Remote ruler reads" dashboards. #15571limitparameter of the/prometheus/api/v1/seriesendpoint to store-gateways, so they stop loading series (and their chunks) once the limit is reached instead of fetching all matching series and discarding the excess downstream. #15834TrackSeriesBatch. #15805cortex_ingest_storage_writer_produce_requests_totalandcortex_ingest_storage_writer_produce_failures_totalmetrics, which were renamed tocortex_ingest_storage_writer_produce_records_enqueued_totalandcortex_ingest_storage_writer_produce_records_failed_totalmore than a year ago. #16035show_multi_zone_write_path_panelsconfig option (disabled by default). When enabled, the gateway and distributor "Requests / sec" and "Kafka produced records / sec" panels break down the traffic by availability zone, and "Latency per zone" panels are added next to the aggregate latency panels. #16206unless on (job)filter againstcortex_partition_ring_partitionsnow also matches on the cluster aggregation labels. #15400MimirRulerInstanceHasNoRuleGroupsto not alert on false-positives when rulers are running in multiple zones. #16029Jsonnet
tests.write-read-series-test.num-seriesandtests.write-read-series-test.max-query-ageto their default values. #15705ephemeral_storage_request_sizeoption (set it tonullto disable). #15916-querier.max-concurrent, to 8. #15984_config.query_sharding_enabledtofalse. #16212compactor_scheduler_enabled: true. #15850autoscaling_compactor_enabled: true. When the compactor-scheduler is enabled, compactors are autoscaled based on the estimated time to drain the scheduler queue instead of CPU utilization. #15850newMimirtoolBlocksJoband subcommand-specific helpers to runmimirtool blocksas Kubernetes Jobs. #15757._config.commonConfigin arguments passed to continuous-test. #15988Documentation
MimirRingMembersMismatchrunbook: check for spurious ingesters left in the ring after a scale down. #16169Tools
mark-blocks,listblocks,copyblocks,splitblocks, andundelete-blockstools are now subcommands ofmimirtool blocks:mark,list,copy,split,undelete. #15757partition-ringsubcommands now accept an optional--partition-ring.keyflag to select the KV store key of the partition ring to operate on. It defaults toingester-partitions. #15719build-mixinandmixin-screenshotscan now be configured to use native histograms for latency panels in dashboards. #15269Configuration
📅 Schedule: (UTC)
🚦 Automerge: Enabled.
♻ Rebasing: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.
🔕 Ignore: Close this PR and you won't be reminded about this update again.
This PR was generated by Mend Renovate. View the repository job log.