The ContainerStats pipeline builds statistical representations of individual containers' resource usage over time. The pipeline is part of the
Ensure you allow ~2hrs for the pipeline to run before issuing a query which leverages this pipeline
The pipeline builds 24 hour "windows" of data. It only builds complete windows, e.g. if now is
2003-08-23T08:30:00Z, the pipeline will only build up to the window from
The pipeline will return an error response if a requested time range of data contains any windows (24 hour chunks) are expected (should be in the store) but not available (the pipeline has not yet built and loaded a complete set of data into the store).
The primary user of ContainerStats pipeline data is v2 of the request right- sizing recommendation API. ContainerStats data is used for quantile-based recommendations. Review the doc for this feature here.
There is an API for introspecting pipeline data available at
/model/containerstats/quantiles. It does not have a stable schema and is not supported as an official product feature. It is only intended for limited debugging.
All ContainerStats-related log messages should contain
ContainerStatsSet. The pipeline logs a few things at
INFOlevel to show that the pipeline is running successfully. Much greater detail is available at the
DEBUGlevel. See the official instructions to learn how to change the log level.
The ContainerStats pipeline's behavior is controller by a few different environment variables.
Helm chart value
Enables the pipeline.
The pipeline inherits most of the existing Prometheus/Thanos configuration because it leverages the same client(s) used by the Asset and Allocation pipelines. Specific deviations will be mentioned.
The pipeline will obey this, but may fail to initialize if this is set below the minimum value supported by the pipeline (10 minutes).
The pipeline inherits most of the existing "store" configuration used by other pipelines like Asset and Allocation. This includes, but is not limited to: store duration, store type (file, federated, etc.), leader election, storage pathing, storage directory, bucket storage, and backup.