Search…
⌃K
Links

Install Kubecost on Redhat OpenShift

Architecture overview:

Currently, there are two main options to deploy Kubecost on Red Hat OpenShift Cluster (OCP).

Standard deployment:

Kubecost is installed with Cost Analyzer and Prometheus as a time-series database. Data is gathered by the Prometheus installed with Kubecost (bundled Prometheus). Other metrics are scraped by bundled Prometheus from OCP monitoring stack managed components like Kube State Metrics (KSM), Openshift Service Mesh (OSM), CAdvisor, etc .... Kubecost then pushes and queries metrics to/from bundled Prometheus. Enterprise setup could also work with Thanos as an additional component.
The standard deployment is illustrated in the following diagram:
Standard deployment

Grafana managed Prometheus deployment:

Kubecost is installed with the core components only (cost model, frontend) without bundled Prometheus and other components. Grafana agent is installed as part of the solution to scrape the metrics from OCP monitoring stack managed components and Kubecost /metrics endpoint to write the data back to the Grafana Cloud managed Prometheus (Grafana Prometheus) instance. Kubecost reads the metrics directly from Grafana managed Prometheus.
The Grafana managed Prometheus deployment is illustrated in the following diagram:
Grafana managed Prometheus deployment

Standard deployment guide

Prerequisites:

  • You have an existing OpenShift cluster.
  • You have appropriate access to that OpenShift cluster to create a new project and deploy new workloads.

Installation:

Run the following Helm install command to install Kubecost:
helm upgrade --install kubecost \
--repo https://raw.githubusercontent.com/kubecost/openshift-helm-chart/main cost-analyzer \
--namespace kubecost --create-namespace \
-f https://raw.githubusercontent.com/kubecost/openshift-helm-chart/main/values-openshift.yaml
If you want to install Kubecost with your desired cluster name, you can use the following commands:
Note: Remember to replace CLUSTER_ID's value by your desired value
export CLUSTER_ID="CLUSTER_OCP"
helm upgrade --install kubecost \
--repo https://raw.githubusercontent.com/kubecost/openshift-helm-chart/main cost-analyzer \
--namespace kubecost --create-namespace \
-f https://raw.githubusercontent.com/kubecost/openshift-helm-chart/main/values-openshift.yaml \
--set kubecostProductConfigs.clusterName=${CLUSTER_ID} \
--set prometheus.server.global.external_labels.cluster_id=${CLUSTER_ID}
Wait for all pods to be ready.
Create a route to the service kubecost-cost-analyzer on port 9090 of the kubecost project. You can learn more about how to do it on your OpenShift portal in this LINK
Kubecost will be collecting data, please wait 5-15 minutes for the UI to reflect the resources in the local cluster.

Grafana managed Prometheus deployment:

Prerequisites:

  • You have created a Grafana Cloud account and you have permissions to create Grafana Cloud API keys
  • Add required service account for grafana-agent to hostmount-anyuid SCC:
oc adm policy add-scc-to-user hostmount-anyuid system:serviceaccount:kubecost:grafana-agent

Installation:

Step 1: Install the Grafana Agent on your cluster.

On the existing K8s cluster that you intend to install Kubecost, run the following commands to install Grafana agent to scrape the metrics from Kubecost /metrics endpoint. The script below installs Grafana agent with the necessary scraping configuration for Kubecost, you may want to add an additional scrape configuration for your setup. Please remember to replace the following values with your actual Grafana cloud's values:
  • REPLACE-WITH-GRAFANA-PROM-REMOTE-WRITE-ENDPOINT
  • REPLACE-WITH-GRAFANA-PROM-REMOTE-WRITE-USERNAME
  • REPLACE-WITH-GRAFANA-PROM-REMOTE-WRITE-API-KEY
  • REPLACE-WITH-YOUR-CLUSTER-NAME
Click to see code
To learn more about how to install and config Grafana agent as well as additional scrape configuration, please refer to Grafana Agent for Kubernetes section of the Grafana Cloud documentation. Or you can check Kubecost Prometheus scrape config at this Github repository

Step 2: Verify if grafana-agent is scraping data successfully.

kubectl -n kubecost logs grafana-agent-0

Step 3: Create dbsecret to allow Kubecost to query the metrics from Grafana Cloud Prometheus.

  • Create two files in your working directory, called USERNAME and PASSWORD respectively
export PASSWORD=<REPLACE-WITH-GRAFANA-PROM-REMOTE-WRITE-API-KEY>
export USERNAME=<REPLACE-WITH-GRAFANA-PROM-REMOTE-WRITE-USERNAME>
printf "${PASSWORD}" > PASSWORD
printf "${USERNAME}" > USERNAME
  • Verify that you can run query against your Grafana Cloud Prometheus query endpoint with your API key (Optional):
cred="$( echo $NAME:$PASSWORD | base64 )"; curl -H "Authorization: Basic $cred" https://<REPLACE-WITH-GRAFANA-PROM-QUERY-ENDPOINT>/api/v1/query?query=up
  • Create K8s secret name dbsecret:
kubectl create secret generic dbsecret \
--namespace kubecost \
--from-file=USERNAME \
--from-file=PASSWORD
  • Verify if the credentials appears correctly - Optional (Any trailing space or new line etc ...)
kubectl -n kubecost get secret dbsecret -o json | jq '.data | map_values(@base64d)'

Step 4 (optional): Configure Kubecost recording rules for Grafana Cloud using cortextool.

To set up recording rules in Grafana Cloud, download the cortextool CLI utility. While they are optional, they offer improved performance.
After installing the tool, create a file called kubecost-rules.yaml with the following command:
Click to see code
Make you are in the same directory as your kubecost-rules.yaml, then load the rules using cortextool. Replace the address with your Grafana Cloud’s Prometheus endpoint (Remember to omit the /api/prom path from the endpoint URL).
cortextool rules load \
--address=<REPLACE-WITH-GRAFANA-PROM-ENDPOINT> \
--id=<REPLACE-WITH-GRAFANA-PROM-REMOTE-WRITE-USERNAME> \
--key=<REPLACE-WITH-GRAFANA-PROM-REMOTE-WRITE-API-KEY> \
kubecost-rules.yaml
Print out the rules to verify that they’ve been loaded correctly:
cortextool rules print \
--address=<REPLACE-WITH-GRAFANA-PROM-ENDPOINT> \
--id=<REPLACE-WITH-GRAFANA-PROM-REMOTE-WRITE-USERNAME> \
--key=<REPLACE-WITH-GRAFANA-PROM-REMOTE-WRITE-API-KEY>

Step 5: Install Kubecost on the cluster.

Install Kubecost on your K8s cluster with Grafana Cloud Prometheus query endpoint and dbsecret you created in Step 4
Note: Remember to replace CLUSTER_ID's value by your desired value
export CLUSTER_ID="CLUSTER_OCP"
# Replace REPLACE-WITH-GRAFANA-PROM-QUERY-ENDPOINT with your Grafana cloud value. Example: https://prometheus-prod-10-prod-us-central-0.grafana.net/api/prom/
export GRAFANA_QUERY_ENDPOINT="REPLACE-WITH-GRAFANA-PROM-QUERY-ENDPOINT"
helm upgrade --install kubecost \
--repo https://raw.githubusercontent.com/kubecost/openshift-helm-chart/main cost-analyzer \
--namespace kubecost --create-namespace \
-f https://raw.githubusercontent.com/kubecost/openshift-helm-chart/main/values-openshift.yaml \
--set kubecostProductConfigs.clusterName=${CLUSTER_ID} \
--set prometheus.server.global.external_labels.cluster_id=${CLUSTER_ID} \
--set kubecostModel.promClusterIDLabel=cluster \
--set global.prometheus.fqdn=${GRAFANA_QUERY_ENDPOINT} \
--set global.prometheus.enabled=false \
--set global.prometheus.queryServiceBasicAuthSecretName=dbsecret
The process is complete. By now, you should have successfully completed the Kubecost integration with Grafana Cloud.
Optionally, you can also add our Kubecost Dashboard for Grafana Cloud to your organization to visualize your cloud costs in Grafana.

Clean up

You can uninstall Kubecost from your cluster with the following command.
helm uninstall kubecost --namespace kubecost
The process is complete. By now, you should have successfully completed the Kubecost integration with Grafana Cloud.
Optionally, you can also add our Kubecost Dashboard for Grafana Cloud to your organization to visualize your cloud costs in Grafana.

Support

For advanced setup or if you have any questions, you can contact us on Slack or email us at [email protected].