Comment on page
Azure Cloud Billing Integration
Connecting your Azure account to Kubecost allows you to view Kubernetes metrics side-by-side with out-of-cluster (OOC) costs (e.g. Azure Database Services). Additionally, it allows Kubecost to reconcile measured Kubernetes spend with your actual Azure bill. This gives teams running Kubernetes a complete and accurate picture of costs. For more information, read Cloud Billing Integrations and this blog post.
To configure Kubecost's Azure Cloud Integration, you will need to set up daily exports of cost reports to Azure storage. Kubecost will then access your cost reports through the Azure Storage API to display your OOC cost data alongside your in-cluster costs.
Follow Azure's Create and Manage Exported Data tutorial to export cost reports. For Metric, make sure you select Amortized cost (Usage and Purchases). For Export type, make sure you select Daily export of month-to-date costs. Do not select File Partitioning. Also, take note of the Account name and Container specified when choosing where to export the data to. Note that a successful cost export will require
Microsoft.CostManagementExportsto be registered in your subscription.
It will take a few hours to generate the first report, after which Kubecost can use the Azure Storage API to pull that data.
Once the cost export has successfully executed, verify that a non-empty CSV file has been created at this path:
If you have sensitive data in an existing Azure Storage account, it is recommended to create a separate Azure Storage account to store your cost data export.
For more granular billing data it is possible to scope Azure cost exports to resource groups, management groups, departments, or enrollments. AKS clusters will create their own resource groups which can be used. This functionality can then be combined with Kubecost multi-cloud to ingest multiple scoped billing exports.
Obtain the following values from Azure to provide to Kubecost. These values can be located in the Azure Portal by selecting Storage Accounts, then selecting your specific Storage account for details.
azureSubscriptionIDis the "Subscription ID" belonging to the Storage account which stores your exported Azure cost report data.
azureStorageAccountis the name of the Storage account where the exported Azure cost report data is being stored.
azureStorageAccessKeycan be found by selecting Access keys in your Storage account left navigation under "Security + networking". Using either of the two keys will work.
azureStorageContaineris the name that you chose for the exported cost report when you set it up. This is the name of the container where the CSV cost reports are saved in your Storage account.
azureContainerPathis an optional value which should be used if there is more than one billing report that is exported to the configured container. The path provided should have only one billing export because Kubecost will retrieve the most recent billing report for a given month found within the path.
azureCloudis an optional value which denotes the cloud where the storage account exist, possible values are
gov. The default is
Next, create a JSON file which must be named cloud-integration.json with the following format:
Next, create the Secret:
$ kubectl create secret generic <SECRET_NAME> --from-file=cloud-integration.json -n kubecost
Next, ensure the following are set in your Helm values:
Next, upgrade Kubecost via Helm:
$ helm upgrade kubecost kubecost/cost-analyzer -n kubecost -f values.yaml
You can verify a successful configuration by checking the following in the Kubecost UI:
- The Assets dashboard will be broken down by Kubernetes assets.
- The Assets dashboard will no longer show a banner that says "External cloud cost not configured".
- The Diagnostics page (via Settings > View Full Diagnostics) view will show a green checkmark under Cloud Integrations.
If there are no in-cluster costs for a particular day, then there will not be out-of-cluster costs either
Kubecost utilizes Azure tagging to allocate the costs of Azure resources outside of the Kubernetes cluster to specific Kubernetes concepts, such as namespaces, pods, etc. These costs are then shown in a unified dashboard within the Kubecost interface.
To allocate external Azure resources to a Kubernetes concept, use the following tag naming scheme:
kubernetes_label_NAMEtag key, the NAME portion should appear exactly as the tag appears inside of Kubernetes. For example, for the tag
app.kubernetes.io/name, this tag key would appear as
To use an alternative or existing Azure tag schema, you may supply these in your values.yaml under the
kubecostProductConfigs.labelMappingConfigs.<aggregation>_external_label. Also be sure to set
kubecostProductConfigs.labelMappingConfigs.enabled = true
To troubleshoot a configuration that is not yet working:
$ kubectl get secrets -n kubecostto verify you've properly configured
$ helm get values kubecostto verify you've properly set
- Verify that a non-empty CSV file has been created at this path in your Azure Portal Storage Account:
<STORAGE_ACCOUNT>/<CONTAINER_NAME>/<OPTIONAL_CONTAINER_PATH>/<COST_EXPORT_NAME>/<DATE_RANGE>/<CSV_FILE>. Ensure new CSVs are being generated every day.
- When opening a cost report CSV, ensure that there are rows in the file that do not have a MeterCategory of “Virtual Machines” or “Storage” as these items are ignored because they are in cluster costs. Additionally, make sure that there are items with a UsageDateTime that matches the date you are interested in.
When reviewing logs:
- The following error is reflective of Kubecost's previous Azure Cloud Integration method and can be safely disregarded.
ERR Error, Failed to locate azure storage config file: /var/azure-storage-config/azure-storage-config.json