Multi-cloud integrations are only officially supported on Kubecost Enteprise plans.
This document outlines how to set up cloud integration for accounts on multiple cloud service providers (CSPs), or multiple accounts on the same cloud provider. This configuration can be used independently of, or in addition, to other cloud integration configurations provided by Kubecost. Once configured, Kubecost will display cloud assets for all configured accounts and perform reconciliation for all federated clusters that have their respective accounts configured.
For each cloud account that you would like to configure, you will need to make sure that it is exporting cost data to its respective service to allow Kubecost to gain access to it.
The secret should contain a file named cloud-integration.json with the following format (only containing applicable CSPs in your setup):
This method of cloud integration supports multiple configurations per cloud provider simply by adding each cost export to their respective arrays in the .json file. The structure and required values for the configuration objects for each cloud provider are described below. Once you have filled in the configuration object, use the command:
kubectl create secret generic <SECRET_NAME> --from-file=cloud-integration.json -n kubecost
Once the secret is created, set
<SECRET_NAME>and upgrade Kubecost via Helm.
The following values can be located in the Azure Portal under Cost Management > Exports, or Storage accounts:
azureSubscriptionIDis the Subscription ID belonging to the Storage account which stores your exported Azure cost report data.
azureStorageAccountis the name of the Storage account where the exported Azure cost report data is being stored.
azureStorageAccessKeycan be found by selecting Access Keys from the navigation sidebar then selecting Show keys. Using either of the two keys will work.
azureStorageContaineris the name that you chose for the exported cost report when you set it up. This is the name of the container where the CSV cost reports are saved in your Storage account.
azureContainerPathis an optional value which should be used if there is more than one billing report that is exported to the configured container. The path provided should have only one billing export because Kubecost will retrieve the most recent billing report for a given month found within the path.
azureCloudis an optional value which denotes the cloud where the storage account exists. Possible values are
gov. The default is
Set these values into the following object and add them to the Azure array:
If you don't already have a GCP service key for any of the projects you would like to configure, you can run the following commands in your command line to generate and export one. Make sure your GCP project is where your external costs are being run.
export PROJECT_ID=$(gcloud config get-value project)
gcloud iam service-accounts create compute-viewer-kubecost --display-name "Compute Read Only Account Created For Kubecost" --format json
gcloud projects add-iam-policy-binding $PROJECT_ID --member serviceAccount:compute-viewer-kubecost@$PROJECT_ID.iam.gserviceaccount.com --role roles/compute.viewer
gcloud projects add-iam-policy-binding $PROJECT_ID --member serviceAccount:compute-viewer-kubecost@$PROJECT_ID.iam.gserviceaccount.com --role roles/bigquery.user
gcloud projects add-iam-policy-binding $PROJECT_ID --member serviceAccount:compute-viewer-kubecost@$PROJECT_ID.iam.gserviceaccount.com --role roles/bigquery.dataViewer
gcloud projects add-iam-policy-binding $PROJECT_ID --member serviceAccount:compute-viewer-kubecost@$PROJECT_ID.iam.gserviceaccount.com --role roles/bigquery.jobUser
gcloud iam service-accounts keys create ./compute-viewer-kubecost-key.json --iam-account compute-viewer-kubecost@$PROJECT_ID.iam.gserviceaccount.com
You can then get your service account key to paste into the UI:
<KEY_JSON>is the GCP service key created above. This value should be left as a JSON when inserted into the configuration object
<PROJECT_ID>is the Project ID in the GCP service key.
<BILLING_DATA_DATASET>requires a BigQuery dataset prefix (e.g.
billing_data) in addition to the BigQuery table name. A full example is
Set these values into the following object and add it to the GCP array:
For each AWS account that you would like to configure, create an Access Key for the Kubecost user who has access to the CUR. Navigate to IAM Management Console dashboard, and select Access Management > Users. Find the Kubecost user and select Security Credentials > Create Access Key. Note the Access Key ID and Secret access key.
Gather each of these values from the AWS console for each account you would like to configure.
<ACCESS_KEY_ID>is the ID of the Access Key created in the previous step.
<ACCESS_KEY_SECRET>is the secret of the Access Key created in the
<ATHENA_BUCKET_NAME>is the S3 bucket storing Athena query results which Kubecost has permission to access. The name of the bucket should match
s3://aws-athena-query-results-*, so the IAM roles defined above will automatically allow access to it. The bucket can have a canned ACL set to Private or other permissions as needed.
<ATHENA_REGION>is the AWS region Athena is running in
<ATHENA_DATABASE>is the name of the database created by the Athena setup. The Athena database name is available as the value (physical id) of
AWSCURDatabasein the CloudFormation stack created above.
<ATHENA_TABLE>is the name of the table created by the Athena setup The table name is typically the database name with the leading
athenacurcfn_removed (but is not available as a CloudFormation stack resource).
<ATHENA_WORKGROUP>is the workgroup assigned to be used with Athena. Default value is
<ATHENA_PROJECT_ID>is the AWS AccountID where the Athena CUR is. For example:
<MASTER_PAYER_ARN>is an optional value which should be set if you are using a multi-account billing set-up and are not accessing Athena through the primary account. It should be set to the ARN of the role in the management (formerly master payer) account, for example:
Set these values into the following object and add them to the AWS array in the cloud-integration.json:
Additionally set the
kubecostProductConfigs.athenaProjectIDHelm value to the AWS account that Kubecost is being installed in.
Kubecost does not support complete integrations with Alibaba, but you will still be able to view accurate list prices for cloud resources. Gather these following values from the Alibaba Cloud Console for your account:
clusterRegionis the most used region
accountIDis your Alibaba account ID
serviceKeyNameis the RAM user key name
serviceKeySecretis the RAM user secret
Set these values into the following object and add them to the Alibaba array in your cloud-integration.json:
"alibaba" : [