This document outlines how to set up cloud integration for accounts on multiple cloud providers, or multiple accounts on the same cloud provider. Multi-Cloud is an enterprise feature. This configuration can be used independently of or in addition to other cloud integration configurations provided by Kubecost. Once configured, Kubecost will display cloud assets for all configured accounts and perform reconciliation for all federated clusters that have their respective accounts configured.
For each Cloud Account that you would like to configure you will need to make sure that it is exporting cost data to its respective service to allow Kubecost to gain access to it.
The secret should contain a file named
cloud-integration.jsonwith the following format:
This method of Cloud-Integration supports multiple configurations per cloud provider simply by adding each cost export to their respective arrays in the .json file. The structure and required values for the configuration objects for each cloud provider are described below. Once you have filled in the configuration object use the command:
kubectl create secret generic <SECRET_NAME> --from-file=cloud-integration.json -n kubecost
Once the secret is created, set
.Values.kubecostProductConfigs.cloudIntegrationSecretto <SECRET_NAME> and upgrade Kubecost via Helm.
The following values can be located in the Azure Portal under Cost Management > Exports or Storage accounts:
azureSubscriptionIDis the "Subscription ID" belonging to the Storage account which stores your exported Azure cost report -ata.
azureStorageAccountis the name of the Storage account where the exported Azure cost report data is being stored.
azureStorageAccessKeycan be found by selecting the "Access Keys" option from the navigation sidebar then selecting "Show -eys". Using either of the two keys will work.
azureStorageContaineris the name that you chose for the exported cost report when you set it up. This is the name of the -ontainer where the CSV cost reports are saved in your Storage account.
azureContainerPathis an optional value which should be used if there is more than one billing report that is exported to the -onfigured container. The path provided should have only one billing export because kubecost will retrieve the most recent -illing report for a given month found within the path.
azureCloudis an optional value which denotes the cloud where the storage account exist, possible values are
gov. The default is
Set these values into the following object and add them to the Azure array:
If you don't already have a GCP service key for any of the projects you would like to configure, you can run the following commands in your command line to generate and export one. Make sure your gcloud project is where your external costs are being run.
export PROJECT_ID=$(gcloud config get-value project)
gcloud iam service-accounts create compute-viewer-kubecost --display-name "Compute Read Only Account Created For Kubecost" --format json
gcloud projects add-iam-policy-binding $PROJECT_ID --member serviceAccount:[email protected]$PROJECT_ID.iam.gserviceaccount.com --role roles/compute.viewer
gcloud projects add-iam-policy-binding $PROJECT_ID --member serviceAccount:[email protected]$PROJECT_ID.iam.gserviceaccount.com --role roles/bigquery.user
gcloud projects add-iam-policy-binding $PROJECT_ID --member serviceAccount:[email protected]$PROJECT_ID.iam.gserviceaccount.com --role roles/bigquery.dataViewer
gcloud projects add-iam-policy-binding $PROJECT_ID --member serviceAccount:[email protected]$PROJECT_ID.iam.gserviceaccount.com --role roles/bigquery.jobUser
gcloud iam service-accounts keys create ./compute-viewer-kubecost-key.json --iam-account [email protected]$PROJECT_ID.iam.gserviceaccount.com
You can then get your service account key to paste into the UI (be careful with this!):
<KEY_JSON>The GCP service key created above. This value should be left as JSON when inserted into the configuration object
<PROJECT_ID>GCP Project ID should match the Project ID in the GCP service key.
<BILLING_DATA_DATASET>BigQuery dataset requires a BigQuery dataset prefix (e.g. billing_data) in addition to the BigQuery table name. A full example is billing_data.gcp_billing_export_v1_018AIF_74KD1D_534A2.
Set these values into the following object and add it to the GCP array:
For each AWS account that you would like to configure, create an Access Key for the Kubecost user who has access to the CUR. Navigate to https://console.aws.amazon.com/iam Access Management > Users. Find the Kubecost user and select Security Credentials > Create Access Key. Note the Access key ID and Secret access key.
Gather each of these values from the AWS console for each account you would like to configure.
<ACCESS_KEY_ID>ID of the Access Key created in the previous step
<ACCESS_KEY_SECRET>Secret of the Access Key created in the
<ATHENA_BUCKET_NAME>An S3 bucket to store Athena query results that you’ve created that kubecost has permission to access The name of the bucket should match
s3://aws-athena-query-results-*, so the IAM roles defined above will automatically allow access to it The bucket can have a Canned ACL of Private or other permissions as you see fit.
<ATHENA_REGION>The AWS region Athena is running in
<ATHENA_DATABASE>the name of the database created by the Athena setup. The Athena database name is available as the value (physical id) of AWSCURDatabase in the CloudFormation stack created above (in Step 2: Setting up the Athena of the AWS guild above)
<ATHENA_TABLE>the name of the table created by the Athena setup The table name is typically the database name with the leading athenacurcfn_ removed (but is not available as a CloudFormation stack resource)
<ATHENA_WORKGROUP>The workgroup assigned to be used with Athena. If omitted, defaults to
<ATHENA_PROJECT_ID>e.g. "530337586277" # The AWS AccountID where the Athena CUR is.
<MASTER_PAYER_ARN>Is an optional value which should be set if you are using a multi-account billing set-up and are not accessing athena through the primary account. It should be set to the arn of the role in the masterpayer account, e.g.
Set these values into the following object and add them to the AWS array in the cloud-integration.json:
Additionally set the
kubecostProductConfigs.athenaProjectIDhelm value to the AWS account that Kubecost is being installed in.