Kubecost Cloud AWS Integration
Kubecost Cloud provides the ability to allocate out of cluster (OOC) costs back to Kubernetes concepts like namespaces and deployments. The following guide provides the steps required for allocating OOC costs in AWS.
Before beginning your integration, you need to create a Cost and Usage Report (CUR) through AWS. Consult AWS' documentation Creating Cost and Usage Reports for step-by-step instructions if needed during this process. When creating your CUR, make sure to configure these settings:
- Time granularity is set to Daily
- Resource IDs and Athena are enabled
Remember the name of the S3 bucket that is created for this CUR. AWS may require up to 24 hours to publish data. Wait until you have received data before proceeding with this integration.
In the Kubecost Cloud UI, begin by selecting Settings in the left navigation. Scroll down to Cloud Integrations, then select View Additional Details. The Cloud Integrations dashboard opens. Select + Add Integration. Then, select AWS Integration from the slide panel.
If your CUR has been properly set up and is now providing data after following the on-screen instructions in the Kubecost UI, select Continue.
It's important to set up an Athena integration so Kubecost can perform reconciliation for providing accurate billing data. The on-screen instructions of the Kubecost Cloud UI are repeated here:
As part of the CUR creation process, Amazon also creates a CloudFormation template that is used to create the Athena integration. It is created in the CUR S3 bucket under
s3-path-prefix/cur-nameand typically has the filename crawler-cfn.yml. This .yml is your necessary CloudFormation template. You will need it in order to complete the CUR Athena integration. You can read more about this here.
Your S3 path prefix can be found by going to your AWS Cost and Usage Reports dashboard and selecting your bucket's report. In the Report details tab, you will find the S3 path prefix.
Once Athena is set up with the CUR, you will need to create a new S3 bucket for Athena query results:
- Select Create bucket. The Create Bucket page opens.
- Use the same region used for the CUR bucket and pick a name that follows the format
- Select Create bucket at the bottom of the page.
- Select Settings, then select Manage. The 'Manage settings' window opens.
- Set Location of query result to the S3 bucket you just created, then select Save.
When you have completed all the above steps, select Continue in the Kubecost Cloud UI.
Before continuing with the integration in the Kubecost Cloud UI, you need to set up IAM permissions in AWS.
- Select Create Stack, then select With existing resources (import resources) from the dropdown. On the 'Identify resources' page, select Next.
- Under Template source, choose Upload a template file.
- Select Choose file, which will open your file explorer. Select the .yaml template, and then select Open. Then, select Next.
- On the 'Identify resources' page, provide any additional resources to import. Then, select Next.
- For Stack name, enter a name for your template.
- Set the following parameters:
- MasterPayerAccountID: The account ID of the management account (formerly called master payer account) where the CUR has been created
- SpotDataFeedBucketName: Optional. The bucket where the Spot data feed is sent to
- Review all provided information, then select Import Resources.
- At the bottom of the page, select I acknowledge that AWS CloudFormation might create IAM resources.
- Select Create Stack.
You will be prompted to provide values for several different fields to finalize your integration. See this table for working definitions of each field:
When you have provided all mandatory fields, select Create Integration to finalize. Be patient while your integration is set up. The Status should initially display as Unknown. This is normal. You should eventually see the integration's Status change from Pending to Successful.