AWS Cloud Integration

Kubecost Cloud provides the ability to allocate out of cluster (OOC) costs back to Kubernetes concepts like namespaces and deployments. The following guide provides the steps required for allocating OOC costs in AWS.

Prerequisites

Before beginning your integration, you need to create a Cost and Usage Report (CUR) through AWS. Consult AWS' documentation Creating Cost and Usage Reports for step-by-step instructions if needed during this process. When creating your CUR, make sure to configure these settings:

• Time granularity is set to Daily

• Resource IDs and Athena are enabled

Remember the name of the S3 bucket that is created for this CUR. AWS may require up to 24 hours to publish data. Wait until you have received data before proceeding with this integration.

Adding an integration

In the Kubecost Cloud UI, begin by selecting Settings in the left navigation. Scroll down to Cloud Integrations, then select View Additional Details. The Cloud Integrations dashboard opens. Select + Add Integration. Then, select AWS Integration from the slide panel.

Step 1: Setting up a CUR

If your CUR has been properly set up and is now providing data after following the on-screen instructions in the Kubecost UI, select Continue.

Step 2: Setting up Athena

It's important to set up an Athena integration so Kubecost can perform reconciliation for providing accurate billing data. The on-screen instructions of the Kubecost Cloud UI are repeated here:

As part of the CUR creation process, Amazon also creates a CloudFormation template that is used to create the Athena integration. It is created in the CUR S3 bucket under s3-path-prefix/cur-name and typically has the filename crawler-cfn.yml. This .yml is your necessary CloudFormation template. You will need it in order to complete the CUR Athena integration. You can read more about this here.

Your S3 path prefix can be found by going to your AWS Cost and Usage Reports dashboard and selecting your bucket's report. In the Report details tab, you will find the S3 path prefix.

Once Athena is set up with the CUR, you will need to create a new S3 bucket for Athena query results:

• Navigate to the S3 Management Console.

• Select Create bucket. The Create Bucket page opens.

• Use the same region used for the CUR bucket and pick a name that follows the format aws-athena-query-results-*.

• Select Create bucket at the bottom of the page.

• Navigate to the Amazon Athena Dashboard.

• Select Settings, then select Manage. The 'Manage settings' window opens.

• Set Location of query result to the S3 bucket you just created, then select Save.

When you have completed all the above steps, select Continue in the Kubecost Cloud UI.

Step 3: Setting up IAM permissions

Before continuing with the integration in the Kubecost Cloud UI, you need to set up IAM permissions in AWS.

Begin by downloading this .yaml template.

Then, navigate to the AWS Console Cloud Formation page.

• Select Create Stack, then select With existing resources (import resources) from the dropdown. On the 'Identify resources' page, select Next.

• Under Template source, choose Upload a template file.

• Select Choose file, which will open your file explorer. Select the .yaml template, and then select Open. Then, select Next.

• On the 'Identify resources' page, provide any additional resources to import. Then, select Next.

• For Stack name, enter a name for your template.

• Set the following parameters:

  • MasterPayerAccountID: The account ID of the management account (formerly called master payer account) where the CUR has been created

  • SpotDataFeedBucketName: Optional. The bucket where the Spot data feed is sent to

• Review all provided information, then select Import Resources.

• At the bottom of the page, select I acknowledge that AWS CloudFormation might create IAM resources.

• Select Create Stack.

Step 4: Provide CUR config values to Kubecost

You will be prompted to provide values for several different fields to finalize your integration. See this table for working definitions of each field:

Field	Description
AWS Account ID	The AWS account ID where the Athena CUR is, likely your management account.
Master Payer ARN	Also known as the management account ARN. Configured in Step 3. The account ID of the management account where the CUR has been created.
Region	The AWS region Athena is running in
Bucket	An S3 bucket to store Athena query results that you’ve created that Kubecost has permission to access. The name of the bucket should match s3://aws-athena-query-results-*
Database	The name of the database created by the Athena setup
Table	The name of the table created by the Athena setup
Workgroup	Optional. Primary workgroup associated with the AWS account where your Athena CUR is.
Access Key Id	In the AWS IAM Console, select Asset Management > Users. Find your user and select Security credentials > Create access key.
Secret Access Key	Use the Access Key associated with the Access Key ID above.

When you have provided all mandatory fields, select Create Integration to finalize. Be patient while your integration is set up. The Status should initially display as Unknown. This is normal. You should eventually see the integration's Status change from Pending to Successful.