See also
Overview
Using AMP allows multi-cluster Kubecost with EKS-Optimized licenses.
This guide will walk you through the steps to deploy Kubecost with AWS Agentless AMP to collect metrics from your Kubernetes cluster.
Keep in mind that "agentless" refers to the Prometheus scraper, not the Kubecost agent. The Kubecost agent is still required to collect metrics from the cluster.
The guide below assumes a multi-cluster setup will be used, which is supported with the EKS-Optimized license that is enabled by following the below guide.
Prerequisites
Follow this Using an AWS managed collector guide to enable the managed collector.
This guide assumes that the Kubecost Helm release name and the Kubecost namespace are equal, which allows a global find-and-replace on $KUBECOST_NAMESPACE
.
Architecture diagram
Agentless AMP Configuration
AMP setup
Clone this poc-common-configurations repository that contains all of the configuration files you will need to deploy Kubecost with AWS Agentless AMP.
Copy git clone https://github.com/kubecost/poc-common-configurations.git
cd poc-common-configurations/aws/amp-agentless
Update all configuration files with your cluster name (replace all YOUR_CLUSTER_NAME_HERE
).
Build the configuration variables:
Copy CLUSTER_NAME = YOUR_CLUSTER_NAME_HERE
CLUSTER_REGION = us-east-2
KUBECOST_NAMESPACE = kubecost
WORKSPACE_ID = ws-YOUR_WORKSPACE_ID
AWS_ACCOUNT_ID = 11111111111
WORKSPACE_ARN = $( aws amp describe-workspace --workspace-id $WORKSPACE_ID --output json | jq -r .workspace.arn )
CLUSTER_JSON = $( aws eks describe-cluster --name $CLUSTER_NAME --region $CLUSTER_REGION --output json )
CLUSTER_ARN = $( echo $CLUSTER_JSON | jq -r .cluster.arn )
SECURITY_GROUP_IDS = $( echo $CLUSTER_JSON | jq -r .cluster.resourcesVpcConfig.clusterSecurityGroupId )
SUBNET_IDS = $( echo $CLUSTER_JSON | jq -r '.cluster.resourcesVpcConfig.subnetIds | @csv' )
Create the Kubecost scraper:
Copy KUBECOST_SCRAPER_OUTPUT = $( aws amp create-scraper --output json \
--alias kubecost-scraper \
--source eksConfiguration="{clusterArn=$CLUSTER_ARN, securityGroupIds=[$SECURITY_GROUP_IDS],subnetIds=[$SUBNET_IDS]}" \
--scrape-configuration configurationBlob= "$( base64 scraper-kubecost-with-networking.yaml | tr -d '\n')" \
--destination ampConfiguration= "{workspaceArn=$WORKSPACE_ARN}" )
echo $KUBECOST_SCRAPER_OUTPUT
KUBECOST_SCRAPER_ID = $( echo $KUBECOST_SCRAPER_OUTPUT | jq -r .scraperId )
echo $KUBECOST_SCRAPER_ID
Get the ARN of the scraper:
Copy ARN_PART=$(aws amp describe-scraper --output json --region $CLUSTER_REGION --scraper-id $KUBECOST_SCRAPER_ID | jq -r .scraper.roleArn | cut -d'_' -f2)
ROLE_ARN_KUBECOST_SCRAPER = "arn:aws:iam::$AWS_ACCOUNT_ID:role/AWSServiceRoleForAmazonPrometheusScraper_$ARN_PART"
echo $ROLE_ARN_KUBECOST_SCRAPER
Add the ARN of the scraper to the kube-system/aws-auth
configMap:
Copy eksctl create iamidentitymapping \
--cluster $CLUSTER_NAME --region $CLUSTER_REGION \
--arn $ROLE_ARN_KUBECOST_SCRAPER \
--username aps-collector-user
Create a scraper for cAdvisor and node exporter. Node exporter is optional. cAdvisor is required, but may already be available.
Copy CADVSIOR_SCRAPER_OUTPUT = $( aws amp create-scraper --output json \
--alias cadvisor-scraper \
--source eksConfiguration="{clusterArn=$CLUSTER_ARN, securityGroupIds=[$SECURITY_GROUP_IDS],subnetIds=[$SUBNET_IDS]}" \
--scrape-configuration configurationBlob= "$( base64 scraper-cadvisor-node-exporter.yaml | tr -d '\n')" \
--destination ampConfiguration= "{workspaceArn=$WORKSPACE_ARN}" )
echo $CADVSIOR_SCRAPER_OUTPUT
CADVSIOR_SCRAPER_ID = $( echo $CADVSIOR_SCRAPER_OUTPUT | jq -r .scraperId )
echo $CADVSIOR_SCRAPER_ID
Get the ARN of the scraper:
Copy ARN_PART=$(aws amp describe-scraper --output json --region $CLUSTER_REGION --scraper-id $CADVSIOR_SCRAPER_ID | jq -r .scraper.roleArn | cut -d'_' -f2)
ROLE_ARN_CADVSIOR_SCRAPER = "arn:aws:iam::$AWS_ACCOUNT_ID:role/AWSServiceRoleForAmazonPrometheusScraper_$ARN_PART"
echo $ROLE_ARN_CADVSIOR_SCRAPER
Add the ARN of the scraper to the kube-system/aws-auth configmap:
Copy eksctl create iamidentitymapping \
--cluster $CLUSTER_NAME --region $CLUSTER_REGION \
--arn $ROLE_ARN_CADVSIOR_SCRAPER \
--username aps-collector-user
Apply the agentless RBAC permissions:
Copy kubectl apply -f rbac.yaml
Kubecost primary cluster installation
Create the Kubecost namespace:
Copy kubectl create ns $KUBECOST_NAMESPACE
Create the AWS IAM policy to allow Kubecost to query metrics from AMP:
Copy aws iam create-policy --policy-name kubecost-read-amp-metrics --policy-document file://iam-read-amp-metrics.json
(Optional) Create the AWS IAM policy to allow Kubecost to find savings in the AWS Account:
Copy aws iam create-policy --policy-name DescribeResources --policy-document file://iam-describeCloudResources.json
(Optional) Create the AWS IAM policy to allow Kubecost to write to find account-level tags:
Copy aws iam create-policy --policy-name OrganizationListAccountTags --policy-document file://iam-listAccounts-tags.json
Configure the Kubecost Service Account:
Copy eksctl create iamserviceaccount \
--name kubecost-sa \
--namespace $KUBECOST_NAMESPACE \
--cluster $CLUSTER_NAME --region $CLUSTER_REGION \
--attach-policy-arn arn:aws:iam:: $AWS_ACCOUNT_ID :policy/kubecost-read-amp-metrics \
--attach-policy-arn arn:aws:iam:: $AWS_ACCOUNT_ID :policy/OrganizationListAccountTags \
--attach-policy-arn arn:aws:iam:: $AWS_ACCOUNT_ID :policy/DescribeResources \
--override-existing-serviceaccounts --approve
Update the placeholder values such as YOUR_CLUSTER_NAME_HERE
in values-kubecost-primary.yaml .
Install Kubecost on your primary:
Copy aws ecr-public get-login-password --region us-east-1 | helm registry login --username AWS --password-stdin public.ecr.aws
helm install $KUBECOST_NAMESPACE -n $KUBECOST_NAMESPACE \
oci://public.ecr.aws/kubecost/cost-analyzer \
-f https://raw.githubusercontent.com/kubecost/cost-analyzer-helm-chart/develop/cost-analyzer/values-eks-cost-monitoring.yaml \
-f values-kubecost-primary.yaml
Kubecost agents installation
Follow the above AMP setup section to configure the scraper(s) on each cluster.
This assumes you have created the AWS IAM policies above. If using multiple AWS accounts, you will need to create the policies in each account.
Update the placeholder values such as YOUR_CLUSTER_NAME_HERE
in values-kubecost-agent.yaml .
Create the Kubecost namespace:
Copy kubectl create ns $KUBECOST_NAMESPACE
Configure the Kubecost Service Account:
Copy eksctl create iamserviceaccount \
--name kubecost-sa \
--namespace $KUBECOST_NAMESPACE \
--cluster $CLUSTER_NAME --region $CLUSTER_REGION \
--attach-policy-arn arn:aws:iam:: $AWS_ACCOUNT_ID :policy/kubecost-read-amp-metrics \
--attach-policy-arn arn:aws:iam:: $AWS_ACCOUNT_ID :policy/OrganizationListAccountTags \
--attach-policy-arn arn:aws:iam:: $AWS_ACCOUNT_ID :policy/DescribeResources \
--override-existing-serviceaccounts --approve
Deploy the Kubecost agent:
Copy aws ecr-public get-login-password --region us-east-1 | helm registry login --username AWS --password-stdin public.ecr.aws
helm install $KUBECOST_NAMESPACE -n $KUBECOST_NAMESPACE \
oci://public.ecr.aws/kubecost/cost-analyzer \
-f https://raw.githubusercontent.com/kubecost/cost-analyzer-helm-chart/develop/cost-analyzer/values-eks-cost-monitoring.yaml \
-f values-kubecost-agent.yaml
Troubleshooting
It will take a few minutes for the scrapers start.
For more help troubleshooting, see our Amazon Managed Service for Prometheus (AMP) Overview doc.
Last updated 5 months ago