AWS/Kubecost Data Audit

Performing a data audit

Node cost details
Next, compare this instance's costs with what is found in AWS Cost Explorer (Amortized). The most accessible way of doing so is Group by -> Resource, and Filters -> Resource -> i-033b92ecd18376946.
A node cost for one day in AWS Cost Explorer
If unable to Group by -> Resource (because it requires enabling hourly/resource granularity in Cost Explorer), try Group by -> Tag -> aws:eks:cluster-name, and Filters -> Tag -> aws:eks:cluster-name -> kc-demo-prod. This will compare the cost of a cluster in AWS Cost Explorer, versus the cost of a cluster in Kubecost.
Note: When grouping by cluster, AWS Cost Explorer only accounts for the Node and Network costs, not the ClusterManagement/ControlPlane, Disks, or LoadBalancer costs. Also keep in mind this method will also be less exact than comparing a specific node between Kubecost and AWS Cost Explorer.
Daily cost of an EKS cluster in AWS Cost Explorer
Kubecost Asset cost for an EKS cluster

Troubleshooting non-matching costs

To determine what could cause a discrepancy between your instances' cost and AWS Cost Explorer, perform these troubleshooting measures:
  • Investigate Prometheus to see if the underlying metrics about the node are sporadic or missing.
  • Cost Explorer CSV export for one day filtered by account and service in AWS. Compare that to the Kubecost /model/asset API request for the same day.
  • Set up CUR to export as CSV file. Investigate to see whether the costs in the CSV file match with what is displayed on the Kubecost Assets page.
  • Test Athena queries.