aj-geddes / cloud-cost-management

Skill Content

---
name: cloud-cost-management
description: Optimize and manage cloud costs across AWS, Azure, and GCP using reserved instances, spot pricing, and cost monitoring tools.
---

# Cloud Cost Management

## Overview

Cloud cost management involves monitoring, analyzing, and optimizing cloud spending. Implement strategies using reserved instances, spot pricing, proper sizing, and cost allocation to maximize ROI and prevent budget overruns.

## When to Use

- Reducing cloud infrastructure costs
- Optimizing compute spending
- Managing database costs
- Storage optimization
- Data transfer cost reduction
- Reserved capacity planning
- Chargeback and cost allocation
- Budget forecasting and alerts

## Implementation Examples

### 1. **AWS Cost Optimization with AWS CLI**

```bash
# Enable Cost Explorer
aws ce get-cost-and-usage \
  --time-period Start=2024-01-01,End=2024-01-31 \
  --granularity MONTHLY \
  --metrics "UnblendedCost" \
  --group-by Type=DIMENSION,Key=SERVICE

# List EC2 instances for right-sizing
aws ec2 describe-instances \
  --query 'Reservations[*].Instances[*].[InstanceId,InstanceType,State.Name,LaunchTime,Tag]' \
  --output table

# Find unattached EBS volumes
aws ec2 describe-volumes \
  --filters Name=status,Values=available \
  --query 'Volumes[*].[VolumeId,Size,State,CreateTime]'

# Identify unattached Elastic IPs
aws ec2 describe-addresses \
  --query 'Addresses[?AssociationId==null]'

# Get RDS instance costs
aws rds describe-db-instances \
  --query 'DBInstances[*].[DBInstanceIdentifier,DBInstanceClass,StorageType,AllocatedStorage]'

# Create budget alert
aws budgets create-budget \
  --account-id 123456789012 \
  --budget BudgetName=MyBudget,BudgetLimit='{Amount=1000,Unit=USD}',TimeUnit=MONTHLY,BudgetType=COST \
  --notifications-with-subscribers \
    'Notification={NotificationType=ACTUAL,ComparisonOperator=GREATER_THAN,Threshold=80},Subscribers=[{SubscriptionType=EMAIL,Address=user@example.com}]'

# List savings plans
aws savingsplans describe-savings-plans

# Get reserved instances
aws ec2 describe-reserved-instances \
  --query 'ReservedInstances[*].[ReservedInstancesId,InstanceType,State,OfferingType,Duration]'
```

### 2. **Terraform Cost Management Configuration**

```hcl
# cost-management.tf
terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  region = var.aws_region
}

variable "monthly_budget" {
  default = 10000
  description = "Monthly budget limit"
}

# CloudWatch Cost Anomaly Detection
resource "aws_ce_anomaly_monitor" "cost_anomaly" {
  monitor_name    = "cost-anomaly-detection"
  monitor_type    = "DIMENSIONAL"
  monitor_dimension = "SERVICE"
  monitor_specification = jsonencode({
    Dimensions = {
      Key          = "SERVICE"
      Values       = ["Amazon EC2", "Amazon RDS", "AWS Lambda"]
    }
  })
}

# Anomaly alert
resource "aws_ce_anomaly_subscription" "cost_alert" {
  account_id    = data.aws_caller_identity.current.account_id
  display_name  = "Cost Alert"
  threshold     = 100
  frequency     = "DAILY"
  monitor_arn   = aws_ce_anomaly_monitor.cost_anomaly.arn
  subscription_type = "EMAIL"

  subscription_notification_type = "FORECASTED"
}

# Budget with alerts
resource "aws_budgets_budget" "monthly" {
  name              = "monthly-budget"
  budget_type       = "COST"
  limit_amount      = var.monthly_budget
  limit_unit        = "USD"
  time_period_start = "2024-01-01_00:00"
  time_period_end   = "2099-12-31_23:59"
  time_unit         = "MONTHLY"

  tags = {
    Name = "monthly-budget"
  }
}

# Budget notification
resource "aws_budgets_budget_notification" "monthly_alert" {
  account_id      = data.aws_caller_identity.current.account_id
  budget_name     = aws_budgets_budget.monthly.name
  comparison_operator = "GREATER_THAN"
  notification_type   = "ACTUAL"
  threshold       = 80
  threshold_type  = "PERCENTAGE"

  notification_subscribers {
    address              = "user@example.com"
    subscription_type    = "EMAIL"
  }
}

# Savings Plan Commitment
resource "aws_savingsplans_savings_plan" "compute" {
  commitment  = 10000
  payment_option = "ALL_UPFRONT"
  plan_type   = "COMPUTE_SAVINGS_PLAN"
  term_in_months = 12

  tags = {
    Name = "compute-savings"
  }
}

# Reserved Instances
resource "aws_ec2_instance" "app" {
  ami           = "ami-0c55b159cbfafe1f0"
  instance_type = "t3.medium"

  tags = {
    Name = "app-instance"
  }
}

# Reserve the instance
resource "aws_ec2_capacity_reservation" "app" {
  availability_zone       = "us-east-1a"
  instance_count          = 1
  instance_platform       = "Linux/UNIX"
  instance_type           = aws_ec2_instance.app.instance_type
  reservation_type        = "default"

  tags = {
    Name = "app-reservation"
  }
}

# CloudWatch Dashboard for cost monitoring
resource "aws_cloudwatch_dashboard" "cost_dashboard" {
  dashboard_name = "cost-dashboard"

  dashboard_body = jsonencode({
    widgets = [
      {
        type = "metric"
        properties = {
          metrics = [
            ["AWS/Billing", "EstimatedCharges", { stat = "Average" }]
          ]
          period = 86400
          stat   = "Average"
          region = var.aws_region
          title  = "Estimated Monthly Charges"
          yAxis = {
            left = {
              min = 0
            }
          }
        }
      }
    ]
  })
}

# Data for current account
data "aws_caller_identity" "current" {}

# Tag compliance and cost allocation
resource "aws_ec2_instance" "tagged_instance" {
  ami           = "ami-0c55b159cbfafe1f0"
  instance_type = "t3.small"

  tags = {
    Name              = "cost-tracked-instance"
    CostCenter        = "engineering"
    Environment       = "production"
    Project           = "web-app"
    ManagedBy         = "terraform"
    ChargebackEmail   = "ops@example.com"
  }
}
```

### 3. **Azure Cost Management**

```bash
# Install Azure CLI
curl -sL https://aka.ms/InstallAzureCLIDeb | bash

# Get cost analysis
az costmanagement query \
  --timeframe MonthToDate \
  --type Usage \
  --dataset aggregation='{"totalCost":{"name":"PreTaxCost","function":"Sum"}}' \
  --dataset grouping='[{"type":"Dimension","name":"ResourceType"}]'

# Create budget alert
az consumption budget create \
  --name MyBudget \
  --category Cost \
  --amount 5000 \
  --time-grain Monthly \
  --start-date 2024-01-01 \
  --notifications-enabled True

# List recommendations
az advisor recommendation list \
  --category Cost

# Export cost data
az costmanagement export create \
  --name MonthlyExport \
  --dataset aggregation='{"totalCost":{"name":"PreTaxCost","function":"Sum"}}' \
  --timeframe TheLastMonth \
  --schedule-status Active

# Get VM sizing recommendations
az advisor recommendation list \
  --category Performance \
  --query "[?properties.category=='Compute']"
```

### 4. **GCP Cost Optimization**

```bash
# Get billing data
gcloud billing accounts list

# Create budget
gcloud billing budgets create \
  --billing-account=BILLING_ACCOUNT_ID \
  --display-name="Monthly Budget" \
  --budget-amount=10000 \
  --threshold-rule=percent=50 \
  --threshold-rule=percent=90 \
  --threshold-rule=percent=100

# List cost recommendations
gcloud compute instances list \
  --format='table(name,machineType.machine_type(),CPUS:format="@(scheduling.nodeAffinities[0].nodeAffinities[0].key): \
  (@(scheduling.nodeAffinities[0].nodeAffinities[0].values[0]))")'

# Enable committed use discounts
gcloud compute commitments create my-commitment \
  --plan=one-year \
  --resources=RESOURCE_TYPE=INSTANCES,RESOURCE_SPEC=MACHINE_TYPE=n1-standard-4,COUNT=10 \
  --region=us-central1

# Get storage cost estimate
gsutil du -s gs://my-bucket
```

### 5. **Cost Monitoring Dashboard**

```python
# Python cost monitoring tool
import boto3
from datetime import datetime, timedelta
from typing import Dict, List
import json

class CloudCostMonitor:
    def __init__(self):
        self.ce_client = boto3.client('ce')
        self.ec2_client = boto3.client('ec2')
        self.rds_client = boto3.client('rds')

    def get_monthly_costs_by_service(self, months=3) -> Dict:
        """Get monthly costs breakdown by service"""
        end_date = datetime.now().date()
        start_date = end_date - timedelta(days=30*months)

        response = self.ce_client.get_cost_and_usage(
            TimePeriod={
                'Start': start_date.isoformat(),
                'End': end_date.isoformat()
            },
            Granularity='MONTHLY',
            Metrics=['UnblendedCost'],
            GroupBy=[
                {'Type': 'DIMENSION', 'Key': 'SERVICE'}
            ]
        )

        costs = {}
        for result in response['ResultsByTime']:
            for group in result['Groups']:
                service = group['Keys'][0]
                cost = float(group['Metrics']['UnblendedCost']['Amount'])

                if service not in costs:
                    costs[service] = []
                costs[service].append({
                    'date': result['TimePeriod']['Start'],
                    'cost': cost
                })

        return costs

    def identify_savings_opportunities(self) -> Dict:
        """Identify resources that can be optimized"""
        opportunities = {
            'unattached_ebs_volumes': [],
            'unassociated_eips': [],
            'underutilized_instances': [],
            'unattached_network_interfaces': []
        }

        # Check EBS volumes
        volumes_response = self.ec2_client.describe_volumes(
            Filters=[{'Name': 'status', 'Values': ['available']}]
        )

        for volume in volumes_response['Volumes']:
            opportunities['unattached_ebs_volumes'].append({
                'volume_id': volume['VolumeId'],
                'size_gb': volume['Size'],
                'estimated_monthly_cost': volume['Size'] * 0.10
            })

        # Check Elastic IPs
        addresses_response = self.ec2_client.describe_addresses()

        for address in addresses_response['Addresses']:
            if 'AssociationId' not in address:
                opportunities['unassociated_eips'].append({
                    'public_ip': address['PublicIp'],
                    'estimated_monthly_cost': 3.60
                })

        # Check underutilized instances
        instances_response = self.ec2_client.describe_instances()

        for reservation in instances_response['Reservations']:
            for instance in reservation['Instances']:
                opportunities['underutilized_instances'].append({
                    'instance_id': instance['InstanceId'],
                    'instance_type': instance['InstanceType'],
                    'state': instance['State']['Name'],
                    'recommendation': 'Consider downsizing or terminating'
                })

        return opportunities

    def calculate_potential_savings(self, opportunities: Dict) -> Dict:
        """Calculate potential monthly savings"""
        savings = {
            'ebs_volumes': sum(op['estimated_monthly_cost'] for op in opportunities['unattached_ebs_volumes']),
            'eips': sum(op['estimated_monthly_cost'] for op in opportunities['unassociated_eips']),
            'total_monthly': 0
        }

        savings['total_monthly'] = savings['ebs_volumes'] + savings['eips']
        savings['total_annual'] = savings['total_monthly'] * 12

        return savings

    def generate_cost_report(self) -> str:
        """Generate comprehensive cost report"""
        costs_by_service = self.get_monthly_costs_by_service()
        opportunities = self.identify_savings_opportunities()
        savings = self.calculate_potential_savings(opportunities)

        report = f"""
        ===== CLOUD COST REPORT =====
        Generated: {datetime.now().isoformat()}

        CURRENT COSTS BY SERVICE:
        """

        for service, costs in costs_by_service.items():
            total = sum(c['cost'] for c in costs)
            report += f"\n{service}: ${total:.2f}"

        report += f"""

        SAVINGS OPPORTUNITIES:
        - Unattached EBS Volumes: ${savings['ebs_volumes']:.2f}/month
        - Unassociated EIPs: ${savings['eips']:.2f}/month

        POTENTIAL MONTHLY SAVINGS: ${savings['total_monthly']:.2f}
        POTENTIAL ANNUAL SAVINGS: ${savings['total_annual']:.2f}
        """

        return report

# Usage
monitor = CloudCostMonitor()
print(monitor.generate_cost_report())
```

## Best Practices

### ✅ DO
- Use Reserved Instances for stable workloads
- Implement Savings Plans for flexibility
- Right-size instances based on metrics
- Use Spot Instances for fault-tolerant workloads
- Delete unused resources regularly
- Enable detailed billing and cost allocation
- Monitor costs with CloudWatch/Cost Explorer
- Set budget alerts
- Review monthly cost reports

### ❌ DON'T
- Leave unused resources running
- Ignore cost optimization recommendations
- Use on-demand for predictable workloads
- Skip tagging resources
- Ignore data transfer costs
- Forget about storage lifecycle policies

## Cost Optimization Strategies

- Reserved Instances: 20-40% savings
- Spot Instances: 70-90% savings
- Savings Plans: 15-30% savings
- Storage optimization: 30-50% savings
- Data transfer optimization: 10-20% savings

## Resources

- [AWS Cost Management](https://aws.amazon.com/aws-cost-management/)
- [Azure Cost Management](https://docs.microsoft.com/en-us/azure/cost-management-billing/)
- [GCP Cost Optimization](https://cloud.google.com/cost-optimization)
- [FinOps Foundation](https://www.finops.org/)