The Lost Notebook

lambda (3)


Get daily cost alerts on AWS

So I wanted to have a better alarm system for when AWS hits us with unexpected costs. It’s better to know there’s something wrong rather quickly and not suffer hundreds of dollars costs for something you don’t really need or want.

The AWS provided alarm checks for hikes on a monthly basis. Here’s the doc they published. So that’s an alarm that sounds when your estimated bill is going to be higher than the budgeted amount, or what you had in mind in the first place. Not very useful honestly in our case. It will just be too late.

The only alternative I found was creating a daily check, that will compare yesterday’s costs against a max_amount set by default. Let’s say you want to have your daily bill not exceed 5$US.

For ease of use and maintainability, I’m using a lambda function triggered by a cron (EventBridge rule) for the daily checks. And I’m sending the Alarm using an SNS topic, this way I can subscribe to it by email, or send it to our Mattermost channel, etc.

Here’s the code: 

import os
import json
import boto3
from datetime import datetime, timedelta


def lambda_handler(event, context):
    yesterday   = datetime.strftime(datetime.now() - timedelta(1), '%Y-%m-%d')
    twodaysago  = datetime.strftime(datetime.now() - timedelta(2), '%Y-%m-%d')
    cost_metric = os.environ.get('cost_metric')
    max_amount  = os.environ.get('max_amount')
    
    ce  = boto3.client('ce')
    sns = boto3.client('sns')
    
    result = ce.get_cost_and_usage(
        TimePeriod={'Start': twodaysago, 'End': yesterday}, 
        Granularity='DAILY', 
        Metrics=[cost_metric]
    )
    total_amount = result['ResultsByTime'][0].get('Total').get(cost_metric).get('Amount')
    
    if total_amount > max_amount:
        sns_topic = sns.create_topic(Name='BillingAlert')
        sns.publish(
            TopicArn=sns_topic.get('TopicArn'),   
            Message='Total cost "{} USD" exceeded max_amount rate: {}'.format(total_amount, max)   
        )
        
    return {
        'statusCode': 200,
        'body': json.dumps('cost check: {}'.format(total_amount))
    }

Note that you will need to add a couple of environment variables to Lambda: cost_metric and max_amount
And the following permissions to the role used by the lambda function: ce:GetCostAndUsage, sns:Publish and sns:CreateTopic

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "ce:GetCostAndUsage",
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogStream",
                "sns:Publish",
                "sns:CreateTopic",
                "logs:PutLogEvents"
            ],
            "Resource": [
                "arn:aws:logs:us-east-1:*:log-group:/aws/lambda/checkDailyCosts:*",
                "arn:aws:sns:*:*:*"
            ]
        },
        {
            "Sid": "VisualEditor2",
            "Effect": "Allow",
            "Action": "logs:CreateLogGroup",
            "Resource": "arn:aws:logs:us-east-1:*:*"
        }
    ]
}

After that’s setup, go to your SNS topic (created by Lambda if it doesn’t exist) and subscribe to it. There you go, daily checks and an alarm if the bill is higher than expected.

Similar Posts:



ChatOps with Mattermost and AWS Lambda

I’ve been working towards making things simpler when managing distributed resources at work. And since we spend most of our day in the chat room (was Slack, now Mattermost) I thought it’s best to get started with ChatOps

It’s just a fancy word for doing stuff right from the chat window. And there’s so much one can do, especially with simple Slash Commands.

Here’s a lambda function I setup yesterday for invalidating CloudFront distributions. 

from time import time
import boto3

import json
import os
import re


EXPECTED_TOKEN = os.environ['mmToken']
ALLOWED_USERS = re.split('[, ]', os.environ['allowedUsers'])
DISTRIBUTIONS = {
    'site-name': 'DISTRIBUTIONID',
    'another.site': 'DISTRIBUTIONID05'
}

def parse_command_text(command_text):
    pattern = r"({})\s+(.*)".format('|'.join(DISTRIBUTIONS.keys()))
    m = re.match(pattern, command_text)
    if m:
        return { 'site': m.group(1), 'path': path}
    else: 
        return False

def lambda_handler(event, context):
    # Parse the request
    try:
        request_data = event["queryStringParameters"]
    except:
        return {
            "statusCode": 400,
            "headers": {"Content-Type": "application/json"},
            "body": '{ "message": "Use GET for setting up mattermost slash command" }'
        }

    # Check the token matches.
    if request_data.get("token", "") != EXPECTED_TOKEN:
        print('Wrong Token!')
        return {
            "statusCode": 401,
            "headers": {"Content-Type": "application/json"},
            "body": '{ "message": "Mattermost token does not match" }'
        }
        
    # Check the user is allowed to run the command
    if request_data.get("user_name", "") not in ALLOWED_USERS:
        print('Wrong User! {} not in {}'.format(request_data['user_name'], ALLOWED_USERS))
        return {
            "statusCode": 401,
            "headers": {"Content-Type": "application/json"},
            "body": '{ "message": "User not allowed to perform action" }'
        }

    # parse the command
    command_text = request_data.get("text", "")
    if not command_text:
        print('Nothing to do, bailing out')
        return {
            "statusCode": 404,
            "headers": {"Content-Type": "application/json"},
            "body": '{ "message": "No command text sent" }'
        }
    parts = parse_command_text(command_text)
    if not parts: 
        print('Bad formatting - command: {}'.format(command_text))
        return {
            "statusCode": 402,
            "headers": {"Content-Type": "application/json"},
            "body": '{ "message": "Wrong pattern" }'
        }


    # Do the actual work
    cf_client = boto3.client('cloudfront')

    # Invalidate
    boto_response = cf_client.create_invalidation(
        DistributionId=DISTRIBUTIONS[parts['site']],
        InvalidationBatch={
            'Paths': {
                'Quantity': len(parts['path']),
                'Items': parts['path'] 
            },
            'CallerReference': str(time()).replace(".", "")
        }
    )['Invalidation']

    # Build the response message text.
    text = """##### Executing invalidation
| Key | Info |
| --- | ---- |
| Site | {} |
| Path | {} |
| ID | {} |
| Status | {} |""".format(
        parts['site'], 
        parts['path'],
        boto_response['Id'], 
        boto_response['Status']
    )

    # Build the response object.
    response = {
        "response_type": "in_channel",
        "text": text,
    }

    # Return the response as JSON
    return {
        "body": json.dumps(response),
        "headers": {"Content-Type": "application/json"},
        "statusCode": 200,
    }

Note that you need to hook that up with an API Gateway in AWS. Once that’s done, you will have a URL endpoint ready for deployment.

Next, I created the slash command in mattermost with the following:

slash command configuration

That’s pretty much it. Rinse and repeat for a different command, different usage.

On my list next is to have more interaction with the user in mattermost per https://docs.mattermost.com/developer/interactive-messages.html
Weekend Project, Yay!

Similar Posts:



AWS Lambda Function Code

Quick snippet to get the function code

wget -O function.zip $(aws lambda get-function --function-name MyFunctionName --query 'Code.Location' --output text)

And another to update lambda with the latest

cd package 
zip -r9 ../function.zip .
cd ..
zip -g function.zip function.py
aws lambda update-function-code --function-name MyFunctionName --zip-file fileb://function.zip

Similar Posts:




The Lost Notebook, 2023. Theme Material for Coders