python (11)


Get daily cost alerts on AWS

So I wanted to have a better alarm system for when AWS hits us with unexpected costs. It’s better to know there’s something wrong rather quickly and not suffer hundreds of dollars costs for something you don’t really need or want.

The AWS provided alarm checks for hikes on a monthly basis. Here’s the doc they published. So that’s an alarm that sounds when your estimated bill is going to be higher than the budgeted amount, or what you had in mind in the first place. Not very useful honestly in our case. It will just be too late.

The only alternative I found was creating a daily check, that will compare yesterday’s costs against a max_amount set by default. Let’s say you want to have your daily bill not exceed 5$US.

For ease of use and maintainability, I’m using a lambda function triggered by a cron (EventBridge rule) for the daily checks. And I’m sending the Alarm using an SNS topic, this way I can subscribe to it by email, or send it to our Mattermost channel, etc.

Here’s the code:

import os
import json
import boto3
from datetime import datetime, timedelta


def lambda_handler(event, context):
    yesterday   = datetime.strftime(datetime.now() - timedelta(1), '%Y-%m-%d')
    twodaysago  = datetime.strftime(datetime.now() - timedelta(2), '%Y-%m-%d')
    cost_metric = os.environ.get('cost_metric')
    max_amount  = os.environ.get('max_amount')
    
    ce  = boto3.client('ce')
    sns = boto3.client('sns')
    
    result = ce.get_cost_and_usage(
        TimePeriod={'Start': twodaysago, 'End': yesterday}, 
        Granularity='DAILY', 
        Metrics=[cost_metric]
    )
    total_amount = result['ResultsByTime'][0].get('Total').get(cost_metric).get('Amount')
    
    if total_amount > max_amount:
        sns_topic = sns.create_topic(Name='BillingAlert')
        sns.publish(
            TopicArn=sns_topic.get('TopicArn'),   
            Message='Total cost "{} USD" exceeded max_amount rate: {}'.format(total_amount, max)   
        )
        
    return {
        'statusCode': 200,
        'body': json.dumps('cost check: {}'.format(total_amount))
    }

Note that you will need to add a couple of environment variables to Lambda: cost_metric and max_amount
And the following permissions to the role used by the lambda function: ce:GetCostAndUsage, sns:Publish and sns:CreateTopic

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "VisualEditor0",
            "Effect": "Allow",
            "Action": "ce:GetCostAndUsage",
            "Resource": "*"
        },
        {
            "Sid": "VisualEditor1",
            "Effect": "Allow",
            "Action": [
                "logs:CreateLogStream",
                "sns:Publish",
                "sns:CreateTopic",
                "logs:PutLogEvents"
            ],
            "Resource": [
                "arn:aws:logs:us-east-1:*:log-group:/aws/lambda/checkDailyCosts:*",
                "arn:aws:sns:*:*:*"
            ]
        },
        {
            "Sid": "VisualEditor2",
            "Effect": "Allow",
            "Action": "logs:CreateLogGroup",
            "Resource": "arn:aws:logs:us-east-1:*:*"
        }
    ]
}

After that’s setup, go to your SNS topic (created by Lambda if it doesn’t exist) and subscribe to it. There you go, daily checks and an alarm if the bill is higher than expected.

Similar Posts:




Update PIP on Ubuntu

From SO answer

Your pip may be outdated. Even in Ubuntu 14.04 LTS, the pip version it installed using apt-get install python-pip was 1.5.4. Try updating pip manually, and possibly the new packages again as well.

pip --version # 1.5.4
curl -O https://bootstrap.pypa.io/get-pip.py
sudo python get-pip.py
hash -r # reset bash cache
pip --version # 6.0.8

The hash -r line is essential!

reference: https://pip.pypa.io/en/latest/installing.html

Similar Posts:




Now Reading Test Driven Development with Python…

Now Reading “Test-Driven Development with Python” http://chimera.labs.oreilly.com/books/1234000000754

Similar Posts:




RimuDNS python module

I wrote previously about building dns zone files mostly by querying DNS and saving the results to a file.

The method is not perfect, and it will not be possible to get all the records. In fact, you could get all the records incorporated in the result file if you already have a list and feed it to your code.

I wrote a python module to manage DNS records using the RimuHosting DNS API. The code is hosted at GitHub.

I also added a guessing feature.

Similar Posts:




Django-TimelineJS Pluggable App

I like VeriteCo’s TimelineJS. It’s elegant, easy to use and very useful. I don’t like editing Google a google spreadsheet every time I want to update the timeline although it might be easier for some people. So, I wrote a small pluggable app for django to embed a timeline in django templates.

The code is loosely based on the code for the WordPress plugin. And I’ll try to copy over more code and fixes from that soon.

For now, it’s a working app with an admin backend to enter the timeline information and events, and to edit the options. It’s also a custom template tag that you can use to embed the timeline in your own templates.

Should be easy enough to use it.

pip install django-timelinejs

then include it in your INSTALLED_APPS and ./manage.py syncdb

Drop me a note if you find a bug or want some help. You can use the issues in GitHub for that.

Similar Posts:




Building DNS Zone Files

The only way to get a full listing of a DNS zone is to use AXFR to request a zone transfer. But that’s usually locked and not available. So building the zone file for a domain is basically guesswork or manual labour at best.

For some background information check out these questions on SO
Here’s a script I wrote to try and automate that as much as possible. Still in beta. Suggestions welcome.

Note: I use DNSPython

Next on my list is using the RimuHosting/Zonomi API to create/import the DNS zone.

Similar Posts:




Testing SMTP Connection

Here’s a quick script to test SMTP connections

Similar Posts:




NatGeo Wallpaper Downloader & Desktop Changer for Gnome Shell

I saw a couple of National Geographic wallpapers and those are pretty nice. I did not find an API on their site to download those on a regular basis, but it seemd pretty simple to just download them.

I just uploaded my code as a gist per https://gist.github.com/3002699 (the gist code should be more up to date than the following code)
And I added the script in my crontab:

@daily	/home/abdallah/Code/scripts/natgeowallpaper.py

Let me know if you have any notes/comments/changes
Here’s the code

Similar Posts:




Django uWSGI and Nginx on Ubuntu 11.10

I’m working my way through installing my simple django app on a new Ubuntu machine using the components mentioned above.

So far, here’s what I’ve done:

  • apt-get install nginx-full
  • apt-get install uwsgi uwsgi-plugin-python
  • installed django from svn described in the docs

The nginx configuration can be found in /etc/nginx/sites-available/default. Of course, it’s best to make a copy of that file and edit it for the site/app I’m working on. Here’s what I put in it:

server {
    listen   80;
    server_name sss.grat.in;
    access_log /var/log/nginx/sss.grat.in-access.log;
    location /media {
        alias /home/sss/app/media/;
    }
    location /static {
        alias /home/sss/app/static/;
    }
    location / {
        uwsgi_pass unix:///var/run/sss.grat.in.sock;
        include uwsgi_params;
    }
}

Now for the uWSGI configuration file. I created the file /etc/uwsgi/apps-available/sss.ini and added the following lines:

[uwsgi]
chdir = /home/sss/app
pythonpath = /home/sss
env = DJANGO_SETTINGS_MODULE=app.settings
# load django
module = django.core.handlers.wsgi:WSGIHandler()

it’s really just that. What took me so long was to figure out that in Debian/Ubuntu you have to install uwsgi-plugin-python, otherwise you’ll keep on getting weird error messages. Also, the default ini file has most of the settings (in /etc/default/uwsgi) so you should either read or edit that to make sure you’re getting all the right values.

Similar Posts:




Site ScreenShot Part 2 – Planning

I was having a little conversation with my friend Ghassan from LebanonDesign and he mentioned his company was actually using something similar to SiteScreenShot to generate thumbnails for their different websites. So I thought I could scale my code to do the same. Of course, plenty of changes should be considered:

  • Snapshot generation currently takes 3-5 seconds to complete. So I need to do 2 things:
    • Tell the user that the app is actually working on generating the snapshot
    • Provide a way to retrieve the pic once the snapshot is generated
    • Make things go faster if possible
  • Getting more users means more load on the server, what should we use?
    • Replace Apache with Nginx for faster static file delivery
    • Use a queue manager like Celery/RabbitQM to distribute the load

I’m currently testing some code with django-celery and I’ll try to post more information here (so I can remember what I’m doing)

Also on my list is making the site accessible via a simple API.

I also need to research a better way to take the screenshots on a headless server.

[adsenseyu3]

Similar Posts: