After setting up a bastion and getting GitLab runner to autoscale on EC2 spot instances, I noticed that some instances are being started but left un-tagged and probably unused. Those seem to slip through the cracks somehow .. I’m still investigating why that’s happening.
Meanwhile, to avoid paying for those instances, I set up a cron to check for non tagged instances on EC2 and terminate them.
Here’s my bash code using aws-cli
#!/bin/bash
INSTANCES=$(aws ec2 describe-instances \
--filters "Name=key-name,Values=runner-*" \
"Name=instance-state-name,Values=running" \
--query 'Reservations[].Instances[?!not_null(Tags[?Key == `Name`].Value)] | [].[InstanceId]' \
--output text)
if [ -n "$INSTANCES" ]; then
aws ec2 terminate-instances --instance-ids $INSTANCES
fi
This looks for running instances, with the Key Name runner-* and where the tag Name is not not_null (so null!)
It’s working so far. Will keep on looking for a more permanent solution.