I started working on this task a week ago. The setup is based on the write-up in https://docs.gitlab.com/runner/configuration/autoscale.html
Almost there …
What I previously had set up was a few persistent EC2 Spot requests for largish machines at half (more) the price. However, I felt (and heard complaints) that those were not enough at rush hour (when everyone and their cat wanted to push their code changes and test them). At the same time, even with Spot instances we had to pay for mostly unused servers for more than half the time. That felt like a huge waste for me and I know many feel the same.
Problems
* Get the right size instances for the right price
* Setup inside a VPC
* Use S3 for caching
* Use docker registry proxy (or setup the gitlab docker registry) for custom docker images used in CI jobs