Auto Scale EC2 GPU Spot Instances When Uploading to S3 Bucket

from blog matt.sh, | ↗ original
Problem description: we want to upload unlimited amounts of content to an S3 bucket, then we want our GPU image processing code to consume those uploads on-demand. We don’t want any idle servers, so instances should be provisioned when files are uploaded to the S3 bucket, live until all the new files are processed, then terminate themselves so we have no long-lived idle servers. AWS can do the above using completely built-in tools and triggers and monitors, so let’s configure...