If you’re just coming back to work today to find your data filters in Terraform (or Pulumi, or whatever) are broken while trying to run a deployment, check this out:
https://aws.amazon.com/blogs/aws/update-on-amazon-linux-ami-end-of-life
Amazon has pulled all of the old Amazon Linux AMIs that might respond to your filter. You need to check and make sure your filter is updated. This issue is a little confusing because you can still launch instances from the public SSM parameter, but the API and console won’t let you see it.
Pro Tip: Don’t use iCloud’s “Hide My Email” feature to create email addresses to use with Amazon accounts. They trip a security rule and AWS is straight up impossible to work with on resolving the situation. We’ve reached the point where it’s so hard to prove your identity and actual use of a product that it’s just not even worth the time.
Probably the toughest and most mind-boggling thing about any cloud profession is that you could easily spend your entire career working and learning about one of the cloud providers, such as Amazon or Azure.
But the reality, in practice, is that CIO’s want you to know them all. Why? They just don’t feel comfortable with “all the eggs in one basket.” But they’re more than happy to pay one person a fraction of their worth to have to know how to do them all.
That part kind of sucks.
I really love Terraform. It’s been driving my career advancements for the past 2-3 years. I’ve really enjoyed working with it. As such, I do try to create useful Terraform modules to give back to the community. Most of the module work I have done has been for my employer, but sometimes I make code that is more for myself. This is one such module.
Last night, we solved a long-standing bug in one of our Terraform modules. It’s been haunting us for a long time - damn near over a year. We managed to produce this bug in a CloudFormation stack as well, but we just couldn’t figure out where it was coming from.
For reference, we were trying to stand up an Elastic Container Service cluster with containers using dynamic port routing. When you do this, you build it with an Application Load Balancer (ALB) plus a target group. The way things are supposed to work is that when a container is spun up on the cluster, it chooses an ephemeral port and that port is registered to the target group’s health checks.
That was essentially working, but something was adding an additional and erroneous health check to the exposed port (https/443) which would cause the auto-scaling group to think things were amiss… and continously terminate/rebuild instances. Not a fun situation. Our workaround was to manually remove the health check. But each time the ASG terminated and added an instance, the bad health check would come back. We finally figured this mess out.
Day 5 of re:Invent 2019 came and my body had already went. My attention span had expired. My energy had expired. Even though I slept a good 7 hours, I still had to take on a nap in the afternoon.
I only attended one session on the last day before the whole show shut down. But the last session didn’t really garner any interesting notes. Sorry.
Read on for my list of sessions that I want to explore on YouTube as additional learning. I’ll provide the session codes only. It’ll be up to use to find them on the AWS YouTube channel.
By day 4, my attention span and body were starting to give out. I really needed to hang in there, though, because this was the night of re:Play, the big party. I always enjoy going to this party to see Jen Lasher play. She is super energetic, talented with mixing beats, and really fun to watch. Her enthusiasm is infectious.
She did not disappoint at all.
I knocked out one session of notes - the other sessions I attended did not garner anything interesting to write down. Read on for the notes on architecting serverless apps at scale.
Day 4 of re:Invent 2019 and my suspicions were correct: This show was being run much better than before. The busses were punctual. Hell, they actually had too many busses. There was no waiting. The conference staff was jubilant. The mood wasn’t stressed at all. Everyone was having a great time.
I only took notes for one session on this day. It was a session about serverless networking, which focused primarily on what happens when you run Lambda “inside” your VPC. Spoiler alert: it’s not running inside your VPC at all. This actually solved a mystery we were having and it was worth the visit for that knowledge alone.
On the evening of this day, I had the honor of attending the Aerosmith concert at the Park MGM. It was wonderfully awesome. Steven Tyler is 71 years old and still going strong like he’s 20. I hope I’m still ticking like that at 71.
Read on for the session notes.
On day 2 of Re:Invent 2019, it was pretty clear that Amazon Web Services had done a lot of work to scale this conference out. Last year, it was 55,000 attendees and the logistics were terrible. The app crashed often and it was damn near impossible to reserve a seat for any sessions. Getting up and down the strip was a nightmare. AWS had promised a shuttle system, but the shuttles were not well thought out and it actually made it even more difficult to get from venue to venue. They didn’t have enough busses. They didn’t have enough personnel. They didn’t have enough of anything. Ironically, it was like they couldn’t scale out the conference like they can scale their compute services.
There were definitely more people this year. How much more? At least 65,000. But later I learned that the final count was somewhere closer to 80,000… and they were handling them all very well. I was happy to see this because I had pretty much sworn that I wouldn’t be attending anymore re:Invent conferences because it was such an awful experience. This conference was already turning into a complete 180.
I skipped Andy Jassey’s keynote. Andy is a competent guy and runs his business very well. He is not, however, a very good presenter. His keynotes run 2-2.5 hours and they’re full of marketing and momentum-bursting interruptions with musical acts. It’s like he’s trying to be Apple, but not sure how to do it. He needs to get some better coaching. I watched his keynote from the comfort of the certification lounge the last few years. This year, I decided to just skip the keynote altogether and read the summaries on the net later. They had already announced so many new services and enhancements that I had no idea what they might want to introduce in the actual keynote. The keynotes were reserved seating only and I didn’t realize that until it was much too late.
I spent the day chasing sessions. It was here that my strategy formulated for this re:Invent and the years to come: favor chalk talks, builders sessions and workshops over sessions. Sessions are recorded for YouTube. The others are not. They’re also highly interactive and much more involved. That’s just my opinion anyway.
Read on for session notes.
I’m at Amazon Web Services’ Re:Invent 2019. I’ve gone to every re:invent except for one. It’s always an interesting mixed bag of experiences, but I can say for sure that I think they’ve just about got this thing down. This conference is run much better than it was last year. There’s enough session repeats, enough venues, enough shuttles, and just generally enough of everything to keep you moving. There’s almost too much to do on the social side of things. There’s no way you can hit every social event. It’s even hard to find them because there’s so many. (Seems strange, but true).
Over the next few days, I intend to keep a log of some of the more important notes I took during sessions and other things. There’s really too much data at this point. AWS is moving too fast and you can definitely make a lifetime career out of specializing in AWS.
Let’s start with my notes from day one.
