From Opsworks to Kubernetes to Fargate

This short writeup is all about how I used different solutions to manage my infrastructure where I host my webapplications. Since I manage quite some webapplications since I started my bussiness the whole Ops side of it get's more complicated.

Managing it should not become too much of a burden. At the same time I'd like to keep as much of control as possible being able to troubleshoot where possible and keeping standards on a high level towards my clients. Next to that, I'd like to focus on creating and maintaining webapplications more than managing infrastructure. The thing is, I'm too much of a geek which makes the whole thing even more complicated. Meaning that I sometimes struggle balancing between what I need to know and figure out and what I'd like to know. You simply just can't have it all.

Having said that. I started out years ago, on AWS, managing bare EC2 instances to host webapplications. Soon after that I discovered AWS Opsworks, which I even use to date in some cases. It serves me well, but maintaining multiple applications on the same host just isn't always that good in terms of seperation of concerns. It's quite cost effective, but at exactly what cost. Not to mention security.

For Opsworks you have to write your Chef recipes, which need to be written in Ruby. Of course not a big issue, but since I'm primarily a .JS guy, it's an extra thing to keep tabs on. Let alone the Chef part you need to debug if things go sideways.

Now, years later I finally dived into the Dockersphere and felt that Opsworks didn't appeal to me in combination with Docker. To make a long story short, I dived into Kubernetes as well. I learned a *lot* by figuring out Kubernetes as even a Docker newbie. Moved over quite some applications to Google Cloud Engine and even sooner than I knew, I had three clusters running.

I like to bring stuff into practice. Since you learn fast but also the hard way. In my experience of several months running those three clusters I can tell you it was more work managing those clusters than those Opsworks stacks. Let alone the costs running those clusters. Quite more expensive than hosting multiple applications on a EC2 instance managed by Opsworks. But going to Opsworks with all my Dockerized applications wasn't an option for me.

Running Kubernetes is really a job on it's self. So one can call him-/herself a full stack developer, but seriously managing such a beast just takes care and time. It's such a big topic in my view, it's not something you also properly manage on the side. Something I just haven't got the time for while doing all the stuff I'm doing day to day. (Mostly coding that is.)

So to me, jumping on the Docker bandwagon was a good choice, kubernetes, not quite. Yes I learned a lot, but it has taken a toll (stress/mentally/costs) in my already too busy schedule.

Of course I keep tabs on AWS news and discovered that AWS Fargate got cheaper to run early 2019 and decided to try out running my applications using Fargate. Now, some weeks passed I can say it feels like a very nice step in between Opsworks and Kubernetes to me and my usecase. But of course I'm not yet convinced untill I'm several months further.