When Jeremy met AWS… a true Seattle love story
A few years ago, I started working with AWS and it was love at first byte (see what I did there?). I worked for NetApp at the time and had just moved from the NetApp Workflow Automation (WFA) Solutions Architecture team to the Cloud Solutions Group. It was an exciting change in pace for me. I had spent the previous year working almost exclusively integrating what is now called the VMware vRealize Automation suite. During this period of my career, I was heavily invested in NetApp’s ability to deliver automation for FlexPod and VMware private cloud. When the opportunity came to transition, I was somewhat hesitant. Wow that changed quickly. I was in nirvana. Everything was an API and everything could be automated. I started working on a project to create a storage management platform for NetApp Private Storage (NPS), think co-located NetApp array near the AWS datacenter, and what would soon be coming as ONTAP Cloud. In one month, we had actually built a pretty awesome prototype though lacking some great graphical appeal. In mere minutes, we could deploy a secure, multi-tenant lab environment to any AWS account and connect the NetApp storage.
Those APIs were just amazing and I had become hooked. I dove very deep into how things like Elastic Beanstalk, Cloud Formation templates, and integration into EC2 and Direct Connect. In less than six months, I had rebuilt our entire prototype from scratch and leveraged extensively the AWS features. We deployed our entire portal using Beanstalk and backend with NetApp NFS. One thing that I learned quickly was that I had to reconsider everything that I would normally do in VMware. It was both confusing and exhilarating. I quickly decided that we would actually get greater flexibility by simply using NetApp NFS volumes mounted to the beanstalk systems. This meant zero warming of the data and our systems became 100% ephemeral. As long as my data was secure, who cares about the servers. This started me down an interesting thought process and was long before the concept of the NetApp Data Fabric became a thing. We knew how to redeploy our entire environment anywhere in mere minutes.
Two months after my complete re-write of the prototype, we had our first challenge. For the past four months, we had run the prototype and later v2.0 in a single site servicing over 300 demo labs. It was just so easy to manage and just about 100% hands off for me at this point. My management decided that we should roll this out to five different AWS regions to distribute the load and open this lab up to more customers. Obviously, if they were going to invest then I needed to push these new environments out as fast as I could. Our first new region was Tokyo and I was very excited. It was my chance to see just how easy it would be to automate NetApp NPS and AWS in a new region.
Four hours. Let that sync in for a moment. Once the field team had finished cabling everything and brought the network circuits online, the Tokyo region into our Data Fabric in four hours. I was paranoid and probably rightly so. How many times have you seen a full hybrid datacenter brought online and serving data in just four hours? It was there and we were provisioning. I pulled out some of my test AWS accounts to verify but everything just worked. The new Tokyo beanstalk was online in minutes. The automation servers were back up and serving in Tokyo. The main portal offered the option to deploy in N. Virginia and Tokyo. This was a fluke right? Nope. We did this four other times and in almost no time at all we had a web of data centers up and communicating. VPN connections between the sites would allow NetApp SnapMirror to replicate the data and because we were using NPS and Direct Connect it all just worked.
It was in that moment that I really felt the love that I still hold for AWS and their APIs. You know, I think that I should start a new series or some articles on some of my experiences with AWS and ONTAP Cloud. I actually did some really cool integration in the past. Heck, I still use it as I develop NetApp automation for clients. Until next time