Strategies To Deal With AWS Rate Limits When Using Spinnaker At Scale

The Armory team did a quick video to discuss strategies to deal with AWS rate limits when using Spinnaker at scale:

Here’s A Transcript:

Ben: Hey, guys. My name is Ben. This is Daniel, Isaac, and Andrew. We’re working on an enterprise Spinnaker version. And we want to talk a little bit today about a common problem that many AWS customers have when they install Spinnaker. So Andrew, do you want to talk about the problem?

Andrew: Yeah, I want to talk about it. So … enterprise-level AWS customer, a lot of the times what happens is you’ve had your … resources extended. So that means … instances will yield these sometimes 10-20X what a regular customer has. But often, when you get these resources extended, you don’t get your … API Linux … proportionally extended … still really … limited on how many … describe … API calls you can do or something like that. So we see this a lot with Spinnaker … this up … makes those API calls a lot. It tries to get … internal vision of what your cloud looks like. So … run into that quite a few times … talk about some of the solutions, I guess.

Isaac: Yeah. So we’ve been working on a solution recently really to be able to add … limiting as well as … interval limiting so that you can configure it to your needs. So obviously, larger organization will have this problem more than smaller ones. And we’ll have it at different rates. There are different scales. Some people have 100 … and some people have 1000 … So routing configuration and logic into the system, to be able to kind of tune it to your needs and specific to what you’re doing. So maybe you have a lot more … than you do instances. So you’ll be able to increase the rate limit for one and not the other. And tune it as AWS also increases your rate limits proportional to your scale. So as your rate limits that have been guaranteed by AWS change, you can change it inside of the configuration. What this will ultimately lead to is just a better experience with Spinnaker and your deployments. So the more up to date you are with the AWS API, the more real time your deployments look like, and you can see in real time that’s actually happening because what we see with Spinnaker a lot of the times is Spinnaker gives you a application view of your deployment versus AWS just gives you just machine’s. So you always really want to end up looking at your deployment through Spinnaker, not AWS, because it’s really hard to find instances and how these things all correlate together. Whereas the Spinnaker view of the world gives you the … view …

Ben: Do you guys have a sense for what types of companies might need this type of solution? So … on AWS, … scale and size of company would this really apply to?

Andrew: The problem will happen to any enterprise-level company … extended, resource limits extended. I guess it could also happen to smaller companies if you just want to refresh … So if you want, you can change the resolution of how often you’re … these APIs. But it’ll make Spinnaker not as accurate. It won’t have as good of a view of the API, or of the cloud, rather. So it could be anybody. But it’s definitely going to be any corporate or enterprise-level AWS …

Isaac: Maybe 100 …, anything over 1000 machines, probably even less … a few hundred machines … going to want this resolution change.

Ben: Awesome. Thanks for watching, guys. Thanks for the explanation.

DROdio: Ciao.