Skip to content

Fixing the RubyGems “Too Many Requests 429” error

lots of gemsA server on which I am working runs this command: /usr/bin/gem install --no-rdoc --no-ri aws-sdk to get the aws-sdk. I was seeing this error message:

Error: Execution of '/usr/bin/gem install --no-rdoc --no-ri aws-sdk' returned 1: ERROR:  While executing gem ... (Gem::RemoteFetcher::FetchError)
    bad response Too Many Requests 429 (https://api.rubygems.org/api/v1/dependencies?gems=aws-sdk-elasticloadbalancingv2)

Every time I ran it I’d see a different gem that triggered the 429 response. There wasn’t much out there when searching, other than a note that I should update to a new version of bundler (which I wasn’t using).

Finally, I figured out how to get past this. What I did was manually run /usr/bin/gem install -f --no-rdoc --no-ri aws-sdk multiple times, and each time the command would get a little further. Finally all the dependencies had been downloaded. Then I was able to run it without the -f switch after that.

AWS Devops Dynamic Languages | moore |

Obstacles to building high availability software systems

Open sign
Is your system available?

I saw a discussion on a slack about obstacles to high availability systems and wanted to record the edited version for posterity (mostly for future me, as I blog for myself). Note that in any mention of high availability systems would be remiss if I didn’t mention the Google SRE book, which is slow reading but free and full of great information.

First, what is high availability? I like this definition from Digital Ocean:

In computing, the term availability is used to describe the period of time when a service is available, as well as the time required by a system to respond to a request made by a user. High availability is a quality of a system or component that assures a high level of operational performance for a given period of time.

Design considerations of a system that will hinder high availability fall into two categories.

The first category is actions that you don’t take, but could take:

  • single points of failure: if you have a piece of your system which is unique and it fails (and everything fails, all the time), the entire system’s availability will be affected.
  • missing or incomplete automation: if you need human beings to resurrect failed parts of your system, it will meaningful amounts of time and will be error prone.
  • failing to build in elasticity and scalability of resources: when usage increases, new resources should be automatically brought online. Failure to do so will impact system performance and that could impact system availability
  • missing or incomplete system instrumentation: if you don’t monitor your system, you won’t be able to even know its availability (until you hear from your users).
  • application statefulness (on the compute nodes): this impacts your ability to use elastic resources and to grow parts of your system that are under load. (If you aren’t designing a greenfield system, this may be an externally imposed requirement due to existing software.)

The second is in actions you can’t take because of external requirements on the system:

  • data sovereignty: if you are legally limited to certain data centers, you have fewer options for your system, this can hinder building the system.
  • tenancy: if you need to have single tenancy for security or legal reasons, you may have fewer options for elastic solutions.
  • data models and authority requirements: poorly performing data models can impact performance. If your application requires certain operations must be from the source of record (permissions checks, for example) then a poorly performing source data model can impact performance which can impact availability.
  • latency: if you have a highly latency sensitive system, then you may need to trade availability for decreased latency. Since availability often means geographic dispersion (to avoid disasters impacting multiple pieces of a system), it impacts latency requirements.
  • cost: high availability systems, because they have no single points of failure, cost more.

Again, this was a discussion from a slack of AWS instructors, but the commentary is mine, as are any mistakes. Thanks to Chad, Richard, Jon, Ryan and everyone else!

AWS Books Cloud Computing Devops Tips | moore |

Hipster Hosting at BSW, Tomorrow Only

Lady with computer mouse
She doesn’t look like she needs hosting, does she?

I’m doing a short presentation with a few other people at Boulder Startup Week on hosting. Tomorrow, Thur, at 10am MT.

Would love to see you there. Feel free to heckle.

If you can’t make it, here is the salient point of my presentation: startups are hard, so you should host your code and infrastructure at the highest level of abstraction that you can, so that your developers can focus on delivering business value through new features rather than doing ops. In practice, prefer hosting options in this order:

  • serverless
  • platform specific hosting (wpengine, etc)
  • general purpose PAAS (heroku, elastic beanstalk)
  • cloud VMs
  • colo
  • server in the closet

Of course, all advice is context dependent; my advice is aimed at small startups and the more flexibility your developers need around aspects of technology the lower on the list you’ll have to go.

Anyway, looking forward to a good discussion.

Presentations Startup Tips | moore |

Trust the compiler

I loved this post not because I love reading assembler but because it just illustrates so perfectly how often, when writing software, we can easily stand on the shoulders of giants.

My point is not that we should take what we’ve learnt from the LLVM-generated code and write a new version of our hand-rolled assembly. The point is that optimising compilers are really good. There are very smart people working on them and computers are really good at this kind of optimisation problem (in the mathematic sense) in a way that humans find quite difficult. It’s the job of language designers to give us the tools we need to inform the optimiser as best we can as to what our true intent is, and larger integer sizes are another step towards that.

Programming | moore | | Leave a comment

Boulder Startup Week Begineth!

Thumb upBoulder Startup Week is this week. If you haven’t been, it’s a great opportunity for a number of reasons. You can get a flavor of the Boulder tech community (though it’s worth remembering that there are numerous firms that don’t play in the startup world that are in Boulder). You can learn a lot about startups from folks who are actively building one, or have built one in the past. You can learn about new technologies and trends that are up and coming, including data science, blockchain and cannibis. And you can meet a lot of great folks.

I’m a bit burned out on startups at the moment, but am still planning to attend a few sessions, mostly on the development track. I’m especially excited for the Boulder Ruby Meetup on Wednesday, where experts will speak about interviewing. I’m also speaking at a session on hosting.

My tips for Boulder Startup Week:

  • go to at least one session in a different area of focus than you normally would.
  • arrive 5-10 minutes early and plan to stay 5-10 minutes after. Use this time to chat with folks. (This is hard for me, but I’ve found that having a canned opening line like “what interesting talks have you seen” or “is this your first time at BSW” is a good way to break the ice.)
  • the above tip will prevent you from trying to attend too many sessions back to back to back. This is a Good Thing(tm).
  • bring business cards, or prepare to exchange emails.
  • thank a volunteer and/or sponsor when you see them. There’s a tremendous amount of effort that goes into this week.
  • be prepared to help someone you meet out, with an intro, feedback on an idea, or even just an interesting article.
  • if a session is full, I’d get on the waitlist and then I’d show up anyway. Because every session is free, I’ve found that oftentimes folks are … over committed and there’s often space for other people.

If your interest has been piqued, please check out the schedule. Hope to see you out there.

Presentations Startup | moore | | Leave a comment

Throwback Thursday: What is the difference between a programmer and a developer?

BicycleOne of the nice things about blogging for so long is that you get to see the wheat separated from the chaff. If I’m still thinking the same thoughts five years or a decade later, that means that I’ve stumbled on one of my truths. (That or I’m just echoing my thoughts because I haven’t learned anything new. I believe it is the former, especially when it comes to development, something I spend a lot of time thinking about.)

Here’s my 2012 post about the difference between a programmer and a developer. A programmer can be a great coder, but stops when the code ends. A developer uses code to build a business solution. (Pssst, a programmer is much more likely to be replaced by an offshore “resource”.)

One of my favorite analogies for developers is the bike messenger. She may not be as fast as a car or as flexible as a pedestrian, but she can weave back and forth between the street and the sidewalk in a way that neither of those two other modes of transportation can. And in the end she’s all about the destination, just like great developers are.

Consulting Technology | moore | | Leave a comment

What do I do when someone’s doing something wrong?

Driving in the rain
Uh, I think you’re in a little deep

Somewhere, sometime, someone will do something wrong. It happens more often than you think. (That person might even be me.)

When they do, I have an innate desire to correct them, to show them the benefit of my mistakes and to enlighten them. This is, in itself, wrong.

What should I do instead? A couple of things.

  1. Make sure I really understand the issue they are facing. Sometimes I can gain an appreciation for their choice once I have more information about the situation, and sometimes my questioning can cause them to make different decisions or to revise their thinking.
  2. Think about whether I have all the context. (This one’s easy; I don’t.) So remember to consider that from their perspective choices make more sense than from mine.
  3. I consider the ramifications of a poor decision. It’s better to let someone choose an obviously wrong course when the stakes are low. We’ve all made mistakes (I deleted an entire table from a production database!) and that’s the best way to learn. However, if the stakes are high, that means a discussion is more warrented.
  4. Contemplate my role. Is this a peer? An old friend? An ex-colleague? A mentor? A mentee? Someone I’ve just met? Each of these types of people have different tolerance for my opinion.

In short, rather than just explaining to someone why they are wrong, I need to stop and think about the entire context of the situation, including what I’ve missed, what the consequences of the action are, and my role.

Lifehack | moore | | Leave a comment

MS Edge

Spelling on chalkboardI use MS Edge a lot (yes, I’m a windows user. Windows + Vagrant means I get to use all the user and developer software I want).

One thing that is an understated benefit of Edge is the autocorrect. Every text box automatically has spelling correction. That’s so nice for tools like slack and webmail where you are writing all day every day. It really means that just like you don’t have to know how to get someplace (because Google Maps) now you don’t even have to look up how to spell a word. Looks like you can turn if off if you want, but its deeply embedded in the OS.

 

Useful Tools | moore | | Leave a comment

Imposter syndrome

This article resonated with me. I became familiar with imposter syndrome when my SO spoke on it several times (she’s available to speak to your group if you’d like).

When you are deep in a discipline, it can be very easy to “know what you don’t know” and downplay your expertise. I often am asked to support desktop computers because I work in software (a la this post). But I know how little I know about the problem.

I think the issue is also exacerbated by the continuous flow of information that we are all offered by the internet. This makes it very easy to compare ourselves with what other folks choose to share (typically, though not always, their best side and successes). This makes me, I will be honest, feel inadequate. Why didn’t I learn more about k8s? Why haven’t I built a successful saas business? Why haven’t I worked at scale like that? Why haven’t I built a react native app? And so on and so on.

And when someone asks me “can you do that?” I always have that moment of fear and have to force myself to say yes.

My answer is to breathe, take chances, remember that failure is an option, and recall that while we see other people’s successes, we rarely see their failures. It isn’t fair to me to compare my “inside” with someone else’s “outside”.

Lifehack Social Media Tips | moore | | Leave a comment