Creating a CircleCI orb to authenticate a user during a build

I’m a big fan of automating all the things and I also believe in the DRY principle. I’ve been using CircleCI for years and noticed that they had added a way to abstract away some repeated configuration called orbs. I recently built one and wanted to share my experience.

If you are thinking about building an orb, first take a look at the list of existing orbs. After all, it’s better to reuse code someone else will maintain if it does what you need.

In my case, I wanted to explore building an orb. I dreamed up a use case for which no one else had written an orb. The situation is that you store user data in FusionAuth. Each time a build runs, you want to verify that the user running the build is active andvalid before continuing the build. If the user can’t authenticate, fail the build.

Set up the authentication server

I set up a FusionAuth server using their instructions on EC2. It can’t be on localhost because CircleCI needs to communicate with it during the build. The server didn’t run well on a t2.micro size so I ended up springing for a t2.large, where it worked fine. I also had some difficulty installing mysql on the Amazon Linux AMI, but this SO answer helped me out. I added a FusionAuth application and user via the admin panel. I also got an API key. I limited the API key drastically; it could only post to the login endpoint. I tested access with curl like so (ALL_CAPS strings are placeholders you’d want to replace with real values):

curl -s -o /dev/null -w "%{http_code}" -XPOST -H 'Authorization: API_KEY' \
-H 'Content-Type: application/json' \
-d '{ "loginId": "USERNAME", "password": "PASSWORD", "applicationId": "APPLICATION_ID" }' \
http://FUSION_AUTH_SERVER_HOSTNAME:9011/api/login|grep 202 > /dev/null

FusionAuth returns a 404 status code if the user isn’t authenticated successfully (they don’t exist or have an incorrect password) and 202 if the user logs in successfully (more API information here). Note that this login process may skew some of the reporting (around DAUs, for example) so you’ll want to create a separate application just for CI/CD.

The curl + grep statement above exits with a value of 0 if the user successfully authenticates and with a value of 1 if authentication fails. The former will allow the build to continue and the latter will stop it.

The check I want to run worked. Now I just needed to build and publish the orb.

 

White orb

Building the orb

An orb is reusable CircleCI configuration. You have three kinds of configuration you can re-use:

  • jobs: a normal CircleCI job, but you can also pass in parameters. A great option if you have a job you want to use across different projects; something like ‘run this formatting tool’.
  • executors: an environment in which to execute code (docker container, vm, etc).
  • commands: a set of steps that can be reused. It is lower level than a job and can be used across different jobs.

I chose to create a single command. I also created a job to run while I was developing (I added it as a project in CircleCI). You can see the full source code here.

Here are the interesting bits where I define the command to be shared (ah, yaml):

commands:
  verifyauth:
    parameters:
      username:
        type: string
        default: "user"
        description: "FusionAuth username to try to validate"
      applicationid:
        type: string
        default: "appid"
        description: "FusionAuth application id"
      hostbaseurl:
        type: string
        default: "http://ec2-52-35-2-20.us-west-2.compute.amazonaws.com:9011/"
        description: "FusionAuth host base url"
      password_env_var_name:
        type: env_var_name
        default: BUILDER_PASS
        description: "The user's FusionAuth password is stored in this environment variable"
      fusionauth_api_key_env_var_name:
        type: env_var_name
        default: FUSION_AUTH_API
        description: "The FusionAuth API key is stored in this environment variable"
    steps:
      - run: |
         curl -s -o /dev/null -w "%{http_code}"   -XPOST -H "Authorization: ${<< parameters.fusionauth_api_key_env_var_name >>}"  -H 'Content-Type: application/json' -d '{ "loginId": "<< parameters.username >>", "password": "'${<< parameters.password_env_var_name >>}'", "applicationId": "<< parameters.applicationid >>" }' << parameters.hostbaseurl >>api/login |grep 202 > /dev/null
      - run: |
         echo User authorized

The command verifyauth takes parameters. These have defaults and descriptions. Anything that you wouldn’t mind seeing checked into a source code repository can be passed as a parameter. You then call the command in your job and pass parameters as needed (we’ll see that below).

However, there are sometimes secrets which need to be stored as environment variables (in the project or the context): API keys or passwords, for example. However, I still wanted to make them configurable by whoever uses the orb. Enter theĀ  env_var_name parameter type. This type lets the user specify the name of the environment variable. If I set the password_env_var_name to AUTH_CHECK_PASS, then I need to make sure there is an AUTH_CHECK_PASS environment variable set somewhere in my project containing the password with which we’ll authenticate against FusionAuth. This lets the orb be both configurable and secure.

Finally, you can see that the first step of the command is posting login data to the authentication server. Again, if we see anything other than 202 we fail and the build stops. (You’ve seen that curl command before.)

Publishing the development orb

To be able to use the orb with a different project, I needed to publish the orb (I could have developed the orb inline to avoid this). The publishing instructions are here. The only issue I ran into was that I had to update my CircleCI organization settings and allow “Uncertified Orbs” before I could create a namespace. After that I was able to publish a development version of my orb:

circleci orb publish .circleci/config.yml mooreds/verifyauth@dev:testing

I was in the directory of my orb code and referenced my config. mooreds is my orb namespace, verifyauth is the orb name (which is arbitrary and not connected to the source repository name in any way) and dev:testing is the version of the orb. Note that there are two types of orb version: production, which strictly follow semantic versioning, and development, prefaced by dev: and after string that can contain “up to 1023 non whitespace characters”. Development orbs have other limitations: they are not public, are mutable and only last for 90 days. You’ll want to publish your orbs with production versions if you are using them for any purpose other than prototyping or exploration.

I published my orb via the command line, but the docs outline publishing via a CircleCI job.

Testing the orb

Now I wanted a second project to test the orb. Here’s the project source code. Here’s the interesting code:

...
orbs:
  verifyauth: mooreds/verifyauth@dev:testing
jobs:
  build:
    steps:
      - verifyauth/verifyauth: # when called from external job, command is namespaced to by the name of the orb
          username: "circlecimooreds"
          applicationid: "98113cee-d1a8-4abf-baf5-a6ea742f80a1"
  ...

You can see that I pull in the orb at the development version, which I’d previously published. Then I call the namespaced command with some parameters. For this command to work, I also needed to set up required environment variables (in this case, BUILDER_PASS and FUSION_AUTH_API because I didn’t pass in any of the env_var_name parameters). If you don’t set those environment variables (or, alternatively, set the parameters to different values, and then set those environment variables), the build will fail no matter what, as the API call won’t succeed.

I then pushed this sample project up to CircleCI and ran a few builds to make sure the parameters were being picked up.

Publishing the production orb

Now that I had an orb that is parameterized and exposed the command we want to share, I needed to publish it for everyone to use. Note that your configuration code is entirely exposed if you publish an orb. You can see the source of any orb via the circleci orb source command. circleci orb source mooreds/verifyauth@0.0.2 will show you the entire source of my sample orb. They warn you a number of times about this.

To promote to production an orb that you have published to development, update the dev version: circleci orb publish .circleci/config.yml mooreds/verifyauth@dev:testing to catch any changes and then promote it: circleci orb publish promote mooreds/verifyauth@dev:testing patch.

Note that the patch argument at the end of the promote command bumps the patch number (0.0.1 -> 0.0.2) but you can also bump the minor and major numbers. Any changes you make to a production orb require you to publish and promote it again; production orb versions are immutable. For instance, I wanted to update the description of some parameters, but had to publish an entirely new version.

After publishing, you’d want to update any projects that use the orb to use the production version.

Publiished orb listing

The listing of my published orb

Areas for further work

This was a slightly contrived example. I wanted to gain some experience both with FusionAuth and with CircleCI (I have friends who work at both companies). There are a number of areas where this could be improved:

  • authenticate against a different authentication server (LDAP, Okta, AWS IAM)
  • store additional information about the user in the authentication database (for instance, which projects they can build) and convert the authentication curl command into an authorization command
  • run the identity server over SSL (I just used HTTP because it was easier to get up and running, but that’s obviously a production no-no)
  • pull the user and password from the build environment. It’s pretty clear to me how you’d pull the user (there’s a CIRCLE_USERNAME environment variable) but I’m not sure how to pass the password. I can think of a couple of solutions:
    • don’t login at all, just allow the API key to pull user data and match on the username (this is probably the best option)
    • pass the password via a pipeline parameter, which means you’d have to set up an API call to build
    • have one common password for all users in the FusionAuth system, and use it only for access control to the build pipeline
    • make the password the same as the username in the FusionAuth system, and use it only for access control to the build pipeline

In conclusion

If you want to interact with external services from within CircleCI, check out the list of existing orbs.

If you have a service that you want to make it easier for CircleCI users to interact with and use, create an orb and publish it.

If you are working with CircleCI and have duplicate configuration that you want to share between projects, setting up your own orbs is a great idea. Orbs are flexible and easy to parameterize. If you’re OK with your configuration being public (it wasn’t clear to me if there was any way to have the configuration kept private), you can encapsulate your build and deploy best practices in an easy to consume manner.


The Case For Space

Sunrise over a planetI recently read “The Case For Space“, by Robert Zubrin. It was great. Now, I have a degree in physics, but it’s been a long time since I did any math more complicated than algebra. So I can’t speak to the nuances of his calculations–I didn’t verify them. (A google search for reviews where people work through the math doesn’t turn up anything either.)

But I thoroughly enjoyed this overview. This book is in two parts.

The first covers where we are in space exploration now, and where the physics can let us be. He spends a lot of time on what we could do right now. But he also write a number of chapters on where we can be if there are scientific advances in energy generation (namely fusion). The author starts with low earth orbit (LEO) and then heads to the moon, Mars, the asteroid belt, the gas giants and beyond.

This section is a lot of fun. Zubrin covers areas that I never considered to be related to space (the power of transorbital flight to make the earth even smaller than it is today for travelers). He also talks about the economics of space flight and colonization. What exactly will the moon settlers or Martians have that they can sell? There’s a reference to a three way trade between Earth, Mars and the asteroid belt. I really enjoyed the details he dove into. For example:

“In the almost Earth normal atmosphere of Titan, you would not need a pressure suit, just a dry suit to keep out the cold. On your back, you could carry a tank of liquid oxygen, which would need no refrigeration in Titan’s environment…and could supply your breathing needs for a weeklong trip…”

The second part is supposed to be more inspirational, as if getting to space wasn’t inspirational enough. He covers various reasons (freedom, security, survival) that we should be getting off Earth. I found this section less exciting, but I can understand why he included this–if you aren’t excited about space travel for the adventure, Zubrin might convince you in this section.

There were parts of this book that dragged on a bit, but for the most part the prose is accessible and the math skippable. The real world plans that he outlines, especially at the beginning of the book focused on reusable spacecraft technology, LEO, the moon and Mars, are fascinating for their audacity and specificity. Recommended.


Why you should have a startup accountability email list

I mentioned this to some friends a while ago and thought it was worth writing up. I think if you are starting a company, you should absolutely have an email list of interested folks. Mailchimp makes this free. You can email to this list:

  • Progress you’ve made
  • Help you need (hiring, funding, finding customers, anything else)
  • Things you are proud of
  • Things you are bummed about
  • Interesting topics in your space

We used this at a startup I was at a couple of years ago and found it super useful. Even if no one reads the email (and people will!) just the act of putting it together will force you to acknowledge how much you have accomplished (or let you know what you haven’t done, which is even more useful). Some of these people will want to help, and this gives them a low effort way to keep in touch and possibly help when you have a need. I think it’s easier for you, too, to ask for help if it’s a regular newsletter that you’ve been sending (that has been providing them some value), rather than a one time ask.

How often should you send this? I think that every week is good at the beginning, when your startup is especially fragile. Then, as things get going, you can dial it back in frequency, but I’d have it happen every month if possible.

Who should you put on the list? Friends, family, possible investors. Anyone who you trust who you want to keep in the loop with your progress. You may share financials or pretty vulnerable asks on this list, so I’d avoid anyone who you wouldn’t ask for assistance from face to face. But if you are chatting with someone at a party and they are interested in and seem like they could help your startup, just say “Hey, I send out a monthly newsletter about our progress, would you mind if I added you?”



© Moore Consulting, 2003-2020