How Trello Wowed Me By Handling An Edge Case

We are using Trello for our product development planning at The Food Corridor. Previously we were using Pivotal Tracker, which I chose, but a new team came on to help us and they were more comfortable with Trello. I may do a compare and contrast of these tools in the future, but for now I wanted to celebrate the beauty of a well designed piece of software.

I don’t mean how Trello looks, though it certainly looks pretty. I mean how they handle UX edge cases. I ran into one the other day, and it blew my mind that Trello acted as I had hoped.

Here’s the situation. We use the Trello numeric card ID along with this git hook:

#!/bin/sh

# from http://stackoverflow.com/a/16061192/203619

if story_id=`git branch |grep '*'|sed 's/.*-//'`
then
    echo "[#$story_id]" >> "$1"
fi

To help tie commits to stories. If someone is working on a story with the id 123, they work on a feature branch called add-new-feature-123. When committing, they may write a message like: “Updated the message to the end user when they save”, and this hook will automatically add “[#123]” onto that commit message.

When someone is looking at the code six months or two years from now, they will be able to look up that story and get context about why the message was changed beyond what was in the commit log.

We were cleaning up old releases in Trello and had moved all the released stories to another board. However, I noticed that the cards were renumbered when they were moved to that board. Whoops! That meant that the commit messages wouldn’t be useful in looking up the cards. I had discussions with the product manager and we decided to keep all future releases on the same board to maintain the numbers–we’d just archive them (it’s worth noting that when you search for 123 and the card is archived, the search won’t return the card unless you add the is:archived search operator to your query).

However, I was ready to write off the cards that had been moved to the other board. What were the chances that if I moved the cards back to the original board, the card numbers would be maintained? I gave it a try just to see.

Trello did the right thing! The cards, when moved back to the original board, assumed their original numeric ID.

I am very impressed, as I imagine there are a very small portion of Trello users who care about this behavior. As someone who doesn’t really care about design but does care about user experience, that is an example of attention to detail that I wanted to call out and praise.


All the ways usernames can go wrong, or a story about why you favor third party solutions

This post on usernames is hitting the top of Hacker News right now. It’s worth a read for an in depth examination of how many ways something that seems fairly simple, usernames, can go wrong. Whether that is allowing impersonation due to unicode code points (see a related XKCD), how to handle email addresses, or what usernames should be prohibited, the simple idea of having someone use a text string as part of their authentication scheme is not so simple.

Here’s a great quote from the post talking about the right way to do it:

So if you’re building an account system from scratch today in 2018, I would suggest reading up on [the tripartite identity] pattern and using it as the basis of your implementation. The flexibility it will give you in the future is worth a little bit of work, and one of these days someone might even build a good generic reusable implementation of it (I’ve certainly given thought to doing this for Django, and may still do it one day).

What I took away from this is that usernames are hard. And that, paradoxically, the amount of business value that I create by doing usernames correct is minimal, until it’s a vector for attack. That means that I’m far better off choosing a third party library or service that focuses on authentication than rolling my own. That third party service is much more likely to have read, understood and implemented the tripartite identity pattern (in addition to any other benefits) than I will. (If I build many systems which require authentication, then maybe I can write my own library of username checks. But then I’d be better off open sourcing it, unless it was a competitive advantage.)

Now, this doesn’t mean I should blindly use any open source authentication library. I still need to examine it, see if it is used and maintained, and determine if it meets my requirements. This is not an “open source good, roll my own bad” post.

But the default should be to find an open source or third party system when I’m working on this type of software plumbing. (For rails and authentication, devise is where I’d start.) If I look around for an hour or two and can’t find anything that meets the needs of the project (either directly, with configuration or with minor code modification), then, and only then, should I start to think about rolling my own. Yes, it’s less fun to configure a third party library than it is to roll your own, but the kind of edge cases that a third party library or service will handle make it a better choice.

This has been an issue for a long time. I still remember walking into a contracting engagement a decade ago and seeing that they had their own database pooling solution. For Java. In 2004. When Apache DBCP had been around for a few years. I’ve been guilty of this myself, often at the beginning of a project when it feels like I need to get up to speed quickly, the problem space seems simpler and sometimes I’m not as familiar with the ecosystem. So this issue isn’t going away, but this post is my plea to my future self to default to third party solutions.


Getting access to the very nice date functionality in non Rails Ruby

I am doing some small ruby scripts for a dashboard and need to do some date calculations, like the timestamp of the first of the previous month and the timestamp of the end of the previous month. Rails makes this so easy with (DateTime.now - 1.month).beginning_of_month. I looked around for a way to do it with straight up Ruby, but didn’t see a good solution.

Luckily, some of the nice parts of Rails have been broken out into the active support gem. You need to add it to your Gemfile or however else you are managing your gems, of course. Confusingly, the gem is activesupport and the require statement is require active_support/... (see the underscore?).

There’s an entire guide on how to pull in just the active support functionality you need. Unfortunately, I couldn’t make the targeted includes work (I was trying to pull in both numeric and date extensions precisely, but kept getting the error message undefined method `month' for 1:Integer (NoMethodError).

Finally, I just pulled in active_support/time and everything worked.


Blast from the past: 5 worlds

5 Worlds is a Joel Spolsky classic. This article needs to be updated (it’s from 2002, when shrinkwrap software was still A Thing) but it still has a lot of wisdom and illustrates just how large the scope of work available to software developers is (even more so now that software is eating the world).

Whenever you read one of those books about programming methodologies written by a full time software development guru/consultant, you can rest assured that they are talking about internal, corporate software development. Not shrinkwrapped software, not embedded software, and certainly not games. Why? Because corporations are the people who hire these gurus. They’re paying the bill.

Note that assuming a software developer is a webdev is like assuming a lawyer is a trial attorney. Just like there’s lots of ways to practice law, there are lots of ways to build software. And, to be honest, this is probably true of every profession. If you go to a party and ask someone “what do you do” and really really listen, chances are you’ll get a startling view of the world, because everyone does something interesting.


Switching Scales

Developers need the ability to switch scales. I don’t know if this is crucial for any other profession (maybe a general contractor building a home?). For developers the ability to zoom in and focus on a specific problem for a few hours, and then zoom out and focus on the bigger picture, whether that is project management and reporting, or system architecture, is crucial.

The way I do it is to compartmentalize rigorously. This means that when I am focused on a bug, I’m focused on the bug. Sometimes I’ll timebox such investigation so I don’t get wrapped around the axle of the problem. If I encounter other issues, I file them off in the issue tracker (you do have an issue tracker, don’t you)?

If, instead, I’m focusing on the big picture, paradoxically it can be harder to focus. I think best when writing, so I’ll start out with a document outlining the issue. This has the added benefit of allowing me to easily collaborate (google docs FTW), refind the problem definition and solution, and revisit it over time. Other folks work better with other artifacts (app click throughs, diagrams) but the information density and flexibility of written communication work best for me. However, either way I need to set aside some time to focus on the question at hand.

Once you compartmentalize successfully, you can start to have the each scale inform decisions made at the other. Bugs that may feel urgent to fix can be deferred because that system is not accessed often, or will be rewritten soon. System diagrams may be reworked because you know that sub components belong together (or don’t). You may know where the “dark, scary” areas of the application are and make time to rework them. There may be patterns of code or services worth extraction.

One of the hardest parts of management, from personal experience and discussions with others, is letting go of the details. As an engineering manager, you should absolutely let go of critical path implementation items–you shouldn’t be working on the hardest problem or any key components. Depending on the size and needs of your team, your focus may be on recruiting, providing political cover or knocking down impediments to team goals. But I think you need to have a touchpoint at the lowest level of your application. Whether that is achieved through non blocking code reviews, fixing non critical bugs, or knocking out a customer support request, zooming in to the implementation level will inform your worldview. Just don’t let it be too large a focus; definitely don’t let it block your team.

If the idea of letting go of product implementation doesn’t appeal to you, you may want to dive in to management more. There are plenty of details and challenges to management (read High Output Management for a great intro). In fact, there are similarities between building a successful piece of software and building a team that builds a successful piece of software. People aren’t code though–imagine if when you ran a function you got a slightly different result based on how the function felt, how its home life was, what previous work the function had done, etc, and you’ll have some idea of the complexity of management.

If you’ve tried management and it doesn’t work for you, you can remain an individual contributor. Realize this will have an impact on your compensation (in most orgs) and your leverage (link) to effect change.

But it may be worth it to remain a builder.


Basecs: Basics of Computer Science

If you backed into software as a career like me (thanks physics degree!), you may not have a firm grounding in basic computer science concepts. This can be hard to gain during your work week, as clients care far more about feature delivery than they do about which data structure you used.

However, having an intuitive sense of how computers work and what basic algorithms and data structures are can enrich your experience as a developer. Why? For the same reason understanding how to grow peas lets you appreciate a fine meal–understanding fundamentals lets you peer down through abstractions and appreciate what you are building on top of.

There are plenty of intro articles and books out there, but I recently stumbled across basecs and have been enjoying it.

The author starts from the beginning (what is binary), covers various data structures and algorithms, and ends with a discussion of just in time compilation. Her prose is comprehensible and she adds illustrations which complement the text.

If you are a developer without fundamentals, like me, you may enjoy this year long series on the basics of computer science.


Feature branch development

I remember when I had lunch with a friend, back when I was using SVN and he was using git.  He said “it’ll change your life”.  I had read about darcs years ago and had read Joel’s post about distributed version control systems.  I was still like “meh.  SVN does what I need it to do–tag releases, keep track of changes, and I can branch if I need to”.

Boy, was I wrong.

I’ve been using git since around 2014, and it’s great.  I don’t remember where I heard it, but someone said “branching in SVN is easy, it’s the merging that is difficult”.  Git takes the pain out of merging (mostly, of course you can still hose yourself pretty well).  I have to also reference this classic XKCD comic whenever I talk about git.

Regardless, one of the things I’m loving about working with git is that if you always use feature branches, use story numbers in your branch name (like story-update-123), and you set up your prepare-commit-msg script, you can track back every change to a story.  Here’s my prepare-commit-msg script:

#!/bin/sh

# from http://stackoverflow.com/a/16061192/203619

if story_id=`git branch |grep '*'|sed 's/.*-//'`
then
    echo "[#$story_id]" >> "$1"
fi

Another nice feature of the branch handling is that you can roll forward and backward branches. I use bitbucket, so I use the GUI revert command, which creates a nice revert PR. (I then immediately revert the revert PR so that I can apply the intended changes in the future by merging the second PR.) This makes it possible to push a feature to staging, realize it isn’t fully baked, and revert it so you can get out another feature release. Perhaps you could do this with SVN or another centralized VCS, but I was never comfortable enough with branches to do it.

All in all, git has been life changing as a developer. Thanks Russ, you were right!


Tips for working with offshore teams

Remember “The World Is Flat”? For software development, the world is definitely getting more planar. I’ve had the privilege of working with a few offshore software development teams directly, and have discussed the practice with other engineers.  I wanted to share some of my tips. My experience has been in web development, but I believe these tips apply to most general software development. (If you’re looking for a tiger team/extreme expertise in a specific area, I’d treat an offshore team just like another vendor.)

If you can, pick a team that you have worked with before. Just like onsite software development, knowing personalities, strengths and weaknesses of team members is crucial to delivering quality software. If you haven’t worked with someone before, then ask for referrals and check references.

Pick an easy first project. This could be something not on the critical path, a rebuild of outside functionality, or a rewrite of an existing piece of software. If you have a prototype that you can point to and say “make it work like that”, that can reduce complicated requirements discussions. One successful offshore project I know of took a Cordova mobile application and rewrote the app in objective c and android java.

In general, projects that have a lot of iteration and experimentation are tough to offshore, because of communication latency.  You lose out on feedback cycles.

Have a working agreement. This can be an informal document, but you want to specify roles and responsibilities (who is releasing? who signs off on stories that have been finished? how does planning happen?). Make this document a living one.

Having regular time overlap can speed up feedback, to some extent. If you are offshoring to a country in the same timezone, this is painless.  If you are working with a team from an offset timezone, you may need to adjust your sleep schedule.  It is best if both parties have to adjust their lifestyles somewhat–it is more equitable. Record these overlap hours in your working agreement. Shifting my schedule to have a significant overlap doubled the number of feedback cycles (code reviews, questions) that can be made between myself and an offshore developer.

You need one person to own the relationship on each shore. If there are process questions, these folks sort it out. They can button up any lingering requirements uncertainties. They may pull other folks in to make decisions, but this pair owns the success of the offshoring relationship.

Leverage asynchronous tools. Make sure you use a progress tracker like pivotal or trello. slack is fantastic. So are the modern source code management SaaS applications like github and bitbucket. A live prototyping tool like Invision is useful. Record decisions in writing.

Finally, decide how closely you need to follow the code delivery. You may or may not want to see “how the sausage is made” This can be based on a number of factors:

* how much technical expertise does the onshore team have? Are they interested in acquiring more?
* how much time does the onshore team have? You don’t want the onshore team to be a roadblock.
* will the project be standalone (a marketing website)? Or integrated into the main codebase?
* is what the offshore team writing core functionality to your application and/or business?
* who is responsible for maintenance and changes after delivery?

Many of these tips are best practices that should be followed wherever your development team is.  But they are especially important when they aren’t in the next office.

Offshore development can extend your budget and pull in timelines.  It can help you build a better product, or allow you to access skillsets you might not be able to hire.  You just want to make sure it works.


Run through the finish line

I used to run cross country way back when. One of the differences between the good runners (like me) and the great runners (Dave F, Todd E) was talent. But drive was more important than talent. One of the key pieces of advice my coach gave us was to “run through the finish line”. On the face of it, it seems obvious–you are at the end of a 5k race, why wouldn’t you want to finish as fast as possible? Why waste that effort? But I can’t tell you how many times I passed (and was passed) in the last 50 feet of a race. I saw the finish line, I relaxed (as much as possible) and slacked. Or, I saw someone else doing that and knew I could move up one more place. If you’ve left it all on the course (easier to do in a 5k than some other races) it is hard to have anything left to run that last bit. But drive can provide that last bit of energy.

In a software project of any size, there’s also that “last 50 feet”. If you are developing, it’s the last 10% of the project that takes 50% of the time. If you are running a project, it’s the last few weeks when all the little things that were put off during the big build out phase come out of the woodwork and need to be dealt with. Or it’s when the project blows up 2 weeks before deadline. When you are dealing with a vendor, it’s the crunch time before you launch, when any organizational complexity ignored comes due.

If you want to get across the finish line successfully, sound of body and mind, and with your team intact, what can you do to finish strong?

Be realistic: I never ran a 15 minute 5k. So starting out at that pace was a sure way to make sure I had nothing left at the 2 mile marker, let alone the finish. Make sure you have realistic expectations at the start of your project. If you or your team don’t know how to scope the work, spend some time learning how to do so upfront.

Cultivate drive: remind the team regularly why this project is important. If there are features that you discover are harder or less useful than supposed at the beginning, don’t be afraid to shift effort. Also, refer back to previous successes or failures for motivation.

Confront complexity early: don’t save it for the finish. That’s like picking up a 40lb sandbag at the 4k mark. Derisk a project by doing the hardest parts first. In some situations you can’t (final integration of components or external system access), but try to do it as early as possible (via CI or mocking up external systems and loudly asking for access as soon as possible).

Hold a reserve: experienced runners know to save a bit for the end. Don’t ask your team to work crazy hours in the beginning or middle of a project, because they’ll have no reserve to push through the finish line. Every software developer I’ve ever talked to knows the last 2 weeks of a project will require extra effort–don’t waste that expectation by requiring more work early on.

Drive is important to finishing a 5k; it’s also important to complete a software project. Running all the way through the finish line can be the difference between a failed delivery and a successful project. Plan accordingly.


Automating one off deployment tasks

When I am deploying a rails application, there are sometimes one off tasks that I need done at release time, but not before. I’ve handled such tasks in the past by:

  • adding a calendar entry (if I know when the release is happening)
  • add a task or a story for the release and capture the tasks there
  • writing a ‘release checklist’ document that I have to remember to check on release.

Depending on release frequency, application complexity and team size, the checklist may be small or it may have many tasks on it. Some tasks on a checklist can include:

  • sending a communication (an email or slack message) to the team detailing released features
  • restarting an external service to pick up configuration or code changes
  • notifying a customer that a bug they reported has been fixed
  • kicking off an external process via an API call now that a required dependency has been released

What these all have in common is that they:

  • are not regular occurrences; they don’t happen every deploy
  • are affecting entities (users or software) beyond code and database
  • may or may not require human interaction

after_party is a gem helps with these tasks. This gem is similar in functionality to database migrations because each after_party task is run once per environment (this is done by checkpointing the task timestamp in the database). However, after_party tasks are arbitrary rake tasks, rather than the database manipulation DSL of migrations.

You add this call to your deployment scripts (the example is for heroku, feel free to replace heroku run with bundle exec):

heroku run rake after_party:run --app appname

This rake task will will run every time, but if an after_party task has already been run and successfully recorded in the database, it will not be run again.

You can generate these tasks files: rails generate after_party:task my_task --description="desc"

Here’s what an after_party task looks like:

namespace :after_party do
  desc 'Deployment task: notify customer about bug fix'
  task notify_issue_111: :environment do
    puts "Running deploy task 'notify_issue_111'"

    TfcAdminNotesMailer.send_release_update_notification("customer X", "issue 111").deliver_now

    AfterParty::TaskRecord.create version: '20180113164945'
  end  # task :notify_issue_111
end  # namespace :after_party

This particular task sends a release update notification email to the customer service user reminding them that we should notify customer X that issue #111 has been resolved. I couldn’t figure out how to make a rake task send email directly, hence the ActionMailer. In general you will want to push all of your logic from the rake task to POROs or other testable objects.

I could have sent the message directly to the customer rather than to customer service. However, I was worried that if a rollback happened, the customer might be informed incorrectly about the state of the application. I also thought it’d be a nice touchpoint, since issues are typically reported to customer service initially. The big win is that ten of these can be added over a period of weeks, and I could have gone on vacation during the release, and the customer update reminders would still be sent.

All is not puppies and rainbows, however. This gem doesn’t appear to be maintained. The last release was over two years ago, though there are forks that have been updated more recently. It works fine with my rails4 app. The main alternative that I’m aware of is hijacking a database migration and calling a rake task or ruby code in the ‘up’ migration clause. That seems non intuitive to me. I wouldn’t expect a database migration to touch anything outside of, well, the database.

Another thing to be aware of is that there is no rollback/roll forward functionality with after_party. Once a task is run, it won’t get run again (unless you run the rake task manually or modify the task_records database table).

This gem will help you automate one off deployment tasks, and I hope you enjoy it.



© Moore Consulting, 2003-2017 +