Useful Rails Gems: dossier

Writing reports is something that is crucial to any business application.  There are a number of nice reporting solutions.  I haven’t done a survey recently, but I know that Pentaho has an offering and Crystal Reports is another one.  Note that the gorilla in the room is MS Excel, because most anyone doing a report is going to want to pull it down and do additional data manipulation.  You should never try to compete with Excel.

For The Food Corridor, we didn’t need a full featured reporting solution, but we did need to pull data from the SQL database and expose it in CSV and HTML tables.  The dossier gem takes care of this.

What I love about dossier is that you can write the reports in either Active Record syntax or in SQL.  I know SQL pretty well, so it’s easier for me to use that avenue, but if you’re just getting started with Rails, I could see how the Active Record syntax would be helpful.  After writing the query, dossier takes care of much of the grunt work of CSV generation or HTML table generation (there are other output formats, but I don’t use them, so can’t speak to them).

You can preform arbitrary formatting on the SQL response, including hiding columns and running ruby code on the value pulled back from the database.  The report class is just a ruby object, so you can also reuse chunks of logic or SQL across reports.  Column names are turned into report headers.  (I wrote previously about testing such reports.)

Authentication is a bit weird out of the box but works.  There’s also documentation for other authentication/authorization options.  Dossier can also take database configuration from the initializer file so you can easily offload your reporting traffic to a read only replica.  (We’re not there yet in terms of traffic, but I can see the day coming.)

The dossier gem is so easy to user that I can get a full featured reporting solution created in only an hour beyond writing the SQL.


Evolution of admin features

Time is short in a startup and the edge cases are many.  I have noticed that when I am building features for The Food Corridor, especially admin or edge case features, there is a progression based on frequency of feature need.  An example of this is something like handling a refund, which is not a core use case.

  1. The first time, the issue is handled via a developer.
  2. If it happens more than once, the process is documented in a google doc.  It might be triggered by an email from another part of the system.  We have an entire google folder called ‘operations’ which is full of such documents.
  3. The developer automates the process via a rake task.  The rake task is a thin layer around a service class, which is tested.
  4. Non technical admins get access to the process via a web interface.  The web interface plugs into the service class as well.  This web interface may handle only part of the issue, and require access to other systems (in the case of a refund, access to the payment processor).
  5. Non admin users get access to the process via a web interface.  This web interface is fully fleshed out.

For core functionality, you obviously want to push the ability to self serve to the end user as soon as possible.  It simply wouldn’t make sense for us to build out the ability to schedule bookings, limit it to a developer or admin user.

But for a lot of functionality, this progression is helpful.  The developer has time to fully understand the issue by handling it him- or herself.  The documentation generated by handling the issue manually is a great start for the requirements document.  And if an issue only pops up once every quarter, or even more rarely, a minimum amount of developer time is spent on it.  If something happens often, it is far more likely to get automated.


Moving from a feature to a product to a company

This post from Seth Levine discusses the path of a startup along this continuum:

Features provide specific point value to users. Products stitch together related features into bundles that are essentially universal in their need across the problem set you are solving (put another way, if each of your users buys your “product” for a different reason you’ve probably just created a feature set, not a true product). Companies have product that is broad enough in its use and impact that a huge number of users gain value from it in a market that is both large and where the user need is similar enough to drive broad adoption of the same solution.

It’s an interesting perspective that is worth evaluating your current organization against.  Are you in the feature, product or company phase?  Rapid application development with libraries and modern frameworks makes it really easy to move from a feature to a product phase, even though some organizations that stay in the feature space–something like this store locator app or a data product like Mattermark where extra work is required to gain value.  We certainly found that when we started The Food Corridor that features alone weren’t going to be defensible or sustainable, so we moved quickly on to being a product.

It’s also worth noting that “Company” status for a VC like Seth is probably a lot larger than a sustainable SaaS company which can feed a small team and provide a lot of value to a niche market–companies like Small Farm Central or Fileboard.  You don’t have to be a “Company” to be a success, in other words.  You don’t even have to be a “Product”.  But knowing where you are in the continuum can help you define your focus.


Founding engineer or Founder/CTO?

I’ve seen a number of great posts about the contrast between VPs of Engineering and CTOs for startups.  Here, here and here. The short version, if you don’t want to read those posts, is that a CTO is technology oriented and a VPE is process and team oriented. At a young startup there is significant overlap. If you are a software startup, you need to be aware of this split.

How quickly these roles need to be delineated depends on the size and growth of your startup. If you have one or two technical folks, no need to do this (but you do need to determine who is in charge of technology). If you have ten engineers and are planning to add five more in the next six months, you should have already split these roles up. But today I wanted to write about the difference between being a CTO, the typical title for the founding technical member, and founding engineer. This is based on some discussions I’ve had or read, most notably with Jason Cole, who runs a consultancy focusing on technical leadership (all erroneous thoughts are of course mine).

When you are early in a startup (seed or pre-seed), you want a developer who can ship product. Nothing is more important than choosing the right market and getting a solution out to that market so that customers can give you feedback (with their dollars). CTOs of bigger companies actually don’t spend any time coding. They work on strategy, partnerships, hiring and other higher leverage activities. But that won’t work for a startup–you actually need someone to build product.

That person is really a founding engineer. Such a person is a special breed. Like an engineer on a larger team, the founding engineer needs to be able to execute and build product, quickly. They need to be aware of technical debt and when assuming it makes sense. They need to write code that works, is testable and performant. They need to evaluate whether to build or buy or download from Github. They’ll be performing devops tasks as well, such as monitoring the application, troubleshooting performance issues, and making sure deployments and rollbacks happen cleanly.

In addition to all of that, the founding engineer must communicate with non technical folks, both within the organization (including the CEO) and outside of the company. The founding engineer may or may not be doing tier 1 customer support, but will be doing tier 2 support. They also have to be willing and able to drive decisions, especially around technical issues. They need to be able to understand and give feedback on the business strategy at some level, as that will determine technical decisions. The founding engineer also needs to able and willing to take market feedback. This may be filtered through others, and any response should be discussed with the other founders, but market feedback and quick iteration is a strength of a startup. The founding engineer may be doing some product management as well, depending on the other team members’ strengths. They’ll be the de-facto UX designer. They also will be interacting with vendors and contractors. They may review budgets and will hire technical team members. It’s possible they’ll interact with investors if funding is sought.

For all the additional responsibilities, the founding engineer should be compensated, well, like a founder. With not much money and gobs of equity. (Not much money is obviously relative, and depends on market value.)

But, at some point the company will either grow or fail. If the company fails, there’s obviously no worries about future roles! But if the company succeeds and grows, the founding engineer can take four paths.

1. CTO

They can grow into the CTO role, focusing on technical strategy. Until there are significant numbers of engineers (10ish) the CTO may continue to code. They may be doing code reviews for some time thereafter. But at some point they’ll be focused on technical direction of the company, competitors and partners, and large scale technical trends that may affect the company’s business.

2. VPE

They can grow into the CTO role, and then transition to a VPE. The reason they have to go through the CTO role first is that it’s unlikely the company will need the process and team building skills of a VPE until a bit later. But at that point they can focus on delivery, team building and process. They’ll likely work with product managers and the CTO on strategic direction of the company.

3. Technical lead

They can choose to remain a technical lead. This will mean that someone is hired over them into the CTO and/or VPE roles. This will be tough on the ego. However, it’s important to realize what brings them joy in life. If they want to stay close to the code, a technical lead role is best. In this role, the entire team needs to be very aware of who is in charge, especially if this transition happens after the CTO or VPE path is attempted.

4. Leave

An early stage startup is a magical thing. Everything is on fire all the time, but the ability to wear many hats, have a direct impact on customer lives and ship quickly may be what they value. If so, as the startup gets bigger, some of these opportunities will disappear. Specialization happens.

There’s nothing wrong with title inflation, but CTO (and CEO, for that matter) titles at a startup aren’t equivalent to the same titles at a company of 100 let alone 500 or 100,000. If you are joining a startup as a technical founder, realize that whatever your title, you’re really the founding engineer, and that, when success occurs, you need to choose.


The power of automated testing

It took me a long time to understand the power of automated testing.  After all, it can end up being a large portion of your codebase and can be brittle.  Sometimes it feels like writing tests “gets in the way” of getting things done.  At one project I worked on, a colleague complained that it felt like you spent 5 minutes changing the production code and an hour changing the tests.  (And to be fair, sometimes that’s true, and there’s a balance to be struck between test code coverage and speed of development.  This can also indicate you need to spend time refactoring your tests, as you have multiple different test components testing the same production code.)

I like to think of tests like a gentle swaddling of your code.  It conforms as the body of your code changes, but changing that code does require some re-work of the tests.  And, if your code fails, it fails into the gentle swaddling, as opposed to the cruel outside world (bleeding all over your production users).  Alright, maybe the analogy fails :).

I write this today because I’m in the middle of a refactor of one of the scariest bits of The Food Corridor.  (Given we’re so young, it’s not that scary, but it’s quite complex–handling the creation and updating of bookings.)  There are many many paths through the code and if I didn’t have automated testing, I’d be far more worried about the changes I’m making.

So, consider this blog post to be a thank you to past me for making future me’s life easier by writing a comprehensive automated test suite.  If you don’t have one, you should.


Rails Gems: paper_trail

Rails has an amazing number of quality libraries (called ‘gems’) and while building The Food Corridor application, I was lucky enough to be able to use some of them.

If you want to easily track changes in your rails models over time, the paper_trail gem has you covered.  Simple to install, compatible with many databases, and very easy to use.  I’d point you to the docs for installation, but for usage, all you have to do is drop has_paper_trail into your model class.

After that’s deployed, every change to those models is tracked, including the entire model state.  From an operational perspective, I’m a bit worried about the size of the audit table (called versions), but thus far it hasn’t grown too fast.  It wouldn’t be hard to prune either.

I use the data this gem records for two reasons.  The primary purpose is to debug the system.  When there’s an issue because the system doesn’t act as expected, or a question on when a model changed, I can go back and see how the model has changed over time.  I can do this either by directly inspecting the versions table or via the rails console.  If I use the latter, then I can actually pull back the model object as it existed in the past.  This has been tremendously helpful in tracking down bugs.

The secondary use of this data is to allow the system to see changes over time, and execute code based on those changes.  This doesn’t happen often, and I’d prefer for such changes to be captured more explicitly, but in at least one case this gem has allowed me to fix an issue in a time efficient manner.

If you’re building a complicated system that changes over time, as most real world applications do, having some kind of audit trail can be extremely helpful.  paper_trail makes it trivial to get an audit trail set up in your rails app.


Useful Tool: Delighted for NPS reports

So, I was looking to add a simple widget to collect net promoter scores (often called NPS).  I was astonished at the dearth of options for standalone NPS tracking.  I assume that most customer relationship software has an NPS tracker embedded in it, but we didn’t want to use anything other than a simple standalone widget.

Two options that popped up: Delighted and Murm.  I did a quick spike to evaluate each of these tools.  At the time I reviewed it, I couldn’t get Murm to work, and they didn’t respond to my customer support request.  Delighted, on the other hand, did so.  They let me have access to web display, which was a beta feature at the time and worked with us on price.  It was trivial to install, though I’m still a bit unclear how the form determines when to display.  Highly recommended.

The nice thing about having a NPS tracker on your website is you can get direct feedback from your users.  This has led to numerous useful conversations and feature requests, as someone who is using our software for the first time brings new clarity to confusing features or user interface.  Plus, it is a great number to track.


Talking to your customers

One of my favorite parts of The Food Corridor is talking to customers.  As the main technical force there, it’s a great opportunity for me to interact with folks whose lives my work is (hopefully) making better (and sometimes making worse).

The two main ways I do this:

  1. I do customer service.  We have zendesk and a common email inbox and I take time away from developing to answer emails.  This gives me a feel for the rough edges of our product and helps me build empathy for our users (“why couldn’t they see that you just click here and then here and then… oh, that’s why”).  It also has led to a number of bug reports that make the product better.  I also answer phone calls from our google voice number.
  2. I schedule a monthly meeting with some of our bigger customers.  These tend to be 15-60 minutes long.  This meeting lets me hear directly from them what they like about the platform, and more importantly, what is missing or broken.  I rotate among a number of different customers because I don’t want to be pulled too far in one direction, and they all have slightly different needs, but hearing from them regularly helps me triangulate.  It’s important to capture what they say in some kind of tracking system, even if you don’t execute against them for a while.  This in person call also lets me let the customer know of certain other features that are new and/or may be of interest.  Frankly, this meeting can be exhausting because of the wish list aspect of it (“oh man, what would it take to implement that?”) but I try to avoid that and just be an open listener.  I think that the customers also enjoy direct access to a developer.  This certainly doesn’t scale as well as option #1.

If you are going to pursue this:

  • make it a priority and realize it is going to affect your ability to deliver code
  • don’t get defensive when your product is criticized
  • take notes
  • seek out customers with a variety of perspectives
  • don’t commit to anything new in this call, but do let them know high level roadmap if they ask
  • ask them about items outside of your product if you have time.  This can clue you in to other problems they may have.

Customer service a different activity from developing software.  It’s very choppy, and you can encounter folks that are … having a rough day and perhaps taking it out on you.  But it also is one of the best ways to make sure that you, the person building a solution, stays in touch with the people using the solution.


Things I wish I knew about Stripe

Caterpillar

Striped, but not charging your credit card

So, at The Food Corridor, we’ve been using Stripe happily since we launched in June of 2016.  As a developer, I’d used Stripe before in a couple of different ways, but this has definitely been my most sustained use of the payment service.  (If you don’t know what Stripe is, it is an API that makes charging customers as easy as an API call.  More here.)

I wanted to outline some of the things I’ve learned from months of using Stripe.

  • Stripe supports pulling money directly from bank accounts, via ACH, but it really isn’t the same ACH as your bank lets you do.  This is because Stripe isn’t a bank.  The biggest thing to be aware of here is that Stripe ACH takes 7 days to arrive in your bank account.  Another issue is that you have to do verification.  They have two ways of doing verification–micro deposits and Plaid.  Plaid is instant, but only supports major banks, which was a non starter for us (updated 9/8: Plaid supports around 1000 banks now).  The code for micro deposits is straightforward, but be prepared for some customer support issues.  Stripe deposits two amounts and withdraws just one amount, which was confusing for some of our users.  It also takes a couple of days, so if your users are hot to spend money, Stripe ACH may not be a fit.  The win?  Definitely cheaper.  (And I didn’t find any other service that would support both credit card and ACH transactions that was developer friendly.)
  • Don’t forget to set up your webhooks out of the gate.  Stripe mentions this, but I glossed over it in the early days, and missed some events that were important.  (The most relevant is that ACH is asynchronous, so when an ACH transfer fails, it is reported via webhook.  If bank account verification doesn’t work, you’ll get a different kind of webhook.  Review the docs and set up webhooks for all the ACH events.)  If you don’t have time for a full featured webhook processing implementation, Zapier can just send the webhook data to your email. This can be a great stopgap solution.  Or you can use stripe_event.
  • Per support, if a webhook post fails (because your app is down, for example), they are retried once an hour for 72 hours.
  • Speaking of stopgap solutions, the Stripe Dashboard is fantastic for manual processes.  Just because you can automate everything via an API, doesn’t mean you should.  There can be some complicated edge cases with payment processing, especially around refunds, but they can easily be handled with a google doc of instructions and the Stripe Dashboard.  I have found only one use case that the API can handle that the dashboard cannot (a partial refund of an ACH transaction).
  • I have found Stripe support to be excellent, quick and knowledgeable.
  • Occasionally customer charges will be declined because of bank fraud triggers.  Expect to occasionally ask your customers to call their bank.  (I think this has happend about once every third month).
  • Disputes are a total pain, because the process is opaque and slow (expect a resolution in about two months and know you are not in possession of the payment during that time).
  • Make sure to capture the payment id anytime you charge a card or run ACH.  It will make future automation a lot easier.
  • Monthly plans are complicated, so if you can lean on Stripe for management, even if you are doing manual plan management (applying coupons, adding, or removing users from plans via the dashboard), do that.
  • The first payment you charge takes 7 days to move from stripe to your​ bank account.  This is for fraud protection.  Payments thereafter typically take 2 days (but it depends on your country and industry).

And here are some special tips if you are using Stripe Connect (their marketplace product).

  • Read the docs!
  • Remember that first payment timeline?  It applies to every one of the connected accounts.  Think about charging your own credit card as soon as you connect an account to help with customer cash flow.
  • Consider whether you want to use managed vs standalone accounts.  Managed accounts are a lot more work but allow you to have a seamless UX that you control.  Standalone accounts, which we use, are far quicker to setup.  I think this depends on the number of sellers you have in your marketplace.
  • You also want to think about whether to place the charges on the platform account or on the connected accounts.  A major factor there is who bears the Stripe fees, the platform or the sellers.  We charged on the platform account because we wanted all our data in one place.  If you are selling plans, you can’t charge on the platform and use Stripe plans.
  • If you are charging on the platform account, and are using standalone accounts (where the sellers have to set up a stripe account) your sellers won’t see charge descriptions unless you manually copy the description over.  The code looks like:

# this will let the sellers know what invoice the charge was for
transfer_id = charge.transfer
transfer = Stripe::Transfer.retrieve(transfer_id, expand: ['destination_
payment'])
payment_id = transfer.destination_payment
payment = Stripe::Charge.retrieve(payment_id, {stripe_account: destinati
on_account_id})
payment.description = description
payment.save

Happy charging!


Advice for the bootstrapped developer

I participated in a panel for Boulder Startup Week, but previous to the panel, thought I’d be giving a 15 minute presentation.  I was going to talk about rules for the bootstrapped developer.  At The Food Corridor, we recently celebrated one year of having customers.  In celebration of that, here are my seven guidelines:

  1. It will take longer than you think.  For any meaningful value of “it”. expect the process to take longer than planned.  Whether that is acquiring your first customer, building your product, finding help, or anything else, plan for it to take longer than you think.  Heck, this blog post is taking longer than I thought it would.
  2. Know your runway.  It’s important to know your financial runway (both for the company and for yourself).  This is fairly easy to calculate–just find out your burn rate and your money in the bank.  If you are looking at your personal financial runway, don’t forget to allow some buffer for you to find a different job–typically you won’t be able to step from a cratered startup on Friday into a full time job on Monday.  However, more important than financial runway is emotional runway.  Talk about the stress with your spouse (if applicable), think about the other emotional difficulties you may encounter, and plan for some high highs and some low lows.
  3. Extend your runway.  Again, for the financial runway, lower your burn rate as much as you can.  This will mean cutting back on spending and savings.  Make sure the family is on board.  Then you will look to find other sources of money to feed and clothe yourself.  This may include digging into savings, borrowing against assets (HELOCs work for this), borrowing from relatives, pitch competitions (TFC won two), selling assets, taking contract work, having a spouse who earns enough for the household, and/or moonlighting.
  4. Talk to your customer. This was one of the great assets that one of my co-founders brought to the table.  She had deep domain expertise and had many connections to potential customers.  We had numerous customers give us feedback, including via interviews, a month long beta test, regular product advisory councils and surveys.  You will note that this point implies you can find your customer and communicate with them in a cost effective manner.  Find where your customer congregates online.  If you don’t find a place, make one and invite your potential customers to join.
  5. Everything is borked all the time. In a startup, you just don’t have time to do everything correctly.  It can be embarrassing, but if you are building something that solves customer pain, and you’ve found the right early adopters, the customers will stick around.  Just keep improving things.  And accept a certain amount of brokenness.
  6. Know your market. This is related to talking to your customer, as that is one fantastic way to learn about your market.  There are other ways too–market research, online reading, etc.  However, as you build your product, you will surprised by what your market wants.  For instance, we thought our market would want a better UX, but were surprised when someone said our v1 UX was “great”.
  7. Make your own rules. Know that my advice and rules above are based on my experience, with my co-founders, product and skillset, in this time.  You need to do the reading and figure out what your rules are.


© Moore Consulting, 2003-2021