Navigating new systems

A mazeHere are some tips and tricks I have for navigating new software systems, which can sometimes be like navigating a maze. If you’re truly unlucky, it’s a maze, but you’re blindfolded and the walls are covered in randomly placed razors.

The first is to get a clear set of expectations. Will I own the system? Who owns it now? How long have they owned it? How often is it modified? When will it need to be modified again? Is it shaky or stable? Getting these questions answered helps me understand and refine further steps.

The next step is to gain access. There are a lot of different pieces of most modern systems, so access can mean different things. Here are some kinds of access which it may be worth seeking:

  • shell
  • version control
  • database
  • ftp
  • http
  • app level (admin, user)
  • documentation
  • project planning
  • different CI environments (prod, UA, staging)
  • build system
  • admin users
  • end users

After I have access, I like to look at the front end and the back end. By the front end, I mean the user interface. And by the back end I mean the data store. Just looking around and seeing what tables and pages an application has can help.

If the system in question is not entirely custom built, googling for the user guide for the default version of the application can be helpful. Finding that user guide and skimming through it can give me more high level understanding, as well as teaching me key nomenclature. Of course if there is any local documentation, that’s helpful too, but I read that with a skeptical eye, as it doesn’t always keep pace with the system.

I also like to look at logfiles. This can help me determine something as simple as if I’m on the correct server (if I reload the page and the access log file doesn’t change, I am looking at the wrong log file or am on the the wrong server). Even better if the system aggregates logs into something like an ELK system or papertrail.

Setting up a local development environment can help. Again, this lets me gain an understanding of the big picture components, and also lets me poke at various parts of a system, possibly breaking them, without affecting other developers or, worse, customers.

Asking questions is really important, but this can be hard because often the folks with the most knowhow are the busiest.

I also like to see what files or database tables change as I move through the system. With a modest sized database, I do this by taking a database dump before taking some action and then after. Then I diff the files, sometimes using sed to break the dump file apart even further (replacing all commas with commas and newlines, for example). If using mysqldump, you can target individual tables and make sure not to use extended inserts, as that makes diffing harder.

For the filesystem, it’s even easier. I touch a file (ts) and then take the action, then run find . -newer ts -print. This command will show me all the files the system has written that are newer than ts.

Hopefully some of these tips will be helpful to you as you navigate your next new system.


Big hammer or small hammer?

HammerI was talking to a friend the other day about startup vs big company life and he used an analogy so good I’m going to steal it (and expand upon it).

If you think about the problem you are trying to solve as a rock, and the business you are in that is trying to solve as a hammer with which to chisel or otherwise transform said rock, you can choose a brick hammer (which is a small hammer) or a sledge hammer (a large, heavy hammer), or anything in between.

The smaller the hammer, the more effort you have to put into the swing. However, it’s fairly easy to pick up, to manipulate and to re-orient if you decide you need to approach the rock from a different viewpoint.

If you, on the other hand, choose the sledgehammer, then when you swing you are wielding a lot of force. It becomes easier to make progress on your initial approach, but if you need to switch up your emphasis, it’s going to take some time, because of the weight of the hammer.

The larger the business, the more leverage and power you have to attack a single problem. I’ve worked at large companies in the past, and I can tell you the size and scope of problems they were able to work on, often in parallel, were amazing. However, there was a lot of time and effort spent on coordinating those efforts, and and a lot of bureaucracy and red tape if there was a process improvement needed. (There was also dead weight at some of these companies.)

At a smaller company or a startup, as I’ve worked at in the past, I didn’t have the bandwidth to take on multiple large projects. Doing more than one or two major projects was a recipe for distraction and impotence. However, when focusing on one effort, it was easy to try different approaches, work really hard and be super flexible when incorporating feedback from customers and iterating.

There are strengths in both the small and big hammer approaches. The important thing is to choose what is a good fit for both the problem you are trying to solve and your working style (which may change over time).


AWS Quick Starts

StopwatchIf you are looking to stand up an application quickly, I often recommend the AWS marketplace. This service has thousands of vendor maintained solutions and is a great way to get going quickly. Note that some of the solutions have extra per hour charges, and if that is the case per second billing won’t apply. These solutions are focused on individual AWS EC2 instance images (so you can quickly stand up a phpbb instance or a redmine server, for example).

However, another good option is AWS Quick Starts. These are recipes for deployments, possibly of multiple virtual machines, and are aimed at handling larger business problems. There are over 80 listed on the Quick Start page right now, ranging from creating a data lake to a HIPAA reference architecture to running devops tools like consul and bitbucket. These solutions may or may not carry additional charges, so make sure review licensing and billing information as well as functionality.

If you are thinking about setting up a complex system in AWS, it’s worth some time to see if someone has put a reference Quick Start together. It may not fit your needs perfectly, but can be a good place to begin.


How would you build a simple CRUD app in 2018?

Interesting discussion on HN about how you’d build a simple CRUD app in 2018. (CRUD stands for Create, Read, Update and Delete of records in a system.) As you’d expect, no shortage of opinions, with a lot of specifics. Also lots of recommendations to use what you know, which is always a good idea. CRUD apps, where you are simply gathering data via a web interface, run the gamut of complexity and usability, but it’s hard to believe how powerful having a centralized repository of data can be.

And the nice thing is that once you get the data into the backed of  a CRUD app (which is likely a SQL compatible database, but could be any other kind of datastore) you can either combine that data with other systems, expose the data via an API, or both.

Here was my answer to the question:

I’d use Rails today. But I can see the value in Django or Express or ASP.Net or Spring Boot.

 

Criteria I’d think about:

 

What is going to be easy for my organization to support (both operationally and with future code changes).

 

What do I know/want to learn? If the CRUD app is complicated, then I want a tech I know well. If the CRUD app is simple, then I may want to experiment with a different technology (again, within the organizational support guardrails).

 


Look for the value of the param, not its existence

We are rewriting the front end of the application on which I work. Several of the forms which gather data that drives the application previously had checkboxes, which would set a value to true or false in the database.

Some of the checkboxes were ancillary, however, and didn’t result in a database value. Instead, they were used to calculate a different result depending on whether the checkbox was checked or not. That result would then be saved to the database. This was all tested via rspec controller tests.

In at least one case, the calculation depended on the existence of the parameter. Checkbox parameters are sent when they are checked and omitted when they are not.

Then we changed the forms to radio buttons.

Uh oh.

Instead of sending the parameter if the box was checked yes or not sending the parameter if it was checked no, we sent yes if the radio button was checked and no if it was not.

Uh oh.

The tests continued to pass, because they hadn’t been updated to the new values.

Uh oh.

So the calculation was incorrect. Which meant that the functionality that depended on the calculation was incorrect. Unfortunately, that functionality was related to billing. So I just spent the last few days backing that out, checking with clients to see what they actually intended to do, and in general fixing the issue.

The underlying code was an easy enough fix, but this story just goes to show that software is complicated. Lessons learned:

  • favor a test for an exact parameter value rather than parameter existence
  • when changing forms, consider writing tests with the new input values
  • integration tests that drive a browser would have caught this issue. These are much slower so need to be used judiciously, but are still a valuable automated test

 


Own your online identity

This post from Fred Wilson echoes a lot of my thoughts. Whether you are posting to twitter, instagram, facebook, Medium, HuffPost, or anything else, it’s always worth remembering that you are leveraging someone else’s platform. That means you are renting it, not owning it. So, you are at the mercy of the platform. Whether in terms of content lifespan, deep linking or even just how your content is presented, you only have as much control as they allow.

That doesn’t mean there isn’t value in these platforms. There is tremendous value because they aggregate eyeballs, have excellent user interfaces and often host your content for free. Just be aware of the price you are paying (TANSTAAFL).

And always maintain a presence that you own if you care about your content. Repost all your content on your own site, so that even if the third party site goes away, you still have your work.  (If that’s not possible, print a PDF and save it in Google Drive, at a minimum.)

It’s not hard to buy a domain and set up a free google site (I wrote up instructions). You can use WordPress.com with your own domain for only a few dollars a month. Even if it isn’t as trafficked or as beautiful as a Facebook note, you control it. Once you’ve seen a few platforms rise and fall and/or change their business plan, you’ll be happy you used them for their leverage, but maintained as much control of your content as you could.


A Few SQL Database Links

So, here are two different relational database links that I am currently interested in.

The first is PostgreSQL Exercises. This is a fun way to sharpen your SQL. While I’m not a big user of PostgreSQL, I have enjoyed it the few times I’ve touched it. And the basics of thinking in SQL are useful across any database.

The second is Modern SQL. I was pointed to this site (in particular this page on pivots) after asking a question on a slack (so I can’t link to it). My question was about how to transform a set of event values (X happened Y ago, then Z happened at Y+1, and so on) up into counts (X happened N times, Z happened M times). But the entire site is worth reading if you are a SQL nerd, especially the use cases.

Bonus, check out this Twitter thread about using a single SQL database as a data source for multiple microservices. Pragmatism!


Tips for working with offshore teams

Remember “The World Is Flat”? For software development, the world is definitely getting more planar. I’ve had the privilege of working with a few offshore software development teams directly, and have discussed the practice with other engineers.  I wanted to share some of my tips. My experience has been in web development, but I believe these tips apply to most general software development. (If you’re looking for a tiger team/extreme expertise in a specific area, I’d treat an offshore team just like another vendor.)

If you can, pick a team that you have worked with before. Just like onsite software development, knowing personalities, strengths and weaknesses of team members is crucial to delivering quality software. If you haven’t worked with someone before, then ask for referrals and check references.

Pick an easy first project. This could be something not on the critical path, a rebuild of outside functionality, or a rewrite of an existing piece of software. If you have a prototype that you can point to and say “make it work like that”, that can reduce complicated requirements discussions. One successful offshore project I know of took a Cordova mobile application and rewrote the app in objective c and android java.

In general, projects that have a lot of iteration and experimentation are tough to offshore, because of communication latency.  You lose out on feedback cycles.

Have a working agreement. This can be an informal document, but you want to specify roles and responsibilities (who is releasing? who signs off on stories that have been finished? how does planning happen?). Make this document a living one.

Having regular time overlap can speed up feedback, to some extent. If you are offshoring to a country in the same timezone, this is painless.  If you are working with a team from an offset timezone, you may need to adjust your sleep schedule.  It is best if both parties have to adjust their lifestyles somewhat–it is more equitable. Record these overlap hours in your working agreement. Shifting my schedule to have a significant overlap doubled the number of feedback cycles (code reviews, questions) that can be made between myself and an offshore developer.

You need one person to own the relationship on each shore. If there are process questions, these folks sort it out. They can button up any lingering requirements uncertainties. They may pull other folks in to make decisions, but this pair owns the success of the offshoring relationship.

Leverage asynchronous tools. Make sure you use a progress tracker like pivotal or trello. slack is fantastic. So are the modern source code management SaaS applications like github and bitbucket. A live prototyping tool like Invision is useful. Record decisions in writing.

Finally, decide how closely you need to follow the code delivery. You may or may not want to see “how the sausage is made” This can be based on a number of factors:

* how much technical expertise does the onshore team have? Are they interested in acquiring more?
* how much time does the onshore team have? You don’t want the onshore team to be a roadblock.
* will the project be standalone (a marketing website)? Or integrated into the main codebase?
* is what the offshore team writing core functionality to your application and/or business?
* who is responsible for maintenance and changes after delivery?

Many of these tips are best practices that should be followed wherever your development team is.  But they are especially important when they aren’t in the next office.

Offshore development can extend your budget and pull in timelines.  It can help you build a better product, or allow you to access skillsets you might not be able to hire.  You just want to make sure it works.


Turning Zendesk tickets into help center articles

We used Zendesk as part of our ticketing system at The Food Corridor. (Other parts are phone calls and an email inbox.)

I’ve been pretty underwhelmed with Zendesk as a ticketing system–I find it hard to understand and the UX is overly complicated for our needs. However, we recently moved a large number of FAQs into the Zendesk help center, and I found that aspect of the software to be awesome. I’m especially excited to see if it cuts down on customer interaction by being integrated into the web widget.

There’s also a cool way to take comments on tickets (typically with answers to common questions) and turn them into help center articles. Here’s how you do that:

An admin needs to install this Zendesk application and I recommend setting the ‘draft only’ option.

Then, the steps are:

  • Go to a ticket.
  • Click on the ‘apps’ button in the upper right hand corner
  • Click ‘post article’
  • Select the comment you want to post (you can only pick one)
  • Enter your title
  • Modify html if you’re comfortable, but realize it will be a draft article regardless, and you’ll have a chance to change it later)
  • Click ‘next’
  • Choose a section of your help center
  • Click ‘post to section’
  • Click the ‘view’ button (for some reason the edit button didn’t work for me)
  • Click ‘edit article’ (upper left)
  • Modify the content if needed using the guide editor, which is nice and WYSIWYG
  • Change from ‘draft’ to ‘published’ to make the answer available

I remember a knowledge base project over 15 years ago that was aiming at sharing knowledge across our organization. It flailed. Zendesk help center seems to be well on the way to achieving such aims.


Ditching Hourly Rates

Back when I was consulting full time, I typically charged by the hour.  For some clients I’d do fixed bid pricing, based on an hourly guess, but that was typically after we had an established relationship.  Otherwise the risk of losing your shirt is just too high.  I’ve done that once or twice–no fun to be working to finish up a project and just knowing your hourly rate is heading far too rapidly towards single digits.

I’m not consulting now, but if I were I’d be following Jonathan Stark’s advice on value pricing.  You can sign up for his free email course, which is valuable.  After that you’re put on his generic email list which is 25% pitches for his business coaching and 75% good tips about value based pricing.

I will be honest, I’d have a lot of fear about moving to value based pricing, the same way I have fear about niching and focusing on a particular market.  Both are scary concepts because when I am a consultant, the feast or famine nature of the business makes me want to say yes to everyone (within my available skillset and time).

But these will be my first two business experiments if I ever go back to my solo consulting practice.  I’ve just read too many success stories (like this one) to not give it a try.



© Moore Consulting, 2003-2017 +