How to solve the “this application is slow” type of problem

How do you feel when your boss says “the application is slow, please speed it up”?

Personally, my heart sinks and then I get excited. Though they can be frustrating, this kind of thorny performance issue is fun if you look at it the right way.

Whenever I’ve tackled problems like this, the first thing to do is define the start and the finish line, as precisely as you can.

This requires understanding the application’s behavior and architecture. Where is the data stored? How is it presented? How is it modified? Are there types of operations that happen regularly? What exactly is “slow” about the application?

Try to avoid jumping to conclusions here. I understand the temptation to make a change as soon as you think of one that might help, but it’s better to approach this systematically.

Suppose you find out the issue is the database. Operations are too slow and the CPU is not pegged. You know the type and version of the database, how the application calls it, and more.

Here, a good finish line might be “we need to be able to handle updating the main table with 50k items in 1 second”. If you don’t have a precise finish line, this type of work can be endless and frustrating. After all, it is almost always possible to “make it faster”, but you will reach the point of diminishing returns.

If possible, set up a test scenario/system that you can run through repeatedly as you make changes. If not, figure out some other way to test that changes have a positive impact.

Next brainstorm possible solutions and think of two numbers for each: level of effort and hoped improvement. Doesn’t need to be too precise, a scale of 1-5 is fine. Here’s an example for a database bottleneck:

• upgrade the size of the database. LOE low, impact medium
• increase the disk speed: LOW low, impact high
• running explain plans and adding suggested indices: LOE medium, impact high
• offloading operations to read replicas: LOE high, impact high
• …etc, etc

Then start doing low effort, high impact changes. Run through your scenario and tests after each one. See if you get closer to the finish line. Rinse and repeat.

This type of performance issue is a case where hiring an outside consultant/contractor can make sense. You don’t have to spend a lot, since the scope of work can be limited. They can work with you to define the start and finish lines as well as possible steps if you don’t have the time or knowledge to do so. Then have an internal team take the specific actions and test each change to see if it helps.