Developers need the ability to switch scales. I don’t know if this is crucial for any other profession (maybe a general contractor building a home?). For developers the ability to zoom in and focus on a specific problem for a few hours, and then zoom out and focus on the bigger picture, whether that is project management and reporting, or system architecture, is crucial.
The way I do it is to compartmentalize rigorously. This means that when I am focused on a bug, I’m focused on the bug. Sometimes I’ll timebox such investigation so I don’t get wrapped around the axle of the problem. If I encounter other issues, I file them off in the issue tracker (you do have an issue tracker, don’t you)?
If, instead, I’m focusing on the big picture, paradoxically it can be harder to focus. I think best when writing, so I’ll start out with a document outlining the issue. This has the added benefit of allowing me to easily collaborate (google docs FTW), refind the problem definition and solution, and revisit it over time. Other folks work better with other artifacts (app click throughs, diagrams) but the information density and flexibility of written communication work best for me. However, either way I need to set aside some time to focus on the question at hand.
Once you compartmentalize successfully, you can start to have the each scale inform decisions made at the other. Bugs that may feel urgent to fix can be deferred because that system is not accessed often, or will be rewritten soon. System diagrams may be reworked because you know that sub components belong together (or don’t). You may know where the “dark, scary” areas of the application are and make time to rework them. There may be patterns of code or services worth extraction.
One of the hardest parts of management, from personal experience and discussions with others, is letting go of the details. As an engineering manager, you should absolutely let go of critical path implementation items–you shouldn’t be working on the hardest problem or any key components. Depending on the size and needs of your team, your focus may be on recruiting, providing political cover or knocking down impediments to team goals. But I think you need to have a touchpoint at the lowest level of your application. Whether that is achieved through non blocking code reviews, fixing non critical bugs, or knocking out a customer support request, zooming in to the implementation level will inform your worldview. Just don’t let it be too large a focus; definitely don’t let it block your team.
If the idea of letting go of product implementation doesn’t appeal to you, you may want to dive in to management more. There are plenty of details and challenges to management (read High Output Management for a great intro). In fact, there are similarities between building a successful piece of software and building a team that builds a successful piece of software. People aren’t code though–imagine if when you ran a function you got a slightly different result based on how the function felt, how its home life was, what previous work the function had done, etc, and you’ll have some idea of the complexity of management.
If you’ve tried management and it doesn’t work for you, you can remain an individual contributor. Realize this will have an impact on your compensation (in most orgs) and your leverage (link) to effect change.
But it may be worth it to remain a builder.