Skip to content

What senior engineers do: fix knowledge holes

I had a bad week a few weeks back, and got together with a friend and former colleague. We started trading war stories. He mentioned one time when he was working for a company which had both a desktop client and a server component. The communication between the two pieces of the system was completely, utterly undocumented. The original engineers who had written each piece of the app had departed. The system was key to the company’s business (it was what they sold!).

This “knowledge hole” was identified by my friend. He was a software engineer. This was a key piece of software, but was unknown. The problem was not getting solved.

So, he solved it.

He installed the client and a TCP sniffer, and clicked around the client. He recorded all the traffic. He literally reverse engineered the client/server communication protocol. He then documented what the communication protocol was, so the the next group of engineers could benefit. He was was a software developer who was responsible for the back end systems. I don’t think he was directly responsible for the client/server protocol and the client was, I believe, outside his purview. (Updated 7/20) Note, he was a software engineer, not QA or a network engineer. But he didn’t let the role definition stop him.

I thought to myself, this is the textbook definition of a senior engineer. You see a problem, you solve it (thoroughly), you document it and you level up your team.

It was pretty awesome to hear about.

6 thoughts on “What senior engineers do: fix knowledge holes

  1. Pingback: New top story on Hacker News: What senior engineers do: fix knowledge holes – News about world

  2. Pingback: New top story on Hacker News: What senior engineers do: fix knowledge holes – Hckr News

  3. Pingback: New top story on Hacker News: What senior engineers do: fix knowledge holes – Outside The Know

  4. Junior Non-Engineer says:

    I don’t see anything “senior” about it, or even “engineer”.  Seeing problems and solving them is what everyone does.  Documenting the solution is one part of solving a problem.  An apprentice carpenter does these things, too, and so does a farmer, and a waiter.

    Unfortunately, it’s not what most software companies reward, or how they operate.  Whenever I did this, my manager, at every software company I’ve worked for, would say:

    “That’s cool, but you’re supposed to add the FooBar feature, and it needs to be done this Friday.  Don’t waste time with reverse-engineering, or documentation.  Just add one new field to the protocol somewhere.  We can clean it up Later(TM).”

    This is Conway’s Law at work.  What sort of company encourages the creation of two critical components which are completely undocumented?  The sort of company which doesn’t reward documentation of critical components.  That’s not likely to change because the engineer that created them happened to leave.  (It took more time to reverse-engineer the protocol than it would have to document it when the knowledge was fresh.)  The PM and QA who allowed this to happen are still there, right?

    What “Senior Engineer” really means is someone who’s spent enough time in the trenches to have earned a job title that allows them the latitude to make these sorts of improvements, and not have a PM question why they aren’t, instead, doing exactly what they were assigned.

    Look back at the story.  Did the “senior engineer” go through proper channels to schedule a “reverse-engineer and document network protocol” task?  No, he clearly didn’t trust that it would happen.  Or maybe it was already there, but lowest priority (way below “fix CSS on IE”, of course).  What was his actual responsibility that week?  The story doesn’t say, but I don’t see any remarks about a PM breathing down his neck asking about the CSS fix he asked for (because that PM is the only user of the system, anywhere, of course, who uses IE and sees that particular bug).  Documentation is not on this week’s “Sprint”!

    The process is fundamentally broken.  We hear fables like this about how life would be better if we all did something one way (you’ll get promoted to Senior Engineer!), while in practice we’re punished for doing so.

  5. moore says:

    > What “Senior Engineer” really means is someone who’s spent enough time in the trenches to have earned a job title that allows them the latitude to make these sorts of improvements, and not have a PM question why they aren’t, instead, doing exactly what they were assigned.

    That is an aspect I hadn’t really considered. I’ve seen it both ways, where the junior people were put in a box as you describe, and other times where the junior folks were content/wanted to stay within a clearly defined sense of responsibility.

    I’d also say that I’ve been where managers trusted people and encouraged cleanup. It’s never a super easy sell, but if a manager isn’t forward looking enough to realize that software needs maintenance, that seems like a signal to move on. Sorry you’ve had a tough time finding good managers.

  6. K Sandvik says:

    I think the core concept is that someone took responsibility to fix it — which is a sign of a senior engineer.

Comments are closed.