Joseph Pacheco is a software engineer who has conducted over 1,400 technical interviews for everything from back-end to mobile to low-level systems and beyond. He’s seen every quirk, hiccup, and one-of-a-kind strength you can think of and wants to share what he’s learned to help engineers grow.
Building software is tough as it is, but then comes along some unknown bug that throws a wrench into all your plans. Our teams and users are relying on us, yet everything is suddenly on hold and we don't even know what's wrong. Oftentimes, our debugging hunches lead down the wrong rabbit hole and we find ourselves at a loss with no end in sight. That stresses me out just thinking about it, yet it's the lived experience for a lot of engineers, including many very smart ones.
Debugging is one of the most challenging activities in all of software engineering. We often look to specific tactics to hone our skill, but they often have limited effect. The real thing that makes for great debuggers is philosophy: foundational principles and orientations that allow for a whole other level of problem-solving skill.
Some Blaze, Most Don’t
Why is it that some of us wade through all the unknowns with relative ease? They don't seem to get especially frustrated, and appear to have this sixth sense for lazering-in on the exact problem with impressive efficiency. This is while others flounder for hours on end with little to no progress. I've personally seen this hundreds of times as a Triplebyte interviewer, albeit on a smaller scale. Some candidates crush all bugs with more than 50% of the time to spare, while others fail to nab one or two before running out. That's a pretty big disparity, and it begs an explanation.
It's easy to chalk this up to intelligence or experience or knowledge of specific tactics. Those things are important and absolutely play some role, but they're not the whole story. There are countless articles with truly excellent tactical advice that not infrequently fall on deaf ears. There's something much deeper going on here. The best debuggers I've seen have one thing in common: a mindset that allows them to wholeheartedly embrace problems. Without this foundational orientation, tactics are useless. It's a prerequisite to effective debugging, and it's grounded in philosophy.
Frustration Is the Root Problem
At a base level, the biggest impediment to successful debugging is frustration. Some activities have a much higher likelihood of being considered frustrating, but no activity necessarily needs to be frustrating. Frustration is a reaction to a situation. It arises when your beliefs about what _should be happening is different from what is actually happening. When debugging, engineers feel frustrated when they want their code to do one thing, but it's doing another — and their expectations are misaligned with reality. It's that feeling of annoyance you have when you internally insist the world be different than it is, and the exhaustion you feel when you deny reality like you're beating a dead horse.
But the biggest problem with frustration is that it literally robs you of the attention you need to focus on solving the problem. All the time you spend focusing on what should be the case is time not spent focusing on what is actually the case. You are paying attention to the wrong thing: your idea of what you want rather than what is. Even if you are actively looking into your code and trying to find the problem, the feeling of being attached to some fantasy ideal gives you less bandwidth to see the problem when you come across it. The only way to clearly see problems is to fully and completely accept and allow the truth to be what it is on an emotional level so your exploration is not encumbered by internal resistance. That's exactly what the best debuggers do, and it's only after that can specific tactics actually be understood fully and applied.
Stoic Optimism: A Cure for Resistance
Letting go of inner resistance is of course easier said than done. It's a knee-jerk reaction that hardly feels like something we have any ability to mitigate. But we do. It's not about denying your emotions (that would just be another form of resisting reality). It's about looking at the world differently so frustration is no longer your natural response to certain forms of adversity.
Lucky for us, this is a
solved problem. In fact, it's been a solved problem for thousands of years in Hindu and Buddhist philosophical traditions. That said, the flavor of these principles that feels most applicable to an activity like debugging is found in the work of Marcus Aurelius. This passage from his _Meditation _ stands out as especially relevant:
The impediment to action advances action. What stands in the way becomes the way.
This is a truly profound point. It speaks to the most fundamental nature of the human being as an actor in the world. What he means is that human beings have goals and objectives, which we can collectively refer to as
desired outcomes. Action is how human beings realize those outcomes. If we could instantaneously manifest anything we want, then action would not be necessary. What stands in the way of action advances action, because it is the reason action happens at all. The obstacle to our desired outcomes is indistinguishable from the path toward those desired outcomes. What stands in the way becomes the way, because, barring magic, the way to the outcome is through one or more of the obstacles.
Ryan Holiday (in his book The Obstacle Is the Way) does a great job of explaining why he considers this idea to be a formula for what he calls
stoic optimism. It's stoic because Marcus Aurelius belongs to the philosophical tradition whose members are known as the stoics, which advocates a philosophy of life that first and foremost accepts that which is not within your control. And it's optimism because by fully and completely accepting that which is not in your control, you are free to see that which is real with such clarity that you are in the best possible position you could be to take action in accordance with what actually is in your control. Stoic optimism is the cure for frustration, because it advocates acceptance of reality as soon as reality presents itself as an obstacle. And by accepting reality you do two things. First, you acknowledge that what currently is cannot be instantaneously changed by snapping your fingers and is thus out of your control. But also, you allow yourself to see what reality actually is so you can take only the kind of action that is truly in alignment with reality which has the highest likelihood of realizing the outcome you want. It's the most empowering principle in all of philosophy and it (perhaps counterintuitively) starts with letting go!
From Unenlightened to Stoic Debugging
The unenlightened debugger digs their heels at every turn. When they encounter a bug, emotional resistance sets in. They look for reasons why it shouldn’t be happening then those reasons occupy their attention. They might criticize the code of other engineers. They might insist something is “impossible” based on their understanding of the code (or overall system in which it exists). They might insist their code is correct and the bug is actually in the Python implementation of a hashmap (or some other outside force). They’re frustrated and annoyed. They might also be indignant. They don’t want to deal with this bug for another second, because they shouldn’t have to because it shouldn’t be happening. They frantically look for the solution, but nothing presents itself and frustration builds even further. Even if they come across the problem, they might miss it because the cause might be something they already declared impossible. In other words, what they don’t realize is that they can’t easily see the problem because they’re not really open-minded about there being a problem in the first place. And emotionally denying the problem is happening makes it basically impossible to consider actual solutions with a clear head.
On the other hand, the stoic debugger is able to get aligned with reality quickly. They see that there's a bug, and they don't waste time by being in denial. They don't get mad at their coworkers for writing bad code, or insist that the bug shouldn't be happening because that doesn't help them solve the bug now! They accept that it's happening right away. And from a place of acceptance, they work within the confines of reality. The assumptions they had going into the bug are all systematically brought into question. Before the bug happened, they might indeed have believed a number of things to be true, but they are now emotionally willing to reconsider every single assumption until they find at least one that is, in reality, not true. That is, they aren't attached to any of their assumptions, and this non-attachment allows them to quickly update their assumptions with new data because there is no emotional resistance to overcome. They test, they experiment, and they develop hypotheses along the way all with a sense of lightness and ease. They go down rabbit holes one-by-one and accept the necessity of doing so as just part of the process. When they finally come across the source of the bug, they notice it, because they’re OK with the bug existing in the first place. They embrace what is completely, so they discover what is easily.
These are of course two extremes, and in reality, we are typically a mix of these two perspectives depending on the day. But if you ever find yourself more aligned with the unenlightened perspective, it’s important to realize that you will get what you want more effectively by integrating stoic principles into your routine. In fact, I’d argue it’s not possible to debug effectively without them.
Non-Attachment and High-Stress Debugging
You may have noticed my use of the term “non-attachment” when describing the stoic debugger. This principle is a foundation of stoicism and the key orientation that allows you to see solutions to problems through a deep embrace of reality. What most of us don’t realize is that this especially applies to high-stress situations. It’s also the antidote to struggling under pressure.
After all, some bugs affect large percentages of your user-base. There are consequences to both introducing bugs and not solving them quickly. You could end up not meeting expectations set by your team, or worse, have your job called into question. For medical apps, bugs can even be an issue of life and death. Even aside from the difficulty of debugging as it is, the pressure can be truly overwhelming in many debugging scenarios.
Yet, the same principle applies. A deep embrace of reality is the key to resolving the stress and solving the problem. Fully embracing reality means not only accepting the details of the bug itself, but also all the possible consequences of the bug existing now and continuing to exist. The only way to focus fully on the bug itself is to let go of your attachment to avoiding those consequences. If you don’t, you again focus on the wrong thing: ideas about bad things that might happen rather than the problem at hand. Embrace of reality means coming into alignment with the present moment so you can, again, work within the confines of what is.
We Can All Reach Mastery
The good news is developing a habit of seeing and accepting reality is a skill that can be learned and honed. I have done it myself over the course of many years, and through a variety of different paths. Stoic philosophers are a great place to start, since they often address these principles directly in a way that's a little more accessible to Western sensitivities. The Power of Now by Eckhart Tolle is similar with a bit of a spiritual twist and a lot more reference to Eastern traditions. Indeed, perhaps no other philosophies embrace non-attachment more fully than those that fall under the Buddhist and Hindu umbrellas. These are all perspectives on the same truths and represent multiple paths to a similar kind of growth. However you come to these truths, the more you practice and internalize them, the more you will be resilient in the face of any obstacle, technical or otherwise. I often say computer science should be taught as a basic skill in schools, because the technical rigor of the kinds of problems you solve forces you to face principles like these on a scale that's digestible, but which can later be applied and expanded as a core problem-solving skill to any area of life. The most successful people in the world know that failure is just a way of updating your data about the world. That's the same for debugging. When you finally approximate the truth well enough, you can solve anything (including all of those pesky technical bugs).