Icon blog
Triplebyte Blog

We help engineers join great companies
Try our coding quiz

Share this article

I know why rejection emails suck. I write them.

By Kelsey Piper on Aug 24, 2018

I know why rejection emails suck. I write them.

Companies rarely give feedback after interviews. They typically reject potential employees with a generic form letter assuring them, often insincerely, that it was a pleasure to talk with them but they’re not a fit just now. Engineers hate this. And yet, this is how nearly every company writes rejections - in the typical job search, you'll stack up dozens of emails just like this, and maybe hear real feedback once. Why is that?

In this employee-friendly market, lots of things about the hiring process have changed to accommodate candidates. But companies don’t seem interested in the goodwill they could earn by giving honest, individual feedback. I work at Triplebyte, and over the last year I’ve written over 3,000 detailed, individual rejection emails. When I started, I wondered why no one else did this. Now, I think I know.

The number one reason companies cite for not sending feedback is legal risk. Interestingly, I don’t think this is true. Companies put themselves at legal risk if they are rejecting candidates for illegitimate reasons, like race, gender, or a disability. If they send feedback which tells candidates, truthfully, that they were rejected because they didn’t get very far on the coding project, then if anything a company reduces their legal risk: they have a transparent track record of evaluating candidates based only on their skills. I recently talked with an employment lawyer about this, and he didn’t think that specific feedback on technical performance put companies at risk. So legal risk, despite being frequently cited, seems unlikely to be the real driver of policies here.

I’ve come to believe that two factors hold companies back from giving honest feedback when they reject candidates. The first is that giving feedback effectively is an enormous amount of work. The second is that candidates often don’t really want the feedback companies are in a position to offer them.

In one sense, it’s obvious that sending feedback would be a lot of work. Nonetheless, I think people underestimate it. Our system involves interviewers writing detailed subjective feedback, rating the interviewee along many axes we care about, and then offering specific bits of applicable advice. Compiling that into an email, making sure it’s all appropriate and relevant, and identifying resources to recommend for each weak area is my job. (Every few months, I check that all our recommended resources are still up-to-date, available, and free.) That’s the approach we arrived at in order to cut down the time commitment as much as possible - it takes a lot of person-hours, but much less than requiring interviewers to write the emails and find the resources themselves.

This approach, though, only works for a standardized process like Triplebyte’s. The time investment would be much higher anywhere else. For most early-stage companies like us, hiring is ad-hoc and the processes around it are constantly evolving. Interviews are conducted by engineers at the company, who have a lot of other demands on their time and don’t want to spend time writing feedback. There’s no one in place to look over the engineers’ notes and make sure the feedback getting sent is constructive, appropriate, and up-to-date. And trying to do a feedback process when you don’t have the time to do it right won’t build the rapport, trust, and transparency which are the whole point of having a feedback process in the first place. Even Triplebyte only sends individualized feedback to candidates who've done a two-hour interview with us - we simply don't have the resources to do it for everyone who takes our online quiz.

There’s yet another reason not to send feedback emails without the resources to do them right. I said earlier that I think legal risk is a weaker consideration when compared to bad reactions from candidates and the high associated workload. If you're sending thorough, truthful technical feedback, and doing so responsibly, you shouldn't be too worried about this. But legal risk is a much bigger worry if your process for sending rejection emails is ad-hoc and improvised. If you directly send interview notes to the candidates, for example, interviewers might comment on candidate appearance, or gender, or race - and in that case you actually are at serious legal risk. If your process is biased, sending feedback might expose that. Even if your process isn’t biased, if you send feedback that creates the perception you’re biased, that’s enough for a costly lawsuit. So while legal risk isn’t a reason not to send detailed, honest technical feedback (as long as you’re not discriminating), it’s a very good reason not to send carelessly compiled feedback through a haphazard process (even if you’re not discriminating). Legal risk becomes yet another reason that feedback emails aren’t worth it unless you have the resources to do them right.

Finally, there’s the fact that candidates really don’t appreciate feedback that much, unless you’re really good at sending it. This claim is usually pretty controversial. But it’s what I witnessed when I first took over our feedback emails. I heard back from many of the engineers I write to, and often, they were furious. They accused me of being a robot. They obsessed over things that seemed like inconsistencies - you said I'm good at talking about programming topics, and you also said my communication sometimes held me back! Pick one! They sent back long explanations of why we made the wrong decision. It’s hard to compare, but I really think they walked away even angrier with us than if they’d just gotten a polite form rejection. Candidates sometimes walked away from an interview feeling that the interviewer misunderstood them, sized them up wrongly, missed their point. Getting feedback often just confirmed that impression.

To some extent, this problem is impossible to avoid. Even if your interviewing process is sensible, fair and transparent, it won’t be perfect. You’ll make some mistakes, and sending detailed feedback will make it obvious when you do. Even when you don’t make a mistake, some people will take a justified rejection badly. No one enjoys getting rejected, and getting rejected in more detail is not significantly better.

With that said, some changes we made to the process at Triplebyte dramatically reduced negative reactions to our feedback. The first was a simple change to what we emphasized. Instead of telling candidates that they’d demonstrated a poor understanding of a technical subject, I changed the wording to tell them that they hadn’t communicated a strong understanding. When someone can’t answer an interview question about relational databases, it might be that they don’t know anything about relational databases. It also might be that they know them inside-and-out but aren’t used to answering questions on the fly, or that our question didn’t use the vocabulary they’re familiar with, or that they misheard the question and answered a different one. It made quite a difference just to phrase the feedback in a way which acknowledged all those possibilities. People are generally open to hearing that, one way or another, they didn’t manage to demonstrate that they understood a topic. They’re more annoyed if you tell them they don’t know it.

We got similar mileage out of other changes to the wording of the feedback. The Triplebyte interview is wide-ranging, and we don’t expect candidates to be good at everything. We never reject someone who is great in a back-end API role because they don’t know anything about memory management in C. In our feedback emails, though, we mentioned our impression of every section of the interview. Candidates complained in their responses to our feedback that they’d been penalized for lacking knowledge of subjects they really didn’t need to know. We keep giving feedback about the whole interview, but we’ve tried to emphasize the sections that were actually decisive.

Finally, we found that candidates absolutely loathe getting feedback that seems inconsistent or contradictory. Sometimes, people generally talk through their coding process well, but they don't have a strong technical vocabulary and get tripped up when answering engineering questions. Their feedback emails used to read you talk well about programming and you should work on your technical communication. Everyone hates this. They feel like the contradictions reveal we weren't really paying attention. I've worked on rewording our feedback to be more specific about both positives and negatives, so we can tell candidates that they have both strengths and weaknesses in an area without being inconsistent.

With these changes, and others, we now get mostly positive reactions to our feedback. Engineers often make use of it to study, improve, pass our interview, and get a job through us. Giving feedback is positive-sum for Triplebyte at this point. Of course, getting to that point took time, effort, and quite a few iterations. Many companies which attempt feedback emails might be deterred by the initial negative response, and not keep at it for long enough to see benefits.

When I started sending feedback emails, I wondered why we were one of few companies that offer this. A year later, I think I have a better sense of the answer. Sending detailed, honest feedback is a core part of Triplebyte’s mission, and one we’re going to continue to do going forwards, but it’s hard, and it doesn’t build candidate rapport all that easily. It’s not surprising that most companies won’t offer it.

(Interested in getting a job through us? Take our adaptive quiz here and then book an interview. If you do well, we’ll get you a job. And if you do badly, well, I’ll write you a really detailed email explaining why.)

Get offers from top tech companies

Take our coding quiz

Liked what you read? Here are some of our other popular posts…

How to Interview Engineers

By Ammon Bartram on Jun 26, 2017

We do a lot of interviewing at Triplebyte. Indeed, over the last 2 years, I've interviewed just over 900 engineers. Whether this was a good use of my time can be debated! (I sometimes wake up in a cold sweat and doubt it.) But regardless, our goal is to improve how engineers are hired. To that end, we run background-blind interviews, looking at coding skills, not credentials or resumes. After an engineer passes our process, they go straight to the final interview at companies we work with (including Apple, Facebook, Dropbox and Stripe). We interview engineers without knowing their backgrounds, and then get to see how they do across multiple top tech companies. This gives us, I think, some of the best available data on interviewing.

Read More

Bootcamps vs. College

By Ammon Bartram on May 19, 2016

Programming bootcamps seem to make an impossible claim. Instead of spending four years in university, they say, you can learn how to be a software engineer in a three month program. On the face of it, this sounds more like an ad for Trump University than a plausible educational model.

But this is not what we’ve found at Triplebyte. We do interviews with engineers, and match them with startups where they’ll be a good fit. Companies vary widely in what skills they look for, and by mapping these differences, we’re able to help engineers pass more interviews and find jobs they would not have found on their own. Over the last year, we’ve worked with about 100 bootcamp grads, and many have gone on to get jobs at great companies. We do our interviews blind, without knowing a candidate's background, and we regularly get through an interview and give a candidate very positive scores, only to be surprised at the end when we learn that the candidate has only been programming for 6 months.

Read More

How to pass a programming interview

By Ammon Bartram on Mar 8, 2016

Being a good programmer has a surprisingly small role in passing programming interviews. To be a productive programmer, you need to be able to solve large, sprawling problems over weeks and months. Each question in an interview, in contrast, lasts less than one hour. To do well in an interview, then, you need to be able to solve small problems quickly, under duress, while explaining your thoughts clearly. This is a different skill. On top of this, interviewers are often poorly trained and inattentive (they would rather be programming), and ask questions far removed from actual work. They bring bias, pattern matching, and a lack of standardization.

Read More