Icon blog
Triplebyte Blog

We help engineers join great companies
Try our coding quiz

Back to Blog

12,000 engineers evaluated

By Triplebyte on Nov 7, 2016

12,000 engineers evaluated

We launched Triplebyte with the goal of building the first credentials blind hiring process for engineers. Our mission is to give anyone who has the right skills, the opportunity to work at the best technology companies in the world regardless of what school they went to or which companies they've worked at.

We've now evaluated over 12,000 engineers without using their resumes. We've done this by designing a two step process. The first is an online programming test. If you do well on the test, the next step is a technical interview with our interviewing team where the interviewer knows nothing about your background.  We've now interviewed over 2,000 engineers and 15% made it through to the final step of being introduced to the companies we work with.

To put that into context, companies at the size of Airbnb or Dropbox would expect to do technical interviews with approximately 50 engineers a month. We're already interviewing 3x that number every month and we've built software to track every tiny detail of what happens during these technical interviews. This means we're getting data on how to accurately interview an engineer, faster than anyone. In total we've now done over 3,000 hours of technical interviewing, or 127 full days. 

Once an engineer makes it through our process, we match them with companies they'll be a good technical fit for. As we wrote before companies disagree significantly about the types of engineers they want to hire. We're optimizing our matching process for accuracy so we gather as much data about the technical preferences of the current engineering team and use that to match engineers with them. We get this by having the current engineering team complete a technical questionnaire that gives us a fingerprint of their hiring preferences. This matching model is working really well. At our partner companies like Dropbox and Cruise, we're seeing offer rates on our candidates of over 60%. That's more than 2x better than the average they see on their own candidates (the industry average is about 25-30% onsite to offer rate i.e. about 1 in 4 engineers who make it to onsite will receive an offer) and our candidates are going straight to an onsite interview, skipping recruiter and phone screens.

What's really exciting about such a high offer rate is that we're achieving it without doing any culture fit screening. Our process only looks at technical skills and that's the data we use for matching engineers to companies. That shows the way for companies to hire more engineers is to get better at identifying candidates with the right skills early on, not doing more culture fit screening early on.

We can also beat companies on the most important metric of all - the number of internal engineering hours they have to spend per new engineering hires. Sequoia recently estimated that it takes a company 82.5 total hours to hire an engineer. Around 30-35 of these are engineering hours. We're able to deliver an engineering hire at an average “cost” of 15 internal engineering hours. 

We're only able to interview so many engineers each month because our programming test can accurately identify good engineers. After many iterations our test now has 70% precision (precision means of the engineers our test identifies as being good, how many of them actually are). This means we can identify great programmers with much higher accuracy than a resume screen and only interview the good ones. 

What's unique about the programming test is that we've iterated on the questions we ask using actual data from interview outcomes at the companies we work with (i.e. we correlate performance on specific questions with performance on the onsite interviews at companies like Dropbox). We only keep the questions that have high signal and continually replace low signal questions with new ones. This process is the only way you can build a test that is actually accurate at screening for what companies want. 

The data set we're building is unique and we use it to improve the accuracy of our process over time. The data set is unique because:

  1. We track a lot more data from each interview than a typical company would. We've built specialized software to track data points like how long it takes to complete each section of a problem and what the interviewer is thinking every 5 minutes throughout the interview.
  2. We also get data on how our engineers perform on the onsite interviews at the companies we work with. We use this data to build a predictive model specific to each company, based on our engineering genome, which we continually update over time.

It's really exciting for us to see that a credentials blind technical evaluation can work at identifying good engineers and matching them to the right companies.

Back to Blog

Liked what you read? Here are some of our other popular posts…

How to Interview Engineers

By Ammon Bartram on Jun 26, 2017

How to Interview Engineers

We do a lot of interviewing at Triplebyte. Indeed, over the last 2 years, I've interviewed just over 900 engineers. Whether this was a good use of my time can be debated! (I sometimes wake up in a cold sweat and doubt it.) But regardless, our goal is to improve how engineers are hired. To that end, we run background-blind interviews, looking at coding skills, not credentials or resumes. After an engineer passes our process, they go straight to the final interview at companies we work with (including Apple, Facebook, Dropbox and Stripe). We interview engineers without knowing their backgrounds, and then get to see how they do across multiple top tech companies. This gives us, I think, some of the best available data on interviewing.

Read More

Does it Make Sense for Programmers to Move to the Bay Area?

By Mark Lane on Dec 14, 2016

Does it Make Sense for Programmers to Move to the Bay Area?

If you’re a programmer considering a move to the Bay Area, you probably know at least two basic facts: 1) tech salaries are higher here than elsewhere, and 2) living here is really expensive. Both facts have been true for a long time, but they have become especially true in the past four years. Since 2012 home prices have risen by about 60% and rents by about 70% in both the San Francisco and San Jose metro areas. The absence of any apparent upper limit to these increases has given rise to a new journalistic subgenre, the Bay Area Housing Horror Story. Maybe you’ve heard about the cheapest house in San Francisco, a $350,000 “decomposing wooden shack” whose interior is “unlivable in its current condition”? Or the tent next to Google X that was renting for $895 a month? Or the guy on Reddit who calculated that it would be cheaper to commute daily to the Bay Area from Las Vegas by plane than to rent an apartment in San Francisco?

Read More

Bootcamps vs. College

By Ammon Bartram on May 19, 2016

Bootcamps vs. College

Programming bootcamps seem to make an impossible claim. Instead of spending four years in university, they say, you can learn how to be a software engineer in a three month program. On the face of it, this sounds more like an ad for Trump University than a plausible educational model.

But this is not what we’ve found at Triplebyte. We do interviews with engineers, and match them with startups where they’ll be a good fit. Companies vary widely in what skills they look for, and by mapping these differences, we’re able to help engineers pass more interviews and find jobs they would not have found on their own. Over the last year, we’ve worked with about 100 bootcamp grads, and many have gone on to get jobs at great companies. We do our interviews blind, without knowing a candidate's background, and we regularly get through an interview and give a candidate very positive scores, only to be surprised at the end when we learn that the candidate has only been programming for 6 months.

Read More