Triplebyte Blog

We help engineers join great companies
Try our coding quiz

Share this article

FizzBuzz 2.0: Pragmatic Programming Questions for Software Engineers

By Mike Robbins on Feb 18, 2020

FizzBuzz 2.0: Pragmatic Programming Questions for Software Engineers

Over 100,000 programmers took Triplebyte’s quiz last year. The data below reveals how five multiple-choice questions easily separate the real software engineers from the rest.

Like the infamous decade-old Fizz Buzz Test, these questions are trivial for anyone who can build software at a professional level, but are likely to stump anyone who can’t hack it. The questions below are the first five on our quiz, and 98% of successful engineers answer at least 4 of 5 correctly. I’m confident that if you’re an engineering manager running an interview, you wouldn’t give an offer to someone who performed below that line.

Results

Before I show the questions, here’s the data on how successful engineers are based on how many questions they answer correctly:

Success versus Questions Answered

I’ve defined “successful” as someone who receives at least one inbound message from a company that matched their personal preferences, and I normalized to the 5/5 group. Engineers who correctly answered 4/5 or 5/5 accounted for 98% of all successful users on our platform. In contrast, 3/5 or below indicated near-certain failure.

If you’re concerned about circular reasoning, note that we train our quiz scoring ML model against downstream outcomes. The Triplebyte network only works because we help good programmers get filtered inbound opportunities that they actually like, based on skills, regardless of resume. And if you read the questions below, I’m sure you’ll agree that these all fit solidly into the FizzBuzz category!

Questions

Here are the five questions. (Note: these are the first five questions from our most popular “Generalist” quiz. If you take one of our specialized quizzes for ML, Data Science, DevOps, Front-End, iOS, or Android, you’ll see different questions, but the same concept applies.)

Question 1 of 5: What kind of SQL statement retrieves data from a table?

This question basically asks, “Have you ever seen a SQL query before?” 80% answered correctly.

Question 2 of 5: Fill in the missing line of code

Basic imperative logic fill-in-the-blank. 76% answered correctly.

If you’re concerned that we’re asking in Python syntax, I’d argue that this is just a convenient pseudocode language for this example, and any programmer should be able to select the correct answer even if they’ve never touched Python before.

Question 3 of 5: Why is caching used to increase read performance?

Caching comes up in so many contexts that it’s basically a universal CS concept, and an incredibly practical one in day-to-day software engineering. 89% answered correctly. Wow! That’s the highest of the five.

Question 4 of 5: Which of the following is used to maintain a user's logged-in state as they browse multiple pages on a website?

“Do you know how websites work?” 80% answered correctly.

Question 5 of 5: What is the value of z after the following code runs?

“Can you read code?” While presented here in JavaScript syntax, the dictionary/hash/associative array/map concept will be familiar to anyone who ever touched a language more powerful than BASIC. (Nothing against BASIC, my own first programming language!)

Only 62% answered correctly. This is, by far, the lowest of the five questions. Personally, I’m very surprised that this one is harder than Question #2, but that’s the data, and it’s statistically significant (p<0.001). (If you have any ideas on why we’re losing so many people on this one in particular, let me know -- email below.)

Data

Out of over 100,000 software engineers who took this quiz last year, here’s the distribution of actual quiz scores:

Distribution of actual quiz scores

Surprisingly, only 42% of people got 5/5 correct! The top 67% correctly answered at least 4 of 5. (And that 67% accounted for 98% of success on our platform.)

Performance on the five questions is not independent. Multiplying together the individual question probabilities, we’d expect only 27% to get 5/5 correct, but the real quiz data shows 42% achieving a perfect score. This is 55% higher than expected if each question was an independent random variable. In fact, a simple model treating all questions as independent implies a peak at 4/5, as well as more weight in the 3/5 bin than the real data:

chart_model.png

The correlation between questions present in the real quiz data suggests that we’re measuring an underlying factor like, “Are you a software engineer?”

Grade Yourself

If you’d have correctly answered at least 4 out of 5 of the questions above without any help, you passed “FizzBuzz 2.0” and should probably take the Triplebyte quiz. (If 3/5 or lower, I’ll be honest: it probably won’t help you much.)

After taking the quiz, you’ll see your scores and have the option to create a profile that lets tech companies contact you when they have opportunities that match your preferences, like “company size 500+,” “remote only,” “inclusive workplace,” “min salary $X,” or even “pet friendly offices.” Whatever you like. We work with hundreds of companies, from public FAANG giants and private unicorns down through stealth-mode startups, and it’s easy to block your current employer from seeing your Triplebyte profile.

Questions? If you have any questions about the questions above, or any questions about how Triplebyte works, email me at mike.robbins@triplebyte.com.

Get offers from top tech companies

Take our coding quiz

Liked what you read? Here are some of our other popular posts…

How to Interview Engineers

By Ammon Bartram on Jun 26, 2017

We do a lot of interviewing at Triplebyte. Indeed, over the last 2 years, I've interviewed just over 900 engineers. Whether this was a good use of my time can be debated! (I sometimes wake up in a cold sweat and doubt it.) But regardless, our goal is to improve how engineers are hired. To that end, we run background-blind interviews, looking at coding skills, not credentials or resumes. After an engineer passes our process, they go straight to the final interview at companies we work with (including Apple, Facebook, Dropbox and Stripe). We interview engineers without knowing their backgrounds, and then get to see how they do across multiple top tech companies. This gives us, I think, some of the best available data on interviewing.

Read More

Bootcamps vs. College

By Ammon Bartram on May 19, 2016

Programming bootcamps seem to make an impossible claim. Instead of spending four years in university, they say, you can learn how to be a software engineer in a three month program. On the face of it, this sounds more like an ad for Trump University than a plausible educational model.

But this is not what we’ve found at Triplebyte. We do interviews with engineers, and match them with startups where they’ll be a good fit. Companies vary widely in what skills they look for, and by mapping these differences, we’re able to help engineers pass more interviews and find jobs they would not have found on their own. Over the last year, we’ve worked with about 100 bootcamp grads, and many have gone on to get jobs at great companies. We do our interviews blind, without knowing a candidate's background, and we regularly get through an interview and give a candidate very positive scores, only to be surprised at the end when we learn that the candidate has only been programming for 6 months.

Read More

How to pass a programming interview

By Ammon Bartram on Mar 8, 2016

Being a good programmer has a surprisingly small role in passing programming interviews. To be a productive programmer, you need to be able to solve large, sprawling problems over weeks and months. Each question in an interview, in contrast, lasts less than one hour. To do well in an interview, then, you need to be able to solve small problems quickly, under duress, while explaining your thoughts clearly. This is a different skill. On top of this, interviewers are often poorly trained and inattentive (they would rather be programming), and ask questions far removed from actual work. They bring bias, pattern matching, and a lack of standardization.

Read More