I was really interested to see how you find the best developer.
This is probably the best article I have ever read on that subject.
I wondered how I fit into your criteria. See my article: Managers: How do You Manage a Cat?
I posed this same question in my Introduction to Computers class at work in 1981 (managers, secretaries, engineers and lab technicians). Who do you hire? There was no correct answer but a very lively discussion.
I never had a project from 1969 to 2016 fail when I was in charge. My first project out of school increased the company’s share of the market by 20%. It did something that was offered as a college class that I did not take, on a computer that I had never seen, using a programming language I had never heard of. I had to read a math book on the day between the first and second interview just to understand the project. Then I said, “Sure, I can do that.” It solved 40 equations with 90 unknowns.
I got my last job at age 65 because the first 12 people failed the test. I also said that the job “sounded like fun” during the interview. The test was simple, though, and the passing score was low. It mainly demonstrated that the candidate could function in the environment. I finished in less than half the time and pointed out that each problem had at least three solutions.
Your coding challenge checklist is spot on.
Quiz questions: You had 3 terms that I did not know. In a similar example, the trainee around 1985 asked if I “normalized my databases?” I was not familiar with the “normalized” term. She was shocked and explained. I said, “Oh, sure. I did that with my first relational database in 1969.”
I had one experience that you did not cover. The young lady was hired as an expert at her past company just to increase their black and female percentages. Her project was cancelled two months before she was hired. She started looking for another job immediately.
I said, “I didn’t recommend you for those reasons. You are smart, can learn on your own and I don’t have to send you to a bunch of classes.” Seemed gruff to me, but that was probably one of the nicest compliments she ever had.
Describe myself in three words? At the top of my resume: “I solve problems.”