I've been asked to assist in the technical side of some interviews (apparently we're hiring). I'm not going to discuss any of the specifics of the candidates or positions, but this has got me thinking about evaluating technical people in interviews.
 
There seem to be a number of schools of thought as to what constitutes the kind of questions you should be asking a potential technical hire. Some I've seen are:
  • Ask abstract logic puzzles. Microsoft used to be well known for this, I have no idea if this is still the case.
  • Post very difficult maths questions on public billboards ala Google.
  • Find some random list of questions regarding obscure bits of whatever your current platform of choice is.
  • Pick a random technical problem x and have the candidate solve it on a blackboard.
I don't like any of these options.
 
Most of the abstract logic puzzles I've seen don't have any correlation to the kinds of skills I'd want in a developer. So being able to come up with a convincing reason as to why manhole covers are round (except of course for those that aren't) doesn't impress me in the context of recommending you to be hired. For a start your reason is almost certainly wrong or missing key elements that people who have experience in this area know of. This is precisely the kind of information that should come from the domain experts who know more about their business than you ever will. Eric Evan discusses these kinds of things in Domain Driven Design (which I highly recommend). So my correct answer here is "find the experts who know this stuff already", not engage in pointless speculation a client almost certainly doesn't want to pay for.
 
Posting difficult questions as Google did certainly has a cool factor. I can't see how to turn "come up with a domain model to support this line of business application" into a nifty billboard though. So Google's approach, which worked very well for them and got the worldwide media coverage, isn't really applicable. There are relatively few companies that can pull a stunt of this type off, you're probably not working for one of them.
 
The list of technical questions as I've generally seen it manages to test all the wrong things. Frankly I don't care if you know what namespace ReallyObscureClass belongs to and which constructor to invoke when you want it to be on double secret probation. We have documentation and Google for that. It's kind of like claiming you need to be able to memorise the atomic weights of all the elements to be a chemist. That information is useful, but there's a lot more to chemistry than that (I assume, not being a chemist).
 
Asking for a problem to be solved on the whiteboard fails for me on many levels. It's my opinion that good coders don't work linearly, but rather accumulate behaviour in cycles. This isn't particularly easy to do on a static whiteboard. It's also remarkably hard to use TDD with code you can't run. Most importantly any code you could write on a whiteboard is too trivial to prove much beyond a basic grasp of the language in question. You may of course not know anything about the platform in question, but knowing basic syntax doesn't tell me what I need to know.
 
So what does work (apart from hiring someone and watching them crash and burn)? And what should you actually be after anyway?
 
What I want to see in a technical candidate is more than just some basic competence with syntax. I'm looking for someone with the ability to solve problems, both technical and business. This includes:
  • An understanding of the platforms involved, their strengths and more importantly their weaknesses and limitations. Far too many people seem to think that their experiences are universal.
  • An understanding of their own strengths and weaknesses. I don't mean the "I work too hard sometimes" answers people give to make themselves look good. I mean things like "I'm terrible at designing user interfaces" (use of the word terrible is probably not advised). No one can be great at everything and if someone claim they are I'm likely to think they're unskilled and unaware of it (pdf).
  • Their opinions on development practice. Disciples of the One True WayTM, whatever it may be, should be a warning here. Preferences which can be backed up by solid reasoning are quite another. In many respects I care more that you think about these things than what your particular conclusions are. Unless you advocate a waterfall model, in which case you shall be summarily dismissed (I'm prejudiced and not ashamed of it).
  • Evidence of thinking outside the mainstream. In the .NET world this would mean something other than just the vanilla Visual Studio install and the default framework and toolset. Examples of things I'd like to
    This is far from a complete list and I'm not really interested in someone being able to name everything, or indeed anything, that's on the specific list above. What I do want to see is someone who's aware that there's a lot more available than what Microsoft ships and an awareness that proper selection of these frameworks and tools is a legitimate and beneficial strategy.
  • A genuine interest in technology and in providing usable solutions to real problems. There's a combination here of interest in things because they're shiny and new, and pragmatism required to know when to choose to use something.
  • An ability to see where they can improve what they've done in the past.
You're not going to get the above off a checklist of questions. This kind of thing is, in the context of an interview, going to be most apparent through discussion. I actually enjoy long rambling conversations on random elements of technology, but something a little more focused is required here. The conversation should cover the points above, with reference to the experience of the candidate and the position they're destined for. The difficulty is in not leading the candidate to the "correct" answer (which only tells you what you want to hear) or discussing your own views (which isn't really the direction information should be following in this context).
 
I've also been considering the concept of a take home exam, a small problem which can be implemented quickly but is sufficiently involved so as to be non-trivial. The point of such an exercise would be to validate a candidates ability to produce a working system, making it more valuable than a whiteboard exercise. I'm currently unsure how this would fit in into the process but I do like the idea (for a suitably bounded list of candidates only).
 
Notice: I don't speak for my employer. I don't make the hiring decisions. These are my thoughts on the single element I'm involved with, trained professionals do the real work.