QA Interviewing: The Phone Screen
-
By: Chris Greacen - 02/05/2008
I have a bunch of notes from when I was putting together the hiring process at Kodak Gallery. I dusted these off recently to help a friend screen candidates for a QA position he's looking to fill.I think back to our problem at kodak: we needed to hire super-geniuses to work on a completely custom-built system. This was before hibernate, before spring/pico, I think it was before the JSP 2.x spec even. We needed to hire people who could 'hit the ground running' and contribute without a hige ramp-up. We needed to perform a thorough search without booking the whole team for all-day interview sessions.
Part of our solution came with a formal phone screen. Once we started applying a standard approach to all candidates we were able to do (something approaching) an apples to apples comparison between the top candidates. We were also able to weed out some bozos who had 10+ years of experience but never encountered an application's error log.
General QA Questions- What's the difference between Blackbox and Whitebox testing? Can you give me examples of how you have done both?
- Can you describe regression, performance, stress testing? Describe when you have conducted each type of testing?
- When you report a defect, what information to you include?
- Why choose to include this info? What's the benefit?
- A new build is available to QA, what's the first thing you test?
- Have you used automated build & deploy systems?
- What is an equivalence class?
Through these questions you can get an idea about how seasoned a candidate is: have they encountered a metrics program? Have they worked on a performance testing project? Do they understand the challenges with managing software deployments in the testing cycle. The 'what's in a good bug' question can open up discussion in a number of areas: is the candidate aware of the connection between a feature, the server source, the browser's role... You can learn a lot from this one.
Teamwork Questions:Sure, this is another section I'll usually skip unless there's lots of time.- Can you give me examples of your experiences working on a team?
- Can you give me examples of your experiences working individually?
Technical Questions:
UNIX- How do you find the timestamp of a file in Unix?
- How do I read a log file as messages are being printed to it?
- How do I determine the location of a file called test123.txt in a Unix file system?
- What's the difference between CLASSPATH and PATH?
- How could I transfer a file from my windows PC to a unix machine?
- What is html or xhtml validation? What is the advantage of having pages that validate?
- Name 4 http methods.
- What's the difference between GET and POST?
- If you don't know how to do one of the above tasks, how would you find out?
- Have you ever used a proxy? Why would this be useful as a testing tool?
The goal here is twofold: first, to see whether the candidate can handle themselves in a 'go get it done' environment. Inevitably, there will come a time (daily in most of the places I've worked) when there's a bug and someone has to do the 'deep dive' to find out details about the problem (one bright QA guy coined the term 'qa forensics') by ssh-ing to the server, finding and watching all available logs while reproducing the problem. Can they search google to get themselves unstuck?
Second, if they mention LINUX or UNIX on their resume, they better damn well know that ls -l is how to get a timestamp of a file. It's worth having a few easy questions about topics that a QA engineer should encounter on a daily (hourly) basis.
The HTTP method question is something you can use to make sure the candidate fully answered the earlier section about performance testing.
SQL Questions There is a person table with columns, id, first_name, last_name, Birthdate- Give me the SQL statement to find all of the people named "Chris".
- How do I find all of the people whose name is "Chris, Christian, Christina, etc"?
- If you don't know how to do one of the above tasks, how would you find out?
There is another table Address with columns id, street, city, state, zip, person_id
- How do I find all of the people whose name is Jane and live in San Francisco?
- How many are there?
- Walk me through JDBC
- You have a Java application running against a RDBMS, what are some common optimizations done to enhance the performance of the application at the database level, java level, or interaction in between?
- If you don't know the answer to one of these questions right now, how would you find out?
Yes these are pretty lightweight questions. Someone who aces this section should be pretty comfortable getting test results from a database. If you're lucky, you can also find someone willing to own the application of db changes to your test environment. The really good candidates will be able to give you a bunch of ways to complete the second tasks: joins, sub queries, etc.
Leadership Questions:
- Have you lead a testing team? Offshore?
- How do you motivate a team?
- Will vs. skill?
- What communication is important? How do you make this happen?
- What do you do when the team comes from a letdown?
I would generally only ask these questions when dealing with a team lead candidate or someone going on the manager track.
It's worth noting that I'll usually tell a candidate the answer to a question if they get it wrong or can't answer. It's the interviewer's call about how to deal with this. It doesn't hurt to always sell your company, and a company that supports its team is a company where people want to work.
The interviewer has the option of scoring the candidate as a whole (1-move now, 2-interview after the other 1s, or 3-pass...) or by section. I'll let you come up with your own scoring system.
By the end of this line of questioning, the interviewer should have a pretty decent picture of your candidate, the accuracy of their resume, and most importantly whether it's worth calling them into the office for a face-to-face interview with your team -- which is usually a big investment!
I'll put together notes on the resume-sorting and interview processes for future posts.
