Go, Do, Be.

Categories

04/09/2008  — 

bashWebTest Lives!

Well, I did it. I tossed a little pile of code onto google's open source site.

A few years ago I wrote (in my spare time) a little test harness around some simple command-line utilities. I wanted something to help me answer some simple questions about what was going on some large clusters of servers. Rather than clicking through a bunch of nicely formatted pages, I wanted something to make a bunch of http requests and give me a 'yes' or 'no' about the response. The trick (for me) was to try to run it on some server in the cluster which was running a really lean installation of Linux. No frills.

I could have probably compiled a jar & dropped it onto the server... but I couldn't edit & recompile a class on the server. I could probably have run a perl script, (geeze why didn't I just write it in perl?) but I think the WWW-Mechanize module wasn't installed. Who knows... anyway, I ended up stumbling upon curl and decided to write a wrapper around it using simple bash scripts.

Guess what? It worked. It was handy. Guess what? I used it at a few jobs since I wrote it. Guess what? It's still (somewhat) handy. So today (or yesterday) I give something back to the internets and interwebs that haz given me so much. I offer:

Tests are pretty simple. I'll toss a few test examples on this blizzog and onto the wiki on code.goog over the next few weeks. If you have any interest at all in using something klunky, and somewhat functional, please contact me and I'll help you get started.

Code.goog doesn't have a way to select it, but I planned on distributing the source under the 'MSL' license.

Anyway, enjoy!

02/09/2008  — 

Sifting through some stuffs on kodak

Yeah, so I admit to doing exactly what I mentioned earlier: I went through a bunch of old albums in my kodakgallery account. I realized that I got a booster applied to my gallery premium account at the last DIVCamp, take a peek:

http://www.kodakgallery.com/greacen

Also, here's a pile of scanned historical photos I've collected along the way:

02/08/2008  — 

KodakGallery FINALLY integrates with the web

Kodak and Slide revealed their integration the other day. Now Kodak users can fill Slide widgets with photos from their albums. Yahoo!'s Flickr has always had (and continues to have) more web 2-dot-0-y choices for integrating photos, feeds, and streams of photos.

Personally I'm psyched to see this finally happen, and not just because I have a ZILLON ZIGABYTES of image data on the gallery (from the old ofoto times). Kodak has secretly had an XMI interface available on the net for years. Why didn't they want to exploit this 3 years ago? Their bandwidth costs can't be anything next to their storage costs. Seems like the surest way to lead people toward a cart (and conversion-event (print purchase)) is to have as many eyeballs as possible on the pictures.

Hopefully the Slide integration and recent Firefox plugin signal a new interest in opening up petabytes of photo data. Hopefully that lures a few old customers back to the site to dust off some of their long-forgotten photos.

02/05/2008  — 

QA Interviewing: The Phone Screen

I have a bunch of notes from when I was putting together the hiring process at Kodak Gallery. I dusted these off recently to help a friend screen candidates for a QA position he's looking to fill.

I think back to our problem at kodak: we needed to hire super-geniuses to work on a completely custom-built system. This was before hibernate, before spring/pico, I think it was before the JSP 2.x spec even. We needed to hire people who could 'hit the ground running' and contribute without a hige ramp-up. We needed to perform a thorough search without booking the whole team for all-day interview sessions.

Part of our solution came with a formal phone screen. Once we started applying a standard approach to all candidates we were able to do (something approaching) an apples to apples comparison between the top candidates. We were also able to weed out some bozos who had 10+ years of experience but never encountered an application's error log.

General QA Questions
  • What's the difference between Blackbox and Whitebox testing? Can you give me examples of how you have done both?
  • Can you describe regression, performance, stress testing? Describe when you have conducted each type of testing?
  • When you report a defect, what information to you include?
  • Why choose to include this info? What's the benefit?
  • A new build is available to QA, what's the first thing you test?
  • Have you used automated build & deploy systems?
  • What is an equivalence class?

Through these questions you can get an idea about how seasoned a candidate is: have they encountered a metrics program? Have they worked on a performance testing project? Do they understand the challenges with managing software deployments in the testing cycle. The 'what's in a good bug' question can open up discussion in a number of areas: is the candidate aware of the connection between a feature, the server source, the browser's role... You can learn a lot from this one.

Teamwork Questions:
  • Can you give me examples of your experiences working on a team?
  • Can you give me examples of your experiences working individually?
Sure, this is another section I'll usually skip unless there's lots of time.

Technical Questions:
UNIX
  • How do you find the timestamp of a file in Unix?
  • How do I read a log file as messages are being printed to it?
  • How do I determine the location of a file called test123.txt in a Unix file system?
  • What's the difference between CLASSPATH and PATH?
  • How could I transfer a file from my windows PC to a unix machine?
WEB
  • What is html or xhtml validation? What is the advantage of having pages that validate?
  • Name 4 http methods.
  • What's the difference between GET and POST?
  • If you don't know how to do one of the above tasks, how would you find out?
  • Have you ever used a proxy? Why would this be useful as a testing tool?

The goal here is twofold: first, to see whether the candidate can handle themselves in a 'go get it done' environment. Inevitably, there will come a time (daily in most of the places I've worked) when there's a bug and someone has to do the 'deep dive' to find out details about the problem (one bright QA guy coined the term 'qa forensics') by ssh-ing to the server, finding and watching all available logs while reproducing the problem. Can they search google to get themselves unstuck?

Second, if they mention LINUX or UNIX on their resume, they better damn well know that ls -l is how to get a timestamp of a file. It's worth having a few easy questions about topics that a QA engineer should encounter on a daily (hourly) basis.

The HTTP method question is something you can use to make sure the candidate fully answered the earlier section about performance testing.

SQL Questions There is a person table with columns, id, first_name, last_name, Birthdate
  • Give me the SQL statement to find all of the people named "Chris".
  • How do I find all of the people whose name is "Chris, Christian, Christina, etc"?
  • If you don't know how to do one of the above tasks, how would you find out?

There is another table Address with columns id, street, city, state, zip, person_id

  • How do I find all of the people whose name is Jane and live in San Francisco?
  • How many are there?
  • Walk me through JDBC
  • You have a Java application running against a RDBMS, what are some common optimizations done to enhance the performance of the application at the database level, java level, or interaction in between?
  • If you don't know the answer to one of these questions right now, how would you find out?

Yes these are pretty lightweight questions. Someone who aces this section should be pretty comfortable getting test results from a database. If you're lucky, you can also find someone willing to own the application of db changes to your test environment. The really good candidates will be able to give you a bunch of ways to complete the second tasks: joins, sub queries, etc.

Leadership Questions:
  • Have you lead a testing team? Offshore?
  • How do you motivate a team?
  • Will vs. skill?
  • What communication is important? How do you make this happen?
  • What do you do when the team comes from a letdown?

I would generally only ask these questions when dealing with a team lead candidate or someone going on the manager track.

It's worth noting that I'll usually tell a candidate the answer to a question if they get it wrong or can't answer. It's the interviewer's call about how to deal with this. It doesn't hurt to always sell your company, and a company that supports its team is a company where people want to work.

The interviewer has the option of scoring the candidate as a whole (1-move now, 2-interview after the other 1s, or 3-pass...) or by section. I'll let you come up with your own scoring system.

By the end of this line of questioning, the interviewer should have a pretty decent picture of your candidate, the accuracy of their resume, and most importantly whether it's worth calling them into the office for a face-to-face interview with your team -- which is usually a big investment!

I'll put together notes on the resume-sorting and interview processes for future posts.