Recently I was trying to decide on a UI change for Ingerchat: whether to replace the text buttons in the draw area (“Done”, “Cancel”, etc) with icons. I felt like this would be a good choice, as icons seem to be more fun than text, but I wanted to validate this hypothesis by showing it to some people before implementing it.

The problem is: where could I find people in my target segment (18-24 y/o college students) to validate this change? My solution was Mechanical Turk.

Mechanical Turk (aka MTurk) pays people do small tasks for small amounts of money. Say five cents to determine if a website is spam or not. Though I haven’t heard of UX/UI designers using it for research, many social science researchers use Mechanical Turk to perform inexpensive surveys. So doing a UX survey seemed like a reasonable (and inexpensive) task.

The first problem was that MTurk doesn’t provide fine-grained demographic control. Basically the only choice I had was country where the workers were located. But I wanted to restrict my survey to 18-24 year-olds.

The solution for this was a two-stage survey. A first stage where I ask demographic information, and then a second stage where I perform the actual survey.

You have a number of choices when creating a project in MTurk: “categorization”, “writing”, “survey”, etc. I chose “Survey”. In the survey, I asked a number of questions, such as age, gender, education, app usage, though I was only interested in age and whether or not the person was in college. I asked the other questions to ensure that it wasn’t obvious what I was looking for, to prevent people from gaming the survey. The instructions were generic, and indicated only that I was looking for information on mobile app usage.

But then it got challenging. The tools MTurk provides to put together a survey are, frankly, crude. MTurk has an API, and I get the impression that most people use the API to construct surveys. Without using the API, it’s not obvious at first glance even how to force survey takers to answer all the questions in the survey! After much investigation, I found that by editing the survey HTML and putting a “required” tag on the first choice in a radio field, the question became mandatory:

<label><input name=”Gender” required=”” type=”radio” value=”Male” />Male </label>

For this reason, any question in the survey that needed exactly one answer, I made into a radio button.

Once the survey was done, I put it out for 20 people for 15 cents each, and found it finished in just a few minutes! Of the 20 people who answered, 5 were in my target segment. (Edit: I then did a second stage for 12 cents each, and got another 5 in the target segment). So now it was time to move on to the second stage of the survey.

Continued in Part 2…