Build an Audience

To conduct a successful survey you need people to take it. Not just anyone will do though. You need feedback from potential users (or current ones if you have those). In order to target your specific user group, you have to build an audience.

Know Your User

Know your user demographics, and start there. What are your users interests? Where do they hang-out online? What communities are they a part of? Just because Facebook has 400 million users, that doesn’t mean you’re going to have much success finding privacy nuts there. Start somewhere you know users will be and grow from there.

How I did it: The first place I went to find a high concentration of functional programmers was Google Groups—specific mailing lists focused on functional programming. Was I right to start there? You bet! The greatest source of traffic to my survey was from the three mailing lists I posted to.

Build an Audience with Social Media

Social media is only as effective as the size and quality of your audience. A message from a Twitter account with zero followers will get zero attention. You have to build an audience before people will listen to you. But how do you do that? Well, how does this work offline? You don’t walk into a crowded room, whisper into the air, and expect people to listen to your every word. They won’t even know you’re there. Instead you seek out people you have a connection with. These people my be friends, acquaintances, the guy you sat next to during a conference, or that someone who’s eyes met with yours for just a moment longer than usual. These are the people who will listen when you speak, and who likely have an interest in what you have to share.

Another important aspect of this is establishing common interests. Even if you manage to capture some one’s attention long enough to start talking, they wont stick around long if you don’t talk about something they’re interested in. So demonstrate that you’re a person or company of interest to the user. Choose a high-quality, relevant avatar. Set a descriptive bio. And by all means, post some interesting information on your account that shows you are not a spambot.

My experience: Soon after posting to mailing lists I sent out a message from the @functionaljobs account on Twitter and Identica which each had 1 follower, me. I re-tweeted the message from my personal account, but had almost no response despite the fact I had crafted the message to ensure maximum retweetability. Why? None of my followers where functional programmers! So I set out to find users with an interest in functional programming and follow them. The results where immediate. Seconds after I started this process I had email coming in to notify me of new followers. After an hour or three of hand-picking people who I was sure had an interest in functional programming, and who would very likely have an interest in my product, I was following 354 people on Twitter. Within a few days I had 117 people follow back. The results on Identica were similar, but on a smaller scale. The numbers aren’t astronomical, but the process was effective—I got survey responses from Twitter/Identica users, and some expressed genuine interest in my product.

Some of the tools I used to discover users with a particular interest in functional programming were: WeFollow, Topsy, ‘following lists’ for users I already discovered, Twibes, Identica groups. Update: Tweetils seems to work really well for this.

Be Cognizant of Bias

Everyone has bias of some sort, and that’s okay. A problem can arise, though, if too many of your respondents have the same bias because this will skew the survey results. This might happen because a bias naturally exists among your entire user group (which is fine and good to know), or because you are only contacting a subset of your users.

What I found: Initially, all the responses I got were from users on the mailing lists I posted to. One of my survey questions was, “How would you prefer to stay abreast of new job openings?”. Guess what? Nearly all responses mentioned “email”. When I finally got traffic from Twitter/Identica nearly all those responses to the same question included “Twitter” and “Identica”. My survey was also posted on the Haskell sub-Reddit, so when users from that source were answering the question “What functional programming languages do you use?”, most said “Haskell”. Don’t conclude humans don’t eat meat if you have only surveyed vegetarians.

A Recipe for Success

1. Ten questions or less is good; five questions or less is ideal. Nobody is going to spend 30 minutes filling out a questionnaire.
2. Choose questions wisely. Make sure you can use all of the information you gather.
3. Continue the conversation. The final question on your survey should invite the user to (optionally) give their email address or phone number so you can follow up with them regarding their responses. Of the responses I got, 56% included contact info.
4. Continue building your audience. After a user completes your survey, direct them to your mailing list, Twitter account, blog, etc. If they had enough interest in you or your product to complete a survey, they probably would like to hear what else you have to say.

Have advice of your own? I’d love to hear what others have learned about conducting customer development surveys.