User feedback is critical to making your application better. I absolutely crave feedback from users. However, it is probably also one of the trickiest things to accomplish. Feedback helps me guide new features and heavily informs upcoming versions.
My application, Worklog Assistant, just helps with JIRA time tracking. In the big picture, it's really quite boring to most people. So if it does the job, most people are content not to say anything. I thought that I should get more people to offer feedback, for better or worse. Maybe there are some obvious things I am missing (hint: there are!)
So, some time in January, I tweeted about an upcoming (very unscientific) experiment. At the time, I did not mention details to anyone. As you can guess, it had something to do with gathering more feedback.
At the top of Worklog Assistant's main screen, there are a few icons as seen below:
The little blue speech bubble has a tooltip which simply said "Send Feedback". Using Thunderbird's filtering capability, let's see how much feedback I got through this button over a seven month period:
That is a grand total of 8. E-i-g-h-t. Wow. According to Google Analytics, that is from a total of about 200 unique visits, for roughly 4% over all of 2009. I repeat: ALL OF 2009! Initially, I thought maybe no one cared (sniffle.) But then I sat back and thought about it from a user's perspective. Do I really want to "send feedback" for any apps that I use? The answer: ABSOLUTELY NOT!
Generally, I want to have a conversation with the authors of the application. And this has been my exact experience when talking to users. So I thought about it for a little bit. Eventually, I settled on changing the blue speech bubble to the following:
Why "Shout!"? I'd like to say this was based on some intuition but basically, I wanted something that people would click on which had some vague relation to its intended purpose and was enough to interest people. I released this change around the end of January and didn't really bother to check into it until a certain someone reminded me. Of course, I tweeted about the "experiment's" success!
Here are the cold hard statistics:
- 170 unique visits to the target page in ONE MONTH due to this simple change. Compare this to 200 for the whole of 2009 and part of 2010.
- Thunderbird says 12 new questions/suggestions as a result, a rate of 7%
The bottom line here seems to be that not only did I get more click-through, but the target page encouraged more people to initiate participation!
But there is another side effect. Some people that had never participated before now started to comment and vote on things they were interested in. So I think I've comfortably got a lower bound for new participation.
Conclusion
There is none. This is not a proper statistical test. But I think there is something there. What do you think?
Nitpicks and clarifications:
- But this is not scientific! Yes, I know.
- But you don't know whether it was the presence of the text or the value of the text which caused the greater traffic! Actually, yes I do (within the boundaries of being unscientific to begin with.) The first screenshot shown above was preceded by exactly the same icon but with "Send Feedback". Miserable, miserable click through.
- But you could have had a 1000-fold increase in customers in that month! Oh, how I wish that were true.
- But it could be because February is after January! Yes, you're 100% right. It could be.
- But you said all of 2009 and 7 months! You liar! Well, I took a shortcut: The 7 months was the entire time the button was active so technically, I'm not lying.
Well I hope that was interesting for you. It certainly was interesting for me. Big thank you to "dermike" for reminding me to check into my experiment. You made my day.
Feel free to nitpick some more.
