"Easiest" and "best" are subjective, but when I was a PM for a ~25M monthly-unique-visitors website, I used classifying survey verbatims to influence our product roadmap, to drive consensus on what we needed to change, and to measure the impact of change over time.
Of course, reading and classifying survey verbatims didn't replace focus groups, 1:1 user interviews, UserVoice and similar feedback tools, etc. But I found that survey verbatims were invaluable tools to expose myself and other colleagues to the "voice of the customer" at massive scale. When one customer complains about something or makes a suggestion, it's hard to know if that user is an outlier. When you read the thoughts of hundreds or thousands of users, you very quickly figure out who's the outlier and who's not. It can be a humbling experience to hear the unvarnished opinions of thousands of people, but as a PM I found it hugely useful to make our product better and to make me a better champion for our customers.
Yes, there are tools that claim to automatically be able to classify verbatims, to measure customer sentiment, etc. If all you need is a pie chart to show your executives, those tools are fine. But here's my suggestion: don't use these tools. Instead, read and classify verbatims yourself. As a PM, your job is to get to know your customer more deeply. In my experience, automated classifiers prevent the kind of immersive get-to-know-the-customer experience that reading verbatims provides.
Here's how I did the classification and made it efficient and consistent over time:
- First, read verbatims. Thousands of them. By downloading the data into excel and scanning down a column of verbatims, I could read many hundreds per hour.
- After reading the first few hundred, patterns started to be obvious. The same ideas, suggestions, sentiments, etc. were repeated by different users in different words. Start defining names for these themes or categories. These are the draft categories for your classification. Write these down. For example, if you read many verbatims like "your site's search sucks" and "can't find anything" then you may want a "Search and Discoverability" category.
- Read more verbatims, trying to assign each verbatim into one of the newly-defined categories. Tune your categories as you read more verbatims. For example, if you are getting 10 verbatims saying "your search sucks" for every 1 verbatim saying "your menu is confusing" then you need a "search" category more than a generic "search and navigation" which doesn't accurately portray what your customers are saying.
- Once you have a solid set of classification categories, write down the implicit rules you've been using to sort verbatims into categories. Typically these rules are simple mappings of keywords into categories, e.g. "any verbatim with 'video' should go into the 'tutorials' category". Sometimes you'll need to be generic beyond specific keywords, e.g. "users mentioning the name of a particular celebrity should go into the 'celebrity news' category." Often users will use different terms than your team uses internally, so you'll have to translate your users' terminology into something your team will understand.
- Test written-down classification rules by getting someone else to classify 50 or 100 verbatims. They will fail the first time, and they will give you feedback that will help you tune the classification rules. Repeat this process until you can get someone else to classify at least 80 or 90 out of 100 like you would.
- Run your rules through a new set of verbatims and (this is really important) repeat over time. This will help you understand if changes you're making to your product are producing change in user sentiment.
- Finally, change your survey using the insights gained above! For example, after classifying users' verbatim answers to "why did you come to our website today?" we found that users came for 6 core tasks: "learning how to use our product", "diagnosing an error message", "finding a specific download", etc. Then we added those categories into the survey so the user was picking a category (as a multiple choice question) and giving us a verbatim explanation with more details. We also asked users "Were you successful?" and correlated the type of task with success rates. This led us to focus on improving site search, recognizing that our tutorials were woefully inadequate, etc.
I realize the above seems like a lot of work, but relative to the time required to make the product better, it was a tiny investment of my time: probably a week to build initial categorization and 1-2 days per quarter to refresh the data with newly classified verbatims.
In exchange, I could sit in front of any VP in the company and have a well-reasoned argument, supported by my customers' own words, about the changes our team wanted to make. It was worth it!