Last week I went to the MRS Connected World conference, a really excellent event gathering together an inspiring crowd to talk about new technologies and consumer behaviours. Not just to listen – though listening was great! I was also putting forward the FACE point of view on a panel with Tom Ewing of Brainjuicer and Paul Edwards of Working Plural & JKR.
Our topic: “cutting through the noise”. Digital media & technology has generated a dramatic shift – for the first time in history, there’s not a shortage of information but an excess. But how to make sense of it all? How to find the insight amid the flood?
Our session was kindly written up by Research Live, so I won’t go into the details here. Instead, I want to pick up on a really smart question from an audience member – How do you do social media research with real rigour?
Great question. How do you move beyond a set of observations made on a vast and potentially rather amorphous dataset, to get to something we might actually call research? On the spot I came up with 3 ways - but on reflection, there are more.
Here’s my top 10 ways to make your social media research rock solid:
1. Capture the complete universe
If the dataset’s incomplete (and especially if you don’t know what’s missing), you can’t say anything about how your findings relate to the wider universe. Tweets found directly through Twitter search are really no more than anecdote until you can contextualise them within a meaningful totality of everything that’s going on in social.
Image source: Mapping The Global Twitter Heartbeat: The Geography of Twitter, by Kalev H. Leetaru et al., 2012
So make sure you’re using a social media research tool that’s built on top of Twitter Firehose (the 100% data API) and robust blog, forum & news data collection.
Of course there’s still a gap between “everything said in social” and “everything people think”. But that’s true for every research method – this is a risk we can only minimise, never remove entirely.
2. Your search strategy is critical
Great data sources aren’t enough on their own – you’ve got to set them up right. If you’re searching for a particular category (e.g. haircare), you need to be confident you’ve collected the whole category – every possible way people can talk about hair, from products to styles and stylists, and verbs & adjectives as well as nouns. Just searching for all mentions of “hair” won’t cut it – you’re not capturing a meaningful totality.
How to build good search syntax: Brainstorm. Then test it in Twitter & Google search, then iterate to add in new words & phrases that come up. Analyst experience is key here to build a search strategy that’s both comprehensive and focused.
3. Qualify your quant insights
Social data is qual data at a mass scale, says Francesco D’Orazio, our chief technologist.
Numbers on their own aren’t insights. Positive sentiment is 20% – so what? What are people saying? What are the needs and emotions driving that figure, and why is it higher for one brand than another? Read, synthesise, code. Quote the actual messages, show the verbatim. Keep the people visible in how you tell your insights.
4. Quantify your qual insights
Say you’re doing an innovation project, find out that fighting frizz is the most important consumer haircare need. Your immediate client might love the depth of qual insight you can build from beauty blogs and forums… But she’s also got to communicate that insight around a larger organisation & to lots of people who won’t ever read your full deck.
So quantify that qual insight and rank it against other needs. Savvy use of Boolean search strings – NEAR operators & smart exclusion terms – can give you sensible approximate volumes for almost any concept. You’ll not capture every nuance, to be sure – but it’ll help support that qual insight as a really solid finding.
(Ok, not really an example of quantifying qual insights – but a very cute example of Boolean syntax!)
5. Can another analyst find the same insights?
Classic research methods such as data coding still can have a key role to play in turning social media data into insight. It provides a structured template for content analysis that helps iron out bias from the analyst’s own preconceptions. Instead you’ve got a random sample of 200 messages and a structured grid, and it’s easy to review across team to help standardise what you mean by particular categories and concepts.
Is this finding real? How much does it actually matter? Display your research findings contextualised against other brands, other categories, or as share of voice – so your reader can get a sense of proportion.
7. State what you don’t know, or can’t prove
- e.g. “This visualisation is based on Twitter data, a channel used by 26% of the UK population.”
- “Social media messages almost never identify a store by its exact street address, and only 1.6% of tweets have geolocation. Consequently we cannot locate the se complaints to specific store, only town or region level.”
- “Social media data includes only information that is publicly available on the web, and not private email or text message data” (yes we get this one!)
Make the gaps explicit. It shows you know what you’re talking about – and helps ensure your insights are interpreted accurately. Overclaim isn’t rigorous!
8. Test hypotheses. Test a null hypothesis.
Having hypotheses makes your data useful – instead of just drawing a picture of the landscape, you’re trying to find out something specific. But in the spirit of scientific enquiry, proving a hypothesis isn’t just going out looking for data that supports it. It’s also about looking for data that supports the null hypothesis – the counter-possibility that nothing is happening, or the opposite. Look for both – and if all the evidence really falls on one side, then you can be confident that your finding is really robust.
Testing the null hypothesis or counter-factuals is also a great way to find interesting things you weren’t expecting (see point 10!)
9. Triangulate against other data sources
Extract everything you can from your client, from sales figures to qual research to semiotics decks. Turn these into hypotheses. Is your research supporting these? Building on them? Taking them a new direction? Or disagreeing entirely? All are legitimate outcomes – and putting your insights in this context makes them much easier for your client to use.
10. Don’t do social media research if it’s not the right way to answer your question
A contrarian point for closing – but here at FACE we’re honest about the fact that social media data can’t answer all research questions. Its genius is that the data we’re analysing is largely spontaneous and unprompted, making it a great way to find “unknown unknowns’ – the things you didn’t even know you wanted to know, or needed to ask.
But sometimes you’ve got really specific questions to answer – how far are consumers prepared to trade off price vs. quality, perhaps, or whether a different shade of blue would make a better bottle top. And I’m afraid people just aren’t talking about bottle cap colours in social media… So you’d need to ask them directly: time for a focus group! Not social.
So that’s 10 ways to make your social media research really robust. Any more to add? Get in touch with us on Twitter – we’re @FaceResearch – and tell us your top tips! I (Jess) do a bunch of tweeting for FACE, so let’s keep the conversation going.
Or if you’ve got a really thorny research problem and you’re looking for a rigorous solution, get in touch with my colleague James on James.Hirst@Facegroup.com – we’d love to talk it through with you.