Vanity Metrics

David, Paul, Amanda, Tomas, Linda, Jenny

Measuring impact through Social Media Metrics

How can we measure engagement and impact?

Using social media as a measure of reach isn't always useful.

Choosing metrics for success? What would be ideal meatrics to measure success vs using the metrics that are available.

Understanding the role of sample size and basic statistics in the value of the metrics

Mapping data metrics onto qualitative metrics. Data culture at Avaaz. Principles around using statistics at avaaz

• Stats is how we listen?
  • Anyone can send an email to 10,000, then measure impact before sending out to more people.
  • Measurements include impact, growth and virality (weighting varies by campaign)
• Stats is a mindset - Grow it
• Stats is a good slave but a terrible master
• Stats are like a crime, investigate them
• Stats have a comfort zone, stay within them
• Stats lie without context

In data visualization:

• There is an issue with the point of view of the client, where every individual sees something different from the same visualization
• When doing this fast, and on a large scale, it becomes harder to interpret

Using field workers to generate metrics

Metrics as a tool for comparing experiments

Organizing work at The Chinese Progressive Association:

• How do we shift value in the community
• The more they talk about positive stories the more people talk about the issues.
• Example: an increase in connections from teenage chinese can be difficult to explain or understand
• Tracking shifts in individuals (i.e. do values shift over time/years) - anecdotal tracking
• Qualitative measurements of community leaders

The Dynamics of lasting change. On one side is the meaningful change done by fieldwork effecting individuals, on the other side avaaz works with large scale technologies to impact change. Avaaz aggresively checks back with supporters (wisdom of the crowd). Testing everything, as a democratic tool.

If you trust the constituents and can ask them, you have a way to measure effectiveness. But if your online supporters are not your constituents, that may be a problem. There may be value in reaching the on-line crowd to get them to impact the actual constituents.

Useful metrics in Social Change and advocacy:

• Measuring virality - Does a specific message resonate beyond your base
  • Number of generations a piece of content has (sharing) vs the time it has existed.
  • Can using testing tools end up simplifying or exagerating your message?
  • Can we measure if people blindly retweet, or do they paraphrase the message? (avaaz says it cna be measured, but has no impact)
  • Clustering as a driver of virality?
• Often we cannot have a success metric, except the judgment of participants or field workers.

Engagement metrics in a network of blogs that cover education policy. As of now there isn't a clear measure of success, but they have access to ethnographers. Education policy may be a good testbed because a particular blog may be the only source of information on school policy in a particular community. Therefore isolating success may be easier. There is an ability to measure the resulting actions (i.e. are people requesting changes in school policy). Combine that with tracking of how datapoints (i.e. spending figures) get used in conversation.

• A conversation or a meme-replication metric. Especially if it includes new data-points (to isolate the cause)
• Measuring influencers. Are these people using our data.