Wednesday, August 28, 2013

Bad Statistics Are Dangerous For Charities

In the last week a few people on my Twitter timeline have posted (some sceptically) the results of Google's research in to how and why people donate on-line.

The conclusions from the statistics included that 75% of people start their donation research on-line, 74% of people donate because they believe in the mission, and 57% of people make a donation after watching a video.

I'm going to go out on a limb here and say that this is all complete bullshit.

Firstly, common sense - think about those numbers. Of course it's not true that more than half of people donate after watching a video. Of course it's not true that three quarters of people donate because they believe in the mission - do you even know what your favourite charity's mission is?

Secondly, let's look at the survey and trace it back.

The article I just read was this one, that exclaims that "57% of Those Who Watch Nonprofit Videos Go On to Make a Donation". Please note that the article is written by a YouTube Video Producer. I'm sure he's very good at that. But his statistic is just not true. That's not his fault...let's look where he gets it from.

He gets it from here. It's a nice article that most subsequent articles are referring to. It casually mentions that "An impressive 57% of people who watch a video for a nonprofit go on to make a donation."

But it doesn't link to the original research, which is a shame.

The original research is quite hard to find. Here it is. It is incredibly vague and unhelpful. It's also notable in the fact that the research was actually carried out by MillwardBrown Digital...in association with Google.

Look a bit further and you'll find the methodology, which is almost always overlooked when we read statistics. This particular methodology really tells us very little.

But it does tell us they surveyed 982 people. I hate surveys - they are generally used by students and people trying to sell you something. The truth is when you ask people about their donation habits it is completely unreliable.

They also 'tracked donors behaviour backwards'. It is possible that they looked at donors, worked backwards and found that 57% of people had come from videos. But that certainly isn't the same thing as 57% of people make a donation after watching a video. I would assume that more than 99% of people watch a video and then do nothing. This is either really badly worded...or they're just wrong.

And the carelessness in this analysing and presentation of statistics is dangerous. I find statistics in charitable fundraising particularly dangerous.

Small, medium and even large charities are going to read these statistics and going to read articles written by very intelligent people. They're going to assume that it's correct - 57% of people who watch a video will go on to donate. And then they're going to put their fundraising budget in to making these videos. They'll put them on-line and they will do nothing. They will not generate income. They will fail. And that money will be gone.

These statistics are dangerous because it draws charities in to the belief that this is what will work for them.

It won't.

Please don't put your fundraising budget in to YouTube videos unless it is part of a bigger and more researched fundraising mix.

4 comments:

  1. This is so weak it's insulting that they think we will accept it without question.

    The Google summary is incedibly vague. It appears there was a good bit of data collected but what is being included in the summary is parsed to drive an agenda.

    - “Process begins with ‘search’. 75% of all contribution-related research begins online.”

    This gives absolutely no indication of what triggered the search. The “Process” begins well before the donor googles a particular charity or issue. Donor's searchs are triggered by a number of channels, TV, Radio, Direct Mail, friends, News stories.

    I'm willing to bet that Google has the exact data on how many of these searches were driven by digital content or digital marketing. I suspect the number is so low that it will not be made public.

    I’m sure there is some useful information in this research but we’re only being shown what suits Google’s agenda. The conflagration of quantitative (actual online tracking) and qualitative (survey of researchers) is a red flag. When quantitative and qualitative research are mixed together with no clear delineation you know you are being spoon fed B.S.

    There are actually so many holes in this research it could keep us blogging for days.

    Weak, very weak

    Denisa
    @DenisaCasement

    ReplyDelete
    Replies
    1. Thanks Denisa. So true! What are people searching for? More often than not they're searching for the charity because they've seen it off-line.

      Delete
  2. We were looking all over for the original Google numbers after sharing the stat - thanks so much for tracking them down! It all makes sense now.

    ReplyDelete
    Replies
    1. Thanks for the comment Claire - I'm glad the blog was of use!

      Delete