Door-to-door Complaints DID NOT Double In The Last Year

The FRSB's 2012 complaint report came out yesterday with the highlighted area of concern that doorstep fundraising complaints increased by 93% in the previous year. Various media took that to mean that doorstep complaints doubled. Danielle Atkinson already wrote a great post here on why that was a terrible headline. At the very least it's misleading...at most it's just wrong.

But I wanted to add to the discussion by looking further at the door stats and seeing how close to doubling they actually were. Warning: this is stats heavy.

So the number of complaints according to the FRSB report are:
2010: 2,360
2011: 2,877
2012: 5,555

But the volume also increased, and that's an important measure of complaints - you can't judge any change in number of complaints without knowing how many more people are being exposed to the medium. I believe the FRSB report is meant to be a measure of quality, rather than a measure of complaints, so we need volume.

Number of approaches on the doorstep were:
2010: 31,266,863
2011: N/A
2012: 43,617,913

Unfortunately, there is no figure for the 2011 approaches and this is part of the problem.

We don't know how much volume increased by from 2011 to 2012, so it's impossible to tell if the 93% increase in complaints is in-line with the increase in approaches. So instead it's tempting to compare the 2-year increase from 2010 to 2012...so let's do that.

In 2010:
31,266,863 approaches generated 2,360 complaints. Meaning every 13,249 approaches generated a complaint.

In 2012:
43,617,913 approaches generated 5,555 complaints. Meaning every 7,852 approaches generated a complaint.

So over two years the drop from 13,249 to 7,852 actually represents a 59% increase in the rates of complaints. [1/(13249/7852)].

But that's over two years. If the increase was linear then you'd actually be talking about a 26% increase year on year. So door-to-door complaints increased by 26% from 2011 to 2012...not doubled.

Compare that to unaddressed mail: Complaints went up 59% while volume DECREASED.
Compare that to telephone: Complaints went up by 64% while volume only went up by 15%.

Now those are 'massive hikes'. Where's that headline?

I'm not saying there's no problem with door. Of course there is - a 26% increase is a huge problem. Of course it can be improved. Of course charities are turning to poorer quality agencies because they're desperate for volume. Of course more 'marketing agencies' who don't know what they're doing are testing the water because they see such a huge demand. Of course the demand to deliver a lower cost donor has meant lower wages and more pressure on fundraisers. All of this contributes - that's not what this post is about.

This post is about statistics. And door-to-door complaints didn't double.