My Issues with the Broadbent Institute's Inequality Report

Ugh.

Apparently I have it in for think tanks or something. Every few months a think tank somewhere comes out with a report that means well, portrays a message with fundamentals I agree with, but manages to mess up some amount of the data handling in a way that gets me riled up.

This time it's the Broadbent Institute. They released a report on income inequality recently, and presented the data in a virtually identical format to an American video from two years ago. While I agree that income inequality is a big issue in Canada, and I'm sure that the average Canadian isn't clear with just how bad it is, I have a pretty big issue with the statistical rigor in their report.

This is a screenshot from their video:

Along the x-axis, they have different population percentiles in 10% chunks. The chunk on the far right represents the richest 10% of the population, the one to its left is the next 10% richest people, etc.

The problem with this chart is that it shows the 50-60th percentiles as being richer than the 60-70th and 70-80th. They're trying to tell us that the 5th richest group is richer than the fourth and third richest groups. 

What!? That doesn't make any sense by definition.

These values appear to come from this table in their report:

Somehow, Canadians apparently consistently think that the middle 20% of the population is supposed to have more money than the 2nd wealthiest 20%. That's not possible, and I can't believe that it got all the way into the report and the video without anyone hitting the emergency stop button. The income curve shown in the first figure ought to be a version of a Lorenz curve, and necessarily should increase from left to right. Even if that is the actual result from the survey, it shows that either the survey wasn't clear enough in its instructions, or that adequate controls weren't in place in the survey to ensure accurate results. 

When I brought this up to them on twitter, their response was:

.@Mikerobe007 We reviewed w/ pollster re aver % for that category. Reflects strong desire of those in middle brackets 4 strong middle class
— Broadbent Institute (@broadbent) December 17, 2014

Which is... silly. There's a logical distinction between a "strong middle class" and a "middle class that's stronger than the upper middle class." They've clearly decided to ignore this distinction.

Finally, a (more than average) nitpicky point. Take one more look at this graph (where the blue line was the original "ideal" values);

The blue line of ideal values has five data points in it, which is great, but they're nowhere near where they should be! Each point (or kink in the blue line) corresponds with a 20% chunk of data, so they might have shown the mid-point 10%, 30%, 50%, 70%, and 90% marks. But instead they've shown the 0%, 25%, 50%, 75%, and 100% marks. This implies that the 0th percentile Canadian (the absolute poorest person in Canada) ought to have the wealth value that the people surveyed thought belonged to the bottom 20% as a whole.

Anyway, the point of all this is that research into wealth inequality is really important, and doesn't deserve to be handled quite this badly. If you're going to be sharing this report, please do so with a grain of salt.