How To Think About Rankings

Ed BurghardEach year with the help of Xavier University I publish two annual rankings.  The first is the American Dream States Report, and the second is the American Dream Cities Report.  Both are based on resident sentiment regarding the degree to which they are achieving their American Dream.  The data I use is from a robust and validated quantitative research study into the American Dream conducted by Xavier University.  To the best of my ability, I try and keep my conclusions within the limits of the data.  My P&G training has taught me to be fairly rigorous in market research analysis.  As a consequence, I tend to over disclose.

With that as background, it should come as no surprise that when my friend Malcolm Allan shared a recently published article entitled The Top 6 Reasons to Be Wary of City Rankings, Ranked, I read it with the intent of ensuring my reports were not violating any good practice guidelines.

What I found was excellent counsel on how to read market research.  The key theme was before drawing any conclusions about a ranking understand what the research actually measures.  This means understand the demographics of the research respondents since conclusions cannot be projected on people who are not represented by the group.  It also means you need to understand what is actually being measured, particularly if you are trying to make comparisons with other study outcomes.  Third, the article suggests it is important to understand the study sponsor’s motivation so you can be on the look out for interpretive bias.

Hopefully you will find that my ranking reports adhere to the guidance provided in the article.  Perhaps the only missing piece is disclosing my motivation.  In short, I publish the reports because 1) I am trying to change the dialogue in economic development from job creation to better enabling residents to achieve their American Dream, and 2) I believe a quantitative understanding of the American Dream will result in better decisions around local asset creation, infrastructure investment, public policies and support of public programs that provide a hand-up.  In full disclosure, I provide the national reports for FREE and offer an opportunity for people to buy specific comparison reports.  The fee for the comparison simply covers the cost of pulling the data and preparing a customized report.  In my mind, the $50 cost represents a breakeven price point.

Let Me Know What You Think

 

What do you think of this post?
  • Awesome (1)
  • Interesting (0)
  • Useful (1)
  • Boring (0)
  • Sucks (0)

1 Comment  |   Forward this to a friend Forward this to a friend   |   Number of emails sent: 542

Category Measures

Bookmark and Share

You can follow any responses to this entry through the RSS 2.0 feed.

1 Comment so far

  1. Gary Sullivan

    June 11, 2015

    Your P&G years have taught you well Ed. Yes, there may be a tendency to over analyse/disclose but you are also expert in the the P&G writing models which clearly train us to ( a ) articulate the background to ensure our reader has the right context, and ( b ) then move on to learnings/insights before arriving at next steps.

One Response to “How To Think About Rankings”




XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

By submitting a comment here you grant Strengthening Brand America a perpetual license to reproduce your words and name/web site in attribution. Inappropriate comments will be removed at admin's discretion.

SBA Blog