Figures often beguile me, particularly when I have the arranging of them myself; in which case the remark attributed to Disraeli would often apply with justice and force: “There are three kinds of lies: lies, damned lies and statistics.”
Quote attributed to: Mark Twain’s Own Autobiography – The Chapters from the North American Review
A lot of times, my private sector clients are confused about the purpose of different analytical methods and as a consequence have trouble interpreting quantitative research results. Two of the methods that are often mind boggling are 1) regression analysis and 2) factor analysis. I thought it might be helpful to provide a quick refresher for each so you could be clearer on which approach might best serve your needs.
The following contains technical language and the subject matter may cause a headache. But, it is important stuff to understand so you don’t get bamboozled by an Agency.
This methodology is typically used to determine which attributes are the most influential on overall perceptions (usually reflected as a composite score). Understanding the specific drivers of perception is very helpful in crafting your communication strategy.
Recession analysis attempts to predict what is commonly referred to as the “dependent variable”. An example would be a consumer’s purchase intent. The basic assumption is that the dependent variable is impacted by a number of independent measures (or attributes). Understanding the influence the independent variable exerts on the dependent variable helps you prioritize the importance of various attributes. The degree of influence any specific attribute exerts is called the correlation coefficient. It can have a value in the range of 0 to +/- 1. A 0 correlation coefficient suggests the attribute exerts no influence on the dependent variable. A value of +1 suggests the attribute and dependent variable are positively correlated. In our example, when the attribute score goes up so does the consumer’s purchase intent. They move in the same direction. In this case you’d want to consider tactics to strengthen the attribute with an expectation it will result in a greater purchase intent. Conversely, a -1 coefficient suggests a negative correlation. In this case, you’d want to weaken the attribute in order to increase consumer purchase intent.
This methodology is used to reduce a number of variables in a data set so you can better manage them. Think of it as helping you identify what is really really important. It is also helpful in understanding how consumers think about a set of attributes. Again, like regression analysis, this analytical approach can help you craft a stronger communication strategy.
Factor analysis looks at all correlation coefficients between attributes, not just in relation to the “dependent variable”. The methodology generates “factors” that represent groups of attributes. The factor loading variable represents the strength with which a given attribute has to the factor overall. The factor loading value can range between -1 and +1. For example, if the factor loading value is 0.8 then the variable is strongly and positively associated with that factor.
This method is a huge help when Management weighs in and provides their thinking on what the key drivers are. Typically, by the time you list all the feedback you’ll have 10+ attributes different managers insist are important. Using factor analysis, you can whittle that list from 10+ to the top 3-4.
The most important thing for either statistical methodology is to understand the purchase behavior. Both rely on a robust identification of attributes though to be choice drivers. If you don’t get this right then it is a GIGO (garbage in – garbage out) exercise.
Let me give you an example I remember from my college days. I was taking a statistics course and the professor was talking about cause and effect relationships. His trick question was – If you find a strong positive correlation in your statistical analysis can you concede the attribute is a key driver of the dependent variable?
He shared the story of a school that wanted to better understand what drove academic performance. The school ran a regression analysis throwing into the mix any variable they had student data on with a blind hope of finding a positive correlation. Their analysis found that shoe size was a great predictor of academic performance of standardized tests (in this case, an IQ test). It was a strong, positive and robust result. Unfortunately, the children studied ranged in age from grade schoolers to college students. Upon further analysis using common sense they concluded that as you got older your foot got larger, and the difference in IQ score was actually a function of the respondent’s age (collected wisdom).
Aid To Judgment, Not A Substitute
Statistical analysis can definitely help you make a better decision. But, the results are simply an aid to your judgment. You still need to use common sense and leverage your understanding of what is really going on. I used to get livid when any of my direct reports would say “based on the statistics …”. I wanted to know what choice to make based on their understanding of the consumer. Of course, I was always happy to hear that was supported (or confirmed) by the statistics. Please don’t take this advice as semantics. If you make the wrong choice because you followed the results of a statistical analysis rather than use you own judgment, you are accountable. Don’t make a habit of hiding behind statistics or using statistics as a way to avoid personal accountability for a decision.
I wish I can tell you statistics is fun. It isn’t. The rules and terminology can cause anybody to have a headache. But, a fundamental understanding of statistical methodologies is crucial to effective decision making. My favorite question was always “What does that really mean?” or sometimes I’d ask “How should I think about those results?” Usually the answer to either of the two questions would be helpful in interpreting the statistics and making an informed decision.
If you read the entire post, kudos! If you actually understood it, I am impressed. If you are like the other 99% of us and your brain now hurts, it is cocktail time 🙂