Volvo Math: the automotive CSI system is broken

 CSI. No, not the TV show. We're talking about the Customer Satisfaction Index here. Every manufacturer, to my knowledge, uses a version of CSI, often with a proprietary moniker. From those I know in the business at the moment, it appears that purchasers of new cars are surveyed, as well as service customers. The purpose of the survey, it has seemed, is to gauge a customer's satisfaction with their servicing or selling dealer, and then for the manufacturers to have leverage over the dealers based on their CSI performance.

A bit of history to start, at least as I have viewed it in my time the the auto business. I have read that JD Power started their automotive quality surveys in 1981. I became aware of manufacturer surveys in the mid-to-late 1980s. It was a different world. There was a lot of boorish behavior in the dealer body. Many manufacturers were producing a lot of junk. Dealers were trying to cope with the junk. Many customers were treated poorly, and sometimes they were completely ripped off. It was a dark time in the auto business in a lot of respects, at least from a customer's perspective. The climate was right for a JD Power to step in and get the survey game going. Good for them.

When the manufacturers started to survey their new car customers and warranty repair customers, immediately, the scumbags in the car business who were doomed to score poorly figured out how to work the system: they bribed their customers. If you agreed to turn over your survey (remember we had but one kind of mail in the 1980s), the dealer would give you a free oil change, or something along those lines. In fact, I am certain that there were manufacturer reps who endorsed or even encouraged this behavior. Why? Because like the dealers, they had skin in the game and benefited from their dealer body having better scores. Why did the dealers care? It wasn't pride or ego. Dealers with better CSI scores got a better allocation of new vehicles

At some point, manufacturers caught on and stopped this practice, sort of. No longer could you physically separate a customer from their survey and fill it out yourself. Dealers were creative, though. Some purchased post office boxes and used these addresses instead of the customers' so that they could still get their hands on the surveys. That was eventually discovered, too. Instead, and likely many of you have heard this, dealership employees were instructed to beg their customers to give them great scores. Again, reps from the manufacturers endorsed this. 

Things have changed a little in the 21st century. Most surveys go through email now. Some manufacturers have banned dealership personnel from asking for good scores. Somehow, I can imagine that there is still a way to cheat that system. Isn't it clear that there is a problem here? What is the point of creating a survey system which is completely corrupted in its implementation? What could be more disingenuous? Plenty, but I'll get to that shortly.

In my previous life with Saab, I knew that we were the best at Charles River Saab. We weren't perfect, but we sure tried hard  and were earnest in our desire to provide our customers with the best service possible. Saab's CSI was based on a 100 point scale. It was an honest 100 point scale. If one customer gave you 100, and another an 80, your score was 90. Makes sense, no? We were always good but never best at CSI with monthly averages in the 93-94 range. We did nothing to enhance our score except to do our jobs as well possible. We never coached or begged for CSI. I knew that the system was really irrelevant because a number of the dealers who scored better than we did were only marginally competent and had to be working the system, something I refused to do. We did a good job, and our customers knew it and the real gauge of CSI was that they kept coming back. To Saab's credit, I thought they ran as good a program as possible. As I mentioned, their score calculation was logical. In addition, they allowed us to delete two surveys every month. Before that, it was so frustrating to get a bad score on a survey which might have condemned service work which a customer got at another dealer, or a customer who was happy but inverted the order of the scores and gave us 1's instead of 10's. Then, they did the best thing ever--they cancelled the program. They determined that the cost of the program did little to enhance anything or bring to their attention any problems they weren't already aware of. It is sad that rational thinking and common sense are so rare that they they come off as exceptional insight.

This brings me to present day Volvo and their survey. As is standard practice, Volvo surveys all new car buyers, and all service customers. Fine. Here's what is not. It is common vernacular around a dealership to hear employees plead for perfect scores because "anything less is a failing grade." Well, in the case of Volvo, this is not hyperbole. Most questions on the survey have a range of answers from 1 (bad) to 5 (outstanding). When computing the score, a 5 counts as 100%. To my way of thinking, that would mean a 4 would be worth 80% and a 3 would be worth 60%. If you were to concur, you would be wrong. 5 counts as 100%. 4 counts as 0, as do 3, 2 and 1. When it comes to the ratings of my advisors, then, truly if any score is given other than a 5, the advisor scores 0%. I discovered this early on when looking at my CSI and saw that we had 4 surveys submitted early in a month, and had a score of 25%. Horrible! When I looked at the surveys, there was one 5 and three 4's. It was true, anything under a perfect score was a zero! In the world of real math we would have had an 85%, but not with Volvo. When I met with representatives from Volvo, they suggested that we needed to do a better job of coaching our customers to give us 5's, especially since those who scored us with 4's usually included glowing remarks. I get this. If I was ignorant of what these surveys were like, I'd never give a perfect score since I can always imagine something better. Shame on us, I suppose, for not playing the game--the oldest game in the world of CSI--as well as others do.

Other than making a bunch of noise about this, I suppose we are left with nothing other than working the system, as disingenuous and distasteful as it is.
Categories: Volvo
Tags: volvo csi
; ;