Metrics for product managers

Back when I was in product management, someone observed to me that product managers have all of the responsibility but none of the authority.

At least in the organizations with which I am familiar, no one reports to the product managers. They need to obtain resources from a variety of different organizations to get a product out.

So how do you measure the effectiveness of a product manager?

Michael Shrivathsan has looked at this, and has found several commonly used metrics wanting. For example:

Product Line Revenue

* PM teams do not control the revenue. Even if they do their jobs extremely well, revenue depends on execution by Marketing & Sales teams - over which PM teams usually have little control, and minimal impact.

Similarly, Shrivathsan notes that other measures such as profit and loss (P&L) look at things outside of the control of product management.

I do not like any of these metrics because they measure the performance of a team (PM team) using metrics over which they have little control, and further only a vague idea of how to influence those metrics.

So what is Shrivathsan's magic bullet? Product satisfaction. And because he's a good product manager, he uses an acronym to describe it - PSS for product satisfaction score.

What is it?

* PSS focuses on customer satisfaction with just the product.

* It excludes other factors such as customer support experience, friendliness of sales teams, etc.

However, I see one big problem with this metric - I'm not sure how an evaluator can focus on just PSS, ignoring all other aspects of the entire experience.

Take my product (well, now it's Bob's product) as an example. By the time that Cool County gets its automated fingerprint identification system, they have not only received a product that has been developed to a set of standards partially set by the product manager, but they have also received a CONFIGURED product which has been tweaked to meet the specific needs of Cool County. The product development was primarily done in various programming languages, while the configuration was done in XML. When most customers look at a particular screen, they are unable to ascertain which parts of the experience were configured, and which were actually coded by the development team. (In fact, if you show me a product with which I am not familiar, even I wouldn't be able to tell one from the other.)

So even if the customer is able to tune the customer support person and the salesperson out of his/her mind, how is the customer going to differentiate the product from its configuration?

Now someone may claim that this is unique to the complex product lines that talented people like me manage. (You can stop laughing now.) But I'm not sure about that.

As I write this post, I have a bottle of Crystal Geyser Sparking Mineral Water sitting on a table next to me. Now I have no idea how Crystal Geyser's product management organization works, and whether a single product manager is responsible for the water and the bottle and the label, or whether Crystal Geyser has a dozen different product managers, including one for the bottle caps.

But assume for the moment that the "product" is the water itself. What if I unknowingly bought a bottle of Crystal Geyser that had been sitting on a store shelf for two years? (I checked the entire bottle, including the aforementioned bottle cap, and could not find any date stamp.) In that case I might be very dissatisfied with the product, but the product manager couldn't be blamed for that.

I don't think.