KM Component 21: KM Metrics and Reporting Process

Metrics and reporting involve capturing operational indicators and producing reports to communicate performance against goals, areas for improvement, and progress toward the desired state. There is a wide spectrum of opinion about the importance of measuring knowledge management activities. 

Some believe that it is essential and want to collect data and create reports on a long list of detailed metrics. Others believe that this is a waste of time, and that effort is better spent on enabling knowledge flow.

Details on the three different kinds of metrics that are typically captured and reported are provided in KM Conversations: Using Analytics Part 2: Metrics and Reporting.

Metrics and Reports to Avoid

Don’t capture metrics for the sake of metrics. Many people express a desire for data that doesn’t drive any action or insight—collecting data for data’s sake. For each metric to be captured and reported, there should be an associated action or insight which is expected to be driven. Avoid collecting every random thing, sliced and diced every possible way, which someone might want to know once, but has no intention to do anything with other than say, “Oh, that’s interesting.”

Don’t establish a long list of arcane metrics. The fewer the number of metrics, and the simpler the reports, the better. Instead of reporting uploads, downloads, site visits, and other similar numbers, report on how the organization is sharing, innovating, reusing, collaborating, and learning.

Don’t attempt to measure knowledge using the metrics of balance sheets. Conventional balance sheet metrics do not adequately measure knowledge.

Avoid chartjunk, infographics, and bad statistics. Nice-looking, but worthless charts, infographics, and stats are a waste of effort. See:

• Chartjunk – for more, read Edward Tufte
• I love a good infographic! – avoid this, and read instead Megan McArdle
• Lies, damned lies, and statistics – fore more,  read Joel Best

Be wary of publicizing numbers that reflect actions you don’t want to encourage. For example, if you don’t want lots of groups being created in your ESN, don’t promote these metrics:

• 30 New Groups Created
• 1,148 Total Public Groups
• 1,186 Total Private Groups
• 2,334 Total Groups

Avoid Measuring Time Saved

I don’t believe that it is worthwhile spending a lot of effort to capture time saved. Here are two articles on the subject:

• Time saved — a misleading justification for KM by Michael Koenig
• Trying to Determine KM Productivity? (Groan) by Dan Kirsch

If you wish to try to capture time saved at the document level, you will have to deal with these challenges:

• When a user downloads the document, they will not yet be able to tell how much time they saved. They will have to return later to input this data, and it is unlikely that they will remember to do so.
• Estimates of time saved are not very accurate.
• The question raised in the article by Michael Koenig is what did they do with the time saved? Was it used productively?

To encourage users to input such data, you can use a KM recognition system to award points for KM activities. Users can be motivated to claim points for reuse, and in claiming those points, they can be required to document the value of that reuse. You can capture the time saved for a particular document by asking for the URL of the document and the time saved as a result of reusing that document as part of the input form in the recognition system.

However, you may find that despite offering recognition and/or rewards, users may still be unwilling to enter this information. So it may be easier to give them a button they can easily click, similar to a Like button, which indicates that they reused the document productively, and not bother trying to capture the amount of time saved.

How to Manage Reporting

When defining how the KM program will be governed, include the process for reporting. Include the reporting schedule in the overall plan of record, maintained on an easily accessible website.

The KM core team should decide on the details for reporting. These should include which metrics to report, the targets for each metric, the format of reports, what level of detail and how granular reports should be, to whom reports will be distributed, where reports will be stored, how frequently reports will be produced, who will produce reports, how and when to revise metrics and targets, and how to produce custom reports.

Serve the needs of your reporting providers and consumers. Ask those who create metrics reports for suggested improvements in the collection and reporting processes. Ask those who use reports how they use them and what changes they would like to see.

Communicating About Metrics

Send monthly KM metrics reports to the senior leadership team. Ask them to publish their own variations for their organizations.

Report on how the organization is doing in a monthly report, and inspect and discuss progress (or the lack thereof) in management team meetings. Request that the organization’s balanced scorecard or equivalent performance indicator reporting be updated to include compliance to KM goals. Review KM indicators along with the usual business metrics, so it will be clear that they are just as important.

Incorporate the metrics into as many parts of the organization as possible. Use newsletters, websites, calls, and meetings.

On the intranet, include the latest key metrics, and links to more detailed reports. Compare actual results to targets to celebrate progress and remind users of goals still to be achieved.

Implement a data warehouse for self-service KM indicator reporting. Make sure that all potentially useful data is updated on a regular and timely basis.

The means of motivating employees include monitoring and reporting on progress against organizational goals. Track and communicate progress against the established KM goals for individuals. Report on KM incentive programs. Include standings, winners, and related statistics.

Include the results of employee satisfaction surveys in your regular program metrics reporting. Indicate what actions will be taken as a result of the feedback received.