Direct Use of Production Data for Performance Metrics

Data access frequently challenges metric developers. Consequently, they may resort to using the most readily available performance data; data that can be obtained through a user defined production application query and downloaded into a Microsoft Excel spreadsheet or Access database. While such practices may be appropriate when developing proof-of-concept metrics, direct use of production data typically leads to metric instability resulting in low metric confidence and driving unintended organizational behaviors.

Why Production Data is Unsuitable for Organizational Performance Measures

Production data possesses qualities making it unsuitable for use in developing longer-term organizational performance measures. Primary among these is the dynamic nature of the data itself. By its very nature, production data is transactional and subject to frequent ongoing change – additions, deletions, and revisions. Consequently, metrics monitoring a series of period-based performance (such as monthly performance periods reflected in one metric covering a rolling 12 month year) can have a given historical period’s performance change from one publication of the metric to the next.

Another significant issue associated with using production data is extraction timing consistency. To consistently measure performance within defined time intervals necessitates data to be extracted at the same end time for each period such as 11:59:59 pm (23:59:59) on the last day of the week, month, quarter, year, etcetera. Not only is it highly unlikely that a manually timed extraction could occur with such timing precision but few individuals work at midnight and fewer at midnight on weekends and holidays where a period will occasionally end.

Benefits of Using a Data Historian

Data historian applications solve the issues associated with using production data as an input to organizational performance measures. Such applications extract a ‘data image’ at a consistent, specified period end time (eliminating extraction timing issue). These moment-in-time data images preserve performance as it was known at the time of extraction. Subsequently, metrics can be derived from a collection of data images – one for each reflected period – ensuring indicated period performance remains constant from one report publication to the next (eliminating the data variability issue).

Final Thought…

As previously eluded to, use of production data benefits the development of proof-of-concept metrics. Production data’s ease of extraction enables developers to more rapidly and cost effectively create and test these pilot metrics. That said, use of production data based organizational performance measures should be limited and done with a full understanding of their potential variability defects so as to not diminish confidence and drive inappropriate behaviors before the metric is properly implemented.

About the Author

Nathan Ives is a StrategyDriven Principal and Host of the StrategyDriven Podcast. For over twenty years, he has served as trusted advisor to executives and managers at dozens of Fortune 500 and smaller companies in the areas of management effectiveness, organizational development, and process improvement. To read Nathan’s complete biography, click here.

Enterprise Performance Measurement

We can work with you to assess and improve your performance measurement system; yielding metrics and reports that are operationally relevant, organizationally consistent, and economically implemented. The resulting system helps improve managerial decision-making, organizational alignment, and individual accountability. Learn more about how we can support your implementation and upgrade efforts or contact us for a personal consultation.