Re: Define an engagement goal and success for a Knowledge Base #metrics #engagement


Stephen Bounds
 

Hi Vandana,

Thanks for that detail, that's really helpful. I know it would be lovely to give you a standard benchmark, but you really will do better if we can find a metric that drives the kinds of behaviours you want.

For any system or process, there are six basic things you can try to optimse:

  • fixed costs (ie baseline costs of staff, buildings, software platforms, etc)
  • incremental costs (ie cost per additional product)
  • cycle time (ie how long one item takes to produce)
  • throughput (ie how many products are being produced)
  • quality (ie accuracy, integrity, value)
  • excellence (ie delivering over and above expectations)

I'm guessing that cost isn't the primary driver, so let's focus on the other aspects. Will your Knowledge Base be successful merely if there are knowledge assets published? What if those assets were really poor quality? Or they weren't of interest to anyone?

Here are some ways to think about metrics within the constraints of the available data. To support optimisation for:

  • cycle time, you might seek a high number of downloads on the presumption that users who download data are doing so in order to support timely completion of their work
  • throughput, you might seek a high number of uploads as confirmation that the data scientists are productive in creating assets for the organisation to reuse
  • quality, you might examine the ratio of views to downloads on the basic assumption that data which is perceived as useful will get downloaded more often
  • excellence, you might simply monitor view number trends as a proxy for demonstrating that the KB is providing ongoing value to its audience

I'll be honest though; none of these are great metrics. They are pretty easily game-able so you'll need to trust that your teams genuinely want to succeed on their merits rather than by seeking to artificially inflate numbers. Of these, I'm most inclined to:

  • Monitor throughput trends (uploads) and aim for a steady-state target.
    What's reasonable will be context specific; how big is the team and do these knowledge assets take days or weeks to produce? Are they responding to specific client requests or just producing whatever research they feel is interesting?).

  • Seek a stable or upwards trend in view traffic every quarter as a proxy for excellence.
    This is pretty straightforward - if the system isn't delivering value to users overall, they'll stop using it.

You might also want to periodically examine quality of the various uploads to determine if there are products being created which aren't adding value and if so, engaging in a conversation about if they should be stopped or adapted to be more relevant. This would be more of a diagnostic than a metric though.

Cheers,
Stephen.

====================================
Stephen Bounds
Executive, Information Management
Cordelta
E: stephen.bounds@...
M: 0401 829 096
====================================
On 28/04/2021 9:54 pm, Vandana Wadhawan via groups.io wrote:

Hi Stephen, 
The aforementioned team is research heavy. They engage in data modelling activities, and performs drug-specific researches and are qualified data scientists. The KB is intended as a storefront to store all their knowledge assets. 

While I'm able to find out their monthly engagement (it's been only a month), I'm also supposed to propose an engagement goal to the team (a realistic percentage) that they could aspire to achieve in 2-4-6 months. 

Is there a standard benchmark that anyone has come across? i could workaround that benchmark and come up with something. 
--
VW

Join main@SIKM.groups.io to automatically receive all group messages.