Hi all,

I'm working on a project where a KB has been offered as a solution to a team. While i see a few engagements and usage, its too early to see the trend. However, the Leader of that team has approached me to create an engagement goal that his team could aspire towards. Any insights on how to plan and define an initial engagement goal? 

Please advise and share insights on what could be the right approach!

The metrics is limited to users (i.e. views), uploads and downloads of knowledge articles.

--
Regards,
VW

Hi Vandana,

If you can monitor uploads by user, think about tracking the number of users who have contributed any articles as a better metric for commitment to use than raw upload numbers. You probably don't want one user to be supplying 50 uploads per week and no-one else adding anything into the system!

However, I suggest that the main issue with the metrics you have described as available is that they all measure proximate goals (ie goals in support of a business outcome) rather than an ultimate business goal (such as reduced average case time, fewer errors, higher customer satisfaction, etc).

You need to find a plausible link between one of your proximate metrics and a valued business outcome to make your metrics meaningful. This link could be done empirically, or anecdotally, quantitatively or qualitatively.

For example, you could survey the team and ask something simple like:

1. The KB has helped me solve cases faster: Never / Occasionally / Sometimes / Frequently
2. The KB has helped me make fewer errors: Never / Occasionally / Sometimes / Frequently
3. My customers have been happier because of a solution I found in the KB: Never / Occasionally / Sometimes / Frequently

If you track your usage stats and repeat this survey every few months you should see increasing usage tracking with increased business benefits (and if not, well ... that's a whole other conversation).

Once you've demonstrated the link between usage and benefits to the satisfaction of your manager, you can decrease the frequency of user surveys or cease them altogether since it will have been established as a proxy for the thing you want to really measure, ie business benefit.

Cheers,
Stephen.

====================================
Stephen Bounds
Executive, Information Management
Cordelta
E: stephen.bounds@...
M: 0401 829 096
====================================

Hi Stephen,

You are spot on in understanding the challenge and limitations of the KB. Thank you for sharing approaches that i can consider.
The idea behind this exercise is to aspire towards a goal (Engagement specific), and then re-look at this change (KB) and see what else can be done to ensure the change (which is adapting this KB) is internalized in day-to-day behaviors of the team members. 

Is there a standard bench-mark i should measure engagement against? I mean the percentage of adoption that is defined and generally observed by KM practitioners during such projects.
--
Vandana W

Hi Vandana,

What kind of work does the team do (broadly described if you can't be specific)? Customer service work will have quite different engagement frequencies and metrics from an engineering team, for example.

Cheers,
Stephen.

====================================
Stephen Bounds
Executive, Information Management
Cordelta
E: stephen.bounds@...
M: 0401 829 096
====================================

Hi Stephen, 
The aforementioned team is research heavy. They engage in data modelling activities, and performs drug-specific researches and are qualified data scientists. The KB is intended as a storefront for all of their knowledge assets. 

While I'm able to find out their monthly engagement (it's been only a month), I'm also supposed to propose an engagement goal to the team (a realistic percentage) that they could aspire to achieve in 2-4-6 months. 

Is there a standard benchmark that anyone has come across? i could workaround that benchmark and come up with something. 
--
VW

Hi Vandana,

Thanks for that detail, that's really helpful. I know it would be lovely to give you a standard benchmark, but you really will do better if we can find a metric that drives the kinds of behaviours you want.

For any system or process, there are six basic things you can try to optimse:

• fixed costs (ie baseline costs of staff, buildings, software platforms, etc)
  
• incremental costs (ie cost per additional product)
  
• cycle time (ie how long one item takes to produce)
• throughput (ie how many products are being produced)
  
• quality (ie accuracy, integrity, value)
  
• excellence (ie delivering over and above expectations)
  

I'm guessing that cost isn't the primary driver, so let's focus on the other aspects. Will your Knowledge Base be successful merely if there are knowledge assets published? What if those assets were really poor quality? Or they weren't of interest to anyone?

Here are some ways to think about metrics within the constraints of the available data. To support optimisation for:

• cycle time, you might seek a high number of downloads on the presumption that users who download data are doing so in order to support timely completion of their work
  
• throughput, you might seek a high number of uploads as confirmation that the data scientists are productive in creating assets for the organisation to reuse
• quality, you might examine the ratio of views to downloads on the basic assumption that data which is perceived as useful will get downloaded more often
• excellence, you might simply monitor view number trends as a proxy for demonstrating that the KB is providing ongoing value to its audience
  

I'll be honest though; none of these are great metrics. They are pretty easily game-able so you'll need to trust that your teams genuinely want to succeed on their merits rather than by seeking to artificially inflate numbers. Of these, I'm most inclined to:

• Monitor throughput trends (uploads) and aim for a steady-state target.
  What's reasonable will be context specific; how big is the team and do these knowledge assets take days or weeks to produce? Are they responding to specific client requests or just producing whatever research they feel is interesting?).
  
  
• Seek a stable or upwards trend in view traffic every quarter as a proxy for excellence.
  This is pretty straightforward - if the system isn't delivering value to users overall, they'll stop using it.

You might also want to periodically examine quality of the various uploads to determine if there are products being created which aren't adding value and if so, engaging in a conversation about if they should be stopped or adapted to be more relevant. This would be more of a diagnostic than a metric though.

Cheers,
Stephen.

====================================
Stephen Bounds
Executive, Information Management
Cordelta
E: stephen.bounds@...
M: 0401 829 096
====================================