Topics

KM ROI #question #value


Sachin Joshi
 

Hello all, this is my first post on this very insightful forum/group.

I am KM consultant in IT organization. We implement and monitor lot of good KM practices in our teams. We also see good benefits of all these, unfortunately showcasing them as quantitative benefits is always a challenge (I am talking about benefits such as reduction in xxxx, improvement in xxxx and so on... not about quantitative KM dashboard data such as no. of SMEs, no. of posts, no. of assets, no. of sessions, etc.).

The question which is always asked on KM performance is, what is the benefit. Change in culture, improvement in collaborating, creation of assets, etc. is not an answer to this. since this is just an output but not measure of benefit/success. Can someone share, how we can correlate KM with business benefits OR how can we show return on investment in KM. Thank you.


 
Edited

Hello Sachin - this is a topic that comes up frequently in KM, and has been discussed several times on this forum (here is my take on a similar question a few months ago: https://sikm.groups.io/g/main/message/7463).  There are a variety of perspectives on it, as I’m sure you will see as responses start coming in to your question here. 

My take on it is based on something I read from management guru Peter Drucker. His basic thesis was that knowledge confers two principle benefits to a firm: increased revenue and increased productivity. I would add a third: increased quality (or effectiveness if you like). 

So what does this tell us as KM practitioners? 

It tells us that if we want to quantify the benefits of our KM efforts, we should start by looking at business challenges that KM can be applied effectively to that are related to one of the above two (or three categories). Where in the business are we seeing productivity-related challenges? Where do we see a lot of effort being expended, driving cost? Better leveraging expertise or lessons learned is one way KM can help.

Where are we feeling the pinch of declining revenues? Is organic growth a priority? KM for innovation is one possible solution.

These are just examples of how KM practitioners can think about the client situation they’re facing, and work backwards from a real business issue to a potential KM intervention that might effectively address it. This is the approach I used to good effect on numerous KM client engagements. It worked because I was able to start with a real business issue that I could quantify from the beginning, and then measure any improvement I could make via a KM solution. 

Good luck with your efforts. 
--
-Tom
--

Tom Short Consulting
TSC
+1 415 300 7457

All of my previous SIKM Posts


Tim Powell
 

Hi Joshi,

 

This is my first post here too.  Excellent and timely question, one that I’m sure will come up often now that (from what I hear) major cutbacks are underway in some IT/KM shops.  This happens during every recession – and bothered me so much that I wrote a new book The Value of Knowledge to address it:

 

https://www.youtube.com/watch?v=A791Ujybg5Y&t=5s

 

In that 10-minute video, there are listed some initial ideas, as well as several free resources that may help you.  If you already have the book, it’s all in Chapter 5.  Sorry for the commercial pitch but this is honestly the best answer I can give you.

 

I draw on Drucker and lots of others – but Nick Milton and Patrick Lambe’s Knowledge Manager’s Handbook was especially helpful to me, particularly Chapter 9.  Their simple but effective “benefits mapping” technique (which I cite in my own work) is a way to ensure what I call “strategic impact” for knowledge initiatives – i.e., making sure that you measure not only the OUTPUTS of your work, but also the results, OUTCOMES, and impact (as you mention.) 

 

My experience has been that, in general, knowledge professionals are far more comfortable and competent at doing their work than in explaining it to their CFO and CEO at budget cut time.  (Or, ideally, long before that.)  As an MBA by training, my personal mission is to help them do just that.

 

I also generally agree with Tom Short’s comments in this thread.

 

Hope this helps, and good luck,

 

Tim

 

TIM WOOD POWELL | President, The Knowledge Agency® | Author, The Value of Knowledge

DIRECT/MOBILE +1.212.243.1200 | ZOOM 212-243-1200

SITE www.KnowledgeAgency.com | BLOG www.KnowledgeValueChain.com

 

 

From: <main@SIKM.groups.io> on behalf of Sachin Joshi <sachinjoshi.a@...>
Reply-To: "main@SIKM.groups.io" <main@SIKM.groups.io>
Date: Thursday, October 15, 2020 at 10:09 AM
To: "main@SIKM.groups.io" <main@SIKM.groups.io>
Subject: [SIKM] KM ROI #question

 

Hello all, this is my first post on this very insightful forum/group.

I am KM consultant in IT organization. We implement and monitor lot of good KM practices in our teams. We also see good benefits of all these, unfortunately showcasing them as quantitative benefits is always a challenge (I am talking about benefits such as reduction in xxxx, improvement in xxxx and so on... not about quantitative KM dashboard data such as no. of SMEs, no. of posts, no. of assets, no. of sessions, etc.).

The question which is always asked on KM performance is, what is the benefit. Change in culture, improvement in collaborating, creation of assets, etc. is not an answer to this. since this is just an output but not measure of benefit/success. Can someone share, how we can correlate KM with business benefits OR how can we show return on investment in KM. Thank you.


 

Hello Sachin, 

At GE Renewable Energy, we come up with the following business metric to justify the investment for the KM program which is "Cost avoidance in productivity in terms of saved engineering hours and mitigated risks" when reusing solutions to previously solved problems or issues (non-conformance, poor quality,...) 

The key elements to enable the calculation are:

1- solve problems.
2- retain your solutions with proper taxonomy,
3- Make the solutions discoverable and searchable (we call it 'critical knowledge finder')
4- measure the reuse or application of these solutions.
5- Recognize and reward the contributors. 

Thank you

Rachad 
 


Carol H. Tucker
 

One thing that I have always noted is that it is a harder sell to implement a dedicated KM program to small business owners as opposed to larger organizations.  I have always espoused, therefore, "stealth KM" where you incorporate good practices into everyday processes and procedures rather than try to pitch a new program with a dedicated staff.

-- 
Carol H. Tucker

"I only care about the words that flutter from your mind. They are the only thing you truly own. The only thing I will remember you by. I will not fall in love with your bones and skin. I will not fall in love with the places you have been. I will not fall in love with anything but the words that flutter from your Extraordinary Mind."
~ Andre Jordan


Nick Milton
 

Hi Sachin

 

You can find some survey data about KM benefits in the following blog post

http://www.nickmilton.com/2020/03/what-measurable-benefits-can-you-get.html

 

The blog also contains 140 case histories of quantified KM value

http://www.nickmilton.com/search/label/quantified

 

We wrote a newsletter on the topic of KM ROI back in 2012 on KM and ROI which you can find here, and which explains ways in which KM ROI can be calculated

https://www.knoco.com/Knoco%20newsletter%20october%202012.pdf

 

Hope these resources help

 

Nick Milton

 

 

From: main@SIKM.groups.io <main@SIKM.groups.io> On Behalf Of Sachin Joshi
Sent: 15 October 2020 15:08
To: main@SIKM.groups.io
Subject: [SIKM] KM ROI #question

 

Hello all, this is my first post on this very insightful forum/group.

I am KM consultant in IT organization. We implement and monitor lot of good KM practices in our teams. We also see good benefits of all these, unfortunately showcasing them as quantitative benefits is always a challenge (I am talking about benefits such as reduction in xxxx, improvement in xxxx and so on... not about quantitative KM dashboard data such as no. of SMEs, no. of posts, no. of assets, no. of sessions, etc.).

The question which is always asked on KM performance is, what is the benefit. Change in culture, improvement in collaborating, creation of assets, etc. is not an answer to this. since this is just an output but not measure of benefit/success. Can someone share, how we can correlate KM with business benefits OR how can we show return on investment in KM. Thank you.


Nirmala Palaniappan
 

Hi Sachin,

There are, as Tom indicates, many ways of approaching this. Here is a fundamental list of business parameters that KM can be mapped to. 

1. Productivity improvement - Is there a business area where productivity is a concern? If so, is it because of something related to how new knowledge is created, transferred, applied or reused? 
2. Competency improvement - Is there a business area where there is a significant percentage of new hires who need to go through an exhaustive induction and competency building activities? How can collective knowledge be leveraged to achieve this, in addition to conventional training?
3. Cycle time reduction - Are there processes that are taking too long and is it because of a gap in how knowledge associated with it is created, consolidated, transferred or applied? 
4. Response time reduction - Is there a customer-facing process that can be improved by reducing the time it takes to resolve a problem that is reported? How can knowledge be better captured, shared and reused to contribute to this objective? 
5. Cost reduction - Are there business areas where we are reinventing the wheel and therefore unwittingly increasing the expenses? Can we leverage on best practices, knowledge sharing and communities to reuse knowledge and save money?
6. Innovation - How can we create teams and communities to create a new revenue-generating opportunity for the business or adopt radically different ways of solving nagging problems? 

Hope this helps! 

Regards
Nirmala 

On Thu, 15 Oct 2020 at 7:38 PM, Sachin Joshi <sachinjoshi.a@...> wrote:
Hello all, this is my first post on this very insightful forum/group.

I am KM consultant in IT organization. We implement and monitor lot of good KM practices in our teams. We also see good benefits of all these, unfortunately showcasing them as quantitative benefits is always a challenge (I am talking about benefits such as reduction in xxxx, improvement in xxxx and so on... not about quantitative KM dashboard data such as no. of SMEs, no. of posts, no. of assets, no. of sessions, etc.).

The question which is always asked on KM performance is, what is the benefit. Change in culture, improvement in collaborating, creation of assets, etc. is not an answer to this. since this is just an output but not measure of benefit/success. Can someone share, how we can correlate KM with business benefits OR how can we show return on investment in KM. Thank you.

--
"The faithful see the invisible, believe the incredible and then receive the impossible" - Anonymous


Stan Garfield
 

Sachin, welcome to the community!  And thanks for posting here.

For previous discussions on this topic, see the threads with the #value hashtag.

See also:


Murray Jennex
 

we statistically validated the following set of KM success measures:

Impact on business processes (6 measures):
1.         Improved the efficiency of the supported processes
2.         Reduced costs for the supported business process
3.         Positive return on investment for the supported processes
4.         Improved the effectiveness of the supported processes
5.         Improved decision making in the supported processes
6.         Improved resource allocation in the supported process
Impact on KM Strategy (4 measures):
1.         Changes to the organization’s KM goals
2.         Creation or modification of knowledge related key performance indicators (KPIs)
3.         Changes to the way the organization assessed knowledge use in the organization
4.         Changes in the organization’s incentives for using and sharing knowledge
Leadership/Management Support (4 measures):
1.         Increased verbal/political support for KM by top management
2.         Increased financial support for KM by top management
3.         Increased awareness of KM by top management
4.         Increased use/reliance on KM by top management
Knowledge Content (3 measures)
1.         Increased use or intention to use of knowledge content
2.         Increased identification of needed knowledge content and knowledge content sources
3.         Increased demand and/or searching for knowledge content

thanks....murray jennex, 

-----Original Message-----
From: Sachin Joshi <sachinjoshi.a@...>
To: main@SIKM.groups.io
Sent: Thu, Oct 15, 2020 7:08 am
Subject: [SIKM] KM ROI #question

Hello all, this is my first post on this very insightful forum/group.

I am KM consultant in IT organization. We implement and monitor lot of good KM practices in our teams. We also see good benefits of all these, unfortunately showcasing them as quantitative benefits is always a challenge (I am talking about benefits such as reduction in xxxx, improvement in xxxx and so on... not about quantitative KM dashboard data such as no. of SMEs, no. of posts, no. of assets, no. of sessions, etc.).

The question which is always asked on KM performance is, what is the benefit. Change in culture, improvement in collaborating, creation of assets, etc. is not an answer to this. since this is just an output but not measure of benefit/success. Can someone share, how we can correlate KM with business benefits OR how can we show return on investment in KM. Thank you.


Stephen Bounds
 

Hi Murray,

What do you mean by "statistically validated"?

Surely saying something like "Changes to the organization's KM goals is a measure of KM success" is tautological at best?

Cheers,
Stephen.

====================================
Stephen Bounds
Executive, Information Management
Cordelta
E: stephen.bounds@...
M: 0401 829 096
====================================
On 16/10/2020 7:26 am, Murray Jennex via groups.io wrote:

we statistically validated the following set of KM success measures:

Impact on business processes (6 measures):
1.         Improved the efficiency of the supported processes
2.         Reduced costs for the supported business process
3.         Positive return on investment for the supported processes
4.         Improved the effectiveness of the supported processes
5.         Improved decision making in the supported processes
6.         Improved resource allocation in the supported process
Impact on KM Strategy (4 measures):
1.         Changes to the organization’s KM goals
2.         Creation or modification of knowledge related key performance indicators (KPIs)
3.         Changes to the way the organization assessed knowledge use in the organization
4.         Changes in the organization’s incentives for using and sharing knowledge
Leadership/Management Support (4 measures):
1.         Increased verbal/political support for KM by top management
2.         Increased financial support for KM by top management
3.         Increased awareness of KM by top management
4.         Increased use/reliance on KM by top management
Knowledge Content (3 measures)
1.         Increased use or intention to use of knowledge content
2.         Increased identification of needed knowledge content and knowledge content sources
3.         Increased demand and/or searching for knowledge content

thanks....murray jennex, 

-----Original Message-----
From: Sachin Joshi <sachinjoshi.a@...>
To: main@SIKM.groups.io
Sent: Thu, Oct 15, 2020 7:08 am
Subject: [SIKM] KM ROI #question

Hello all, this is my first post on this very insightful forum/group.

I am KM consultant in IT organization. We implement and monitor lot of good KM practices in our teams. We also see good benefits of all these, unfortunately showcasing them as quantitative benefits is always a challenge (I am talking about benefits such as reduction in xxxx, improvement in xxxx and so on... not about quantitative KM dashboard data such as no. of SMEs, no. of posts, no. of assets, no. of sessions, etc.).

The question which is always asked on KM performance is, what is the benefit. Change in culture, improvement in collaborating, creation of assets, etc. is not an answer to this. since this is just an output but not measure of benefit/success. Can someone share, how we can correlate KM with business benefits OR how can we show return on investment in KM. Thank you.


Stephen Bounds
 

Hi Sachin,

KM4Dev Journal was recently good enough to publish a journal article on mine on the topic. It is my view that relative return on investment (RROI) is generally the correct benefits evaluation tool for KM programs.

Excerpting from the article, I examine four ways of determining benefit:

  • ROI can be used by a government regulator for the purpose of optimising like-for-like process efficiency (i.e.performance improvements that can be made without compromising the quality of their work).

  • SROI measures may be adopted by a service enterprise to demonstrate the theory of change (TOC) underpinning its corporate social responsibility (CSR) activities, increasing the benefits accruing to the company brand.

  • RROI measures can be used by a public company CEO to justify their strategic choices to a board, for a marketing division to justify their spending mix – or indeed, for a KM manager to sell their program to their boss. It is worth noting that RROI is also increasingly relevant for the purposes of valuing impact on the creation or curation of external communities.

  • CBA assessments may be undertaken by not-for-profits to make a case for the release of funds by government or philanthropic services for the use of the non-profit.

Cheers,
Stephen.

====================================
Stephen Bounds
Executive, Information Management
Cordelta
E: stephen.bounds@...
M: 0401 829 096
====================================
On 16/10/2020 12:08 am, Sachin Joshi wrote:

Hello all, this is my first post on this very insightful forum/group.

I am KM consultant in IT organization. We implement and monitor lot of good KM practices in our teams. We also see good benefits of all these, unfortunately showcasing them as quantitative benefits is always a challenge (I am talking about benefits such as reduction in xxxx, improvement in xxxx and so on... not about quantitative KM dashboard data such as no. of SMEs, no. of posts, no. of assets, no. of sessions, etc.).

The question which is always asked on KM performance is, what is the benefit. Change in culture, improvement in collaborating, creation of assets, etc. is not an answer to this. since this is just an output but not measure of benefit/success. Can someone share, how we can correlate KM with business benefits OR how can we show return on investment in KM. Thank you.


Murray Jennex
 

by statistically validated I mean we surveyed KM practitioners and researchers on what they saw as success on a KM initiative as well as how they judged the success of that project/initiative.  we used principle component factor analysis to assess the factors and their grouping, chronbach alphas to judge internal consistency, correlation analysis to make sure all factors were necessary, and regression analysis of each factor group to KM project/initiative success.  The list I sent out were all factors considered statistically significant.  Yes, some are qualitative but the item was stated for instance: "as a result of the KM project/initiative we made changes to our organization's KM goals."  It turns out that this happens more in successful KM projects and initiatives than it does in those not as successful....murray


-----Original Message-----
From: Stephen Bounds <km@...>
To: main@SIKM.groups.io
Sent: Fri, Oct 16, 2020 5:25 pm
Subject: Re: [SIKM] KM ROI #question

Hi Murray,
What do you mean by "statistically validated"?
Surely saying something like "Changes to the organization's KM goals is a measure of KM success" is tautological at best?
Cheers,
Stephen.
====================================
Stephen Bounds
Executive, Information Management
Cordelta
E: stephen.bounds@...
M: 0401 829 096
====================================
On 16/10/2020 7:26 am, Murray Jennex via groups.io wrote:
we statistically validated the following set of KM success measures:

Impact on business processes (6 measures):
1.         Improved the efficiency of the supported processes
2.         Reduced costs for the supported business process
3.         Positive return on investment for the supported processes
4.         Improved the effectiveness of the supported processes
5.         Improved decision making in the supported processes
6.         Improved resource allocation in the supported process
Impact on KM Strategy (4 measures):
1.         Changes to the organization’s KM goals
2.         Creation or modification of knowledge related key performance indicators (KPIs)
3.         Changes to the way the organization assessed knowledge use in the organization
4.         Changes in the organization’s incentives for using and sharing knowledge
Leadership/Management Support (4 measures):
1.         Increased verbal/political support for KM by top management
2.         Increased financial support for KM by top management
3.         Increased awareness of KM by top management
4.         Increased use/reliance on KM by top management
Knowledge Content (3 measures)
1.         Increased use or intention to use of knowledge content
2.         Increased identification of needed knowledge content and knowledge content sources
3.         Increased demand and/or searching for knowledge content

thanks....murray jennex, 

-----Original Message-----
From: Sachin Joshi <sachinjoshi.a@...>
To: main@SIKM.groups.io
Sent: Thu, Oct 15, 2020 7:08 am
Subject: [SIKM] KM ROI #question

Hello all, this is my first post on this very insightful forum/group.

I am KM consultant in IT organization. We implement and monitor lot of good KM practices in our teams. We also see good benefits of all these, unfortunately showcasing them as quantitative benefits is always a challenge (I am talking about benefits such as reduction in xxxx, improvement in xxxx and so on... not about quantitative KM dashboard data such as no. of SMEs, no. of posts, no. of assets, no. of sessions, etc.).

The question which is always asked on KM performance is, what is the benefit. Change in culture, improvement in collaborating, creation of assets, etc. is not an answer to this. since this is just an output but not measure of benefit/success. Can someone share, how we can correlate KM with business benefits OR how can we show return on investment in KM. Thank you.


Tim Powell
 

Hi Murray,

 

From your list of success measures, it occurs to me that only the first group (impact on business processes) are measures of KM Outcome.  The other three groups are measures of KM Output – a critical distinction, in my view.

 

Also, input from KM practitioners (i.e., knowledge producers) should accompany – but not replace – input from knowledge users (client, patrons, customers).  The former are apt to be seen as positively biased, or even self-fulfilling, in my experience.

 

This is all 20/20 hindsight, I realize -- but offered in the hope that it helps your important work gain even greater credibility and traction with skeptical Corner Office types.

 

Kind regards,

 

Tim

 

TIM WOOD POWELL | President, The Knowledge Agency® | Author, The Value of Knowledge

DIRECT/MOBILE +1.212.243.1200 | ZOOM 212-243-1200

SITE www.KnowledgeAgency.com | BLOG www.KnowledgeValueChain.com

 

 

From: <main@SIKM.groups.io> on behalf of "Murray Jennex via groups.io" <murphjen@...>
Reply-To: "main@SIKM.groups.io" <main@SIKM.groups.io>
Date: Friday, October 16, 2020 at 10:14 PM
To: "km@..." <km@...>, "main@SIKM.groups.io" <main@SIKM.groups.io>
Subject: Re: [SIKM] KM ROI #question

 

by statistically validated I mean we surveyed KM practitioners and researchers on what they saw as success on a KM initiative as well as how they judged the success of that project/initiative.  we used principle component factor analysis to assess the factors and their grouping, chronbach alphas to judge internal consistency, correlation analysis to make sure all factors were necessary, and regression analysis of each factor group to KM project/initiative success.  The list I sent out were all factors considered statistically significant.  Yes, some are qualitative but the item was stated for instance: "as a result of the KM project/initiative we made changes to our organization's KM goals."  It turns out that this happens more in successful KM projects and initiatives than it does in those not as successful....murray

-----Original Message-----
From: Stephen Bounds <km@...>
To: main@SIKM.groups.io
Sent: Fri, Oct 16, 2020 5:25 pm
Subject: Re: [SIKM] KM ROI #question

Hi Murray,

What do you mean by "statistically validated"?

Surely saying something like "Changes to the organization's KM goals is a measure of KM success" is tautological at best?

Cheers,
Stephen.

====================================
Stephen Bounds
Executive, Information Management
Cordelta
E: stephen.bounds@...
M: 0401 829 096
====================================

On 16/10/2020 7:26 am, Murray Jennex via groups.io wrote:

we statistically validated the following set of KM success measures:

 

Impact on business processes (6 measures):

1.         Improved the efficiency of the supported processes

2.         Reduced costs for the supported business process

3.         Positive return on investment for the supported processes

4.         Improved the effectiveness of the supported processes

5.         Improved decision making in the supported processes

6.         Improved resource allocation in the supported process

Impact on KM Strategy (4 measures):

1.         Changes to the organization’s KM goals

2.         Creation or modification of knowledge related key performance indicators (KPIs)

3.         Changes to the way the organization assessed knowledge use in the organization

4.         Changes in the organization’s incentives for using and sharing knowledge

Leadership/Management Support (4 measures):

1.         Increased verbal/political support for KM by top management

2.         Increased financial support for KM by top management

3.         Increased awareness of KM by top management

4.         Increased use/reliance on KM by top management

Knowledge Content (3 measures)

1.         Increased use or intention to use of knowledge content

2.         Increased identification of needed knowledge content and knowledge content sources

3.         Increased demand and/or searching for knowledge content

 

thanks....murray jennex, 

-----Original Message-----
From: Sachin Joshi <sachinjoshi.a@...>
To: main@SIKM.groups.io
Sent: Thu, Oct 15, 2020 7:08 am
Subject: [SIKM] KM ROI #question

Hello all, this is my first post on this very insightful forum/group.

I am KM consultant in IT organization. We implement and monitor lot of good KM practices in our teams. We also see good benefits of all these, unfortunately showcasing them as quantitative benefits is always a challenge (I am talking about benefits such as reduction in xxxx, improvement in xxxx and so on... not about quantitative KM dashboard data such as no. of SMEs, no. of posts, no. of assets, no. of sessions, etc.).

The question which is always asked on KM performance is, what is the benefit. Change in culture, improvement in collaborating, creation of assets, etc. is not an answer to this. since this is just an output but not measure of benefit/success. Can someone share, how we can correlate KM with business benefits OR how can we show return on investment in KM. Thank you.


Murray Jennex
 

I agree with you sort of Tim.  KM output is also a reflection of KM outcome as what we found is that the output doesn't happen if the KM participants/users don't see a value in what KM has done.  The output measures are the qualitative measures of KM outcome.  For example, leadership doesn't increase support unless there is a value story there to begin with.  Users don't increase knowledge contributions unless they see value in KM.  Even our own list server operates this way.  Some questions get a lot of responses while others get less, our group responds to questions they see value in answering.  Sure it might be a output (number of responses, length of responses, even amount of arguing) but it still reflects which questions were more successful and hence outcome.

The theory of why this is okay is based on Theory of Reasoned Action and Intent to Use.  Output measures indicate an intent to use KM at some point while the outcome measures reflect KM use.  I've done much research and in the voluntary use environment that KM is, intent to use is a critical measure to ensure continued KM use.  Continued KM use is critical as it is important to know that users WILL use KM in the future when appropriate, and this is a measure of KM success.  

So yes, two types of measures, those that reflect outcome of actual KM use, and those that reflect outputs that indicate Intent to Use.....murray jennex


-----Original Message-----
From: Tim Powell <tim.powell@...>
To: main@SIKM.groups.io <main@SIKM.groups.io>; km@... <km@...>
Sent: Tue, Oct 20, 2020 11:05 am
Subject: Re: [SIKM] KM ROI #question

Hi Murray,
 
From your list of success measures, it occurs to me that only the first group (impact on business processes) are measures of KM Outcome.  The other three groups are measures of KM Output – a critical distinction, in my view.
 
Also, input from KM practitioners (i.e., knowledge producers) should accompany – but not replace – input from knowledge users (client, patrons, customers).  The former are apt to be seen as positively biased, or even self-fulfilling, in my experience.
 
This is all 20/20 hindsight, I realize -- but offered in the hope that it helps your important work gain even greater credibility and traction with skeptical Corner Office types.
 
Kind regards,
 
Tim
 
TIM WOOD POWELL | President, The Knowledge Agency® | Author, The Value of Knowledge
DIRECT/MOBILE +1.212.243.1200 | ZOOM 212-243-1200
 
 
From: <main@SIKM.groups.io> on behalf of "Murray Jennex via groups.io" <murphjen@...>
Reply-To: "main@SIKM.groups.io" <main@SIKM.groups.io>
Date: Friday, October 16, 2020 at 10:14 PM
To: "km@..." <km@...>, "main@SIKM.groups.io" <main@SIKM.groups.io>
Subject: Re: [SIKM] KM ROI #question
 
by statistically validated I mean we surveyed KM practitioners and researchers on what they saw as success on a KM initiative as well as how they judged the success of that project/initiative.  we used principle component factor analysis to assess the factors and their grouping, chronbach alphas to judge internal consistency, correlation analysis to make sure all factors were necessary, and regression analysis of each factor group to KM project/initiative success.  The list I sent out were all factors considered statistically significant.  Yes, some are qualitative but the item was stated for instance: "as a result of the KM project/initiative we made changes to our organization's KM goals."  It turns out that this happens more in successful KM projects and initiatives than it does in those not as successful....murray

-----Original Message-----
From: Stephen Bounds <km@...>
To: main@SIKM.groups.io
Sent: Fri, Oct 16, 2020 5:25 pm
Subject: Re: [SIKM] KM ROI #question
Hi Murray,
What do you mean by "statistically validated"?
Surely saying something like "Changes to the organization's KM goals is a measure of KM success" is tautological at best?
Cheers,
Stephen.
====================================
Stephen Bounds
Executive, Information Management
Cordelta
E: stephen.bounds@...
M: 0401 829 096
====================================
On 16/10/2020 7:26 am, Murray Jennex via groups.io wrote:
we statistically validated the following set of KM success measures:
 
Impact on business processes (6 measures):
1.         Improved the efficiency of the supported processes
2.         Reduced costs for the supported business process
3.         Positive return on investment for the supported processes
4.         Improved the effectiveness of the supported processes
5.         Improved decision making in the supported processes
6.         Improved resource allocation in the supported process
Impact on KM Strategy (4 measures):
1.         Changes to the organization’s KM goals
2.         Creation or modification of knowledge related key performance indicators (KPIs)
3.         Changes to the way the organization assessed knowledge use in the organization
4.         Changes in the organization’s incentives for using and sharing knowledge
Leadership/Management Support (4 measures):
1.         Increased verbal/political support for KM by top management
2.         Increased financial support for KM by top management
3.         Increased awareness of KM by top management
4.         Increased use/reliance on KM by top management
Knowledge Content (3 measures)
1.         Increased use or intention to use of knowledge content
2.         Increased identification of needed knowledge content and knowledge content sources
3.         Increased demand and/or searching for knowledge content
 
thanks....murray jennex, 
-----Original Message-----
From: Sachin Joshi <sachinjoshi.a@...>
To: main@SIKM.groups.io
Sent: Thu, Oct 15, 2020 7:08 am
Subject: [SIKM] KM ROI #question
Hello all, this is my first post on this very insightful forum/group.

I am KM consultant in IT organization. We implement and monitor lot of good KM practices in our teams. We also see good benefits of all these, unfortunately showcasing them as quantitative benefits is always a challenge (I am talking about benefits such as reduction in xxxx, improvement in xxxx and so on... not about quantitative KM dashboard data such as no. of SMEs, no. of posts, no. of assets, no. of sessions, etc.).

The question which is always asked on KM performance is, what is the benefit. Change in culture, improvement in collaborating, creation of assets, etc. is not an answer to this. since this is just an output but not measure of benefit/success. Can someone share, how we can correlate KM with business benefits OR how can we show return on investment in KM. Thank you.


Stephen Bounds
 

Hi Murray,

That's interesting. You've provided some additional support for a VINE research article I was reading describing a similar phenomenon. It did a meta-analysis which found a positive correlation between the following factors and actual Knowledge Sharing Behaviours:

  • Personal positive attitude towards knowledge sharing = 0.42-0.53
  • Intent to share knowledge = 0.30-0.46
  • Perceived behavioural control to perform knowledge sharing = 0.15-0.42
  • Social norms encouraging knowledge sharing = 0.21-0.35

Put another way, the study found that your personal belief that knowledge sharing is important trumps even a genuine intent of people to actually share when it comes to factors of success for your Knowledge Sharing Program.

Cheers,
Stephen.

====================================
Stephen Bounds
Executive, Information Management
Cordelta
E: stephen.bounds@...
M: 0401 829 096
====================================
On 21/10/2020 5:27 am, Murray Jennex wrote:

I agree with you sort of Tim.  KM output is also a reflection of KM outcome as what we found is that the output doesn't happen if the KM participants/users don't see a value in what KM has done.  The output measures are the qualitative measures of KM outcome.  For example, leadership doesn't increase support unless there is a value story there to begin with.  Users don't increase knowledge contributions unless they see value in KM.  Even our own list server operates this way.  Some questions get a lot of responses while others get less, our group responds to questions they see value in answering.  Sure it might be a output (number of responses, length of responses, even amount of arguing) but it still reflects which questions were more successful and hence outcome.

The theory of why this is okay is based on Theory of Reasoned Action and Intent to Use.  Output measures indicate an intent to use KM at some point while the outcome measures reflect KM use.  I've done much research and in the voluntary use environment that KM is, intent to use is a critical measure to ensure continued KM use.  Continued KM use is critical as it is important to know that users WILL use KM in the future when appropriate, and this is a measure of KM success.  

So yes, two types of measures, those that reflect outcome of actual KM use, and those that reflect outputs that indicate Intent to Use.....murray jennex


-----Original Message-----
From: Tim Powell <tim.powell@...>
To: main@SIKM.groups.io <main@SIKM.groups.io>; km@... <km@...>
Sent: Tue, Oct 20, 2020 11:05 am
Subject: Re: [SIKM] KM ROI #question

Hi Murray,
 
From your list of success measures, it occurs to me that only the first group (impact on business processes) are measures of KM Outcome.  The other three groups are measures of KM Output – a critical distinction, in my view.
 
Also, input from KM practitioners (i.e., knowledge producers) should accompany – but not replace – input from knowledge users (client, patrons, customers).  The former are apt to be seen as positively biased, or even self-fulfilling, in my experience.
 
This is all 20/20 hindsight, I realize -- but offered in the hope that it helps your important work gain even greater credibility and traction with skeptical Corner Office types.
 
Kind regards,
 
Tim
 
TIM WOOD POWELL | President, The Knowledge Agency® | Author, The Value of Knowledge
DIRECT/MOBILE +1.212.243.1200 | ZOOM 212-243-1200
 
 
From: <main@SIKM.groups.io> on behalf of "Murray Jennex via groups.io" <murphjen@...>
Reply-To: "main@SIKM.groups.io" <main@SIKM.groups.io>
Date: Friday, October 16, 2020 at 10:14 PM
To: "km@..." <km@...>, "main@SIKM.groups.io" <main@SIKM.groups.io>
Subject: Re: [SIKM] KM ROI #question
 
by statistically validated I mean we surveyed KM practitioners and researchers on what they saw as success on a KM initiative as well as how they judged the success of that project/initiative.  we used principle component factor analysis to assess the factors and their grouping, chronbach alphas to judge internal consistency, correlation analysis to make sure all factors were necessary, and regression analysis of each factor group to KM project/initiative success.  The list I sent out were all factors considered statistically significant.  Yes, some are qualitative but the item was stated for instance: "as a result of the KM project/initiative we made changes to our organization's KM goals."  It turns out that this happens more in successful KM projects and initiatives than it does in those not as successful....murray

-----Original Message-----
From: Stephen Bounds <km@...>
To: main@SIKM.groups.io
Sent: Fri, Oct 16, 2020 5:25 pm
Subject: Re: [SIKM] KM ROI #question
Hi Murray,
What do you mean by "statistically validated"?
Surely saying something like "Changes to the organization's KM goals is a measure of KM success" is tautological at best?
Cheers,
Stephen.
====================================
Stephen Bounds
Executive, Information Management
Cordelta
E: stephen.bounds@...
M: 0401 829 096
====================================
On 16/10/2020 7:26 am, Murray Jennex via groups.io wrote:
we statistically validated the following set of KM success measures:
 
Impact on business processes (6 measures):
1.         Improved the efficiency of the supported processes
2.         Reduced costs for the supported business process
3.         Positive return on investment for the supported processes
4.         Improved the effectiveness of the supported processes
5.         Improved decision making in the supported processes
6.         Improved resource allocation in the supported process
Impact on KM Strategy (4 measures):
1.         Changes to the organization’s KM goals
2.         Creation or modification of knowledge related key performance indicators (KPIs)
3.         Changes to the way the organization assessed knowledge use in the organization
4.         Changes in the organization’s incentives for using and sharing knowledge
Leadership/Management Support (4 measures):
1.         Increased verbal/political support for KM by top management
2.         Increased financial support for KM by top management
3.         Increased awareness of KM by top management
4.         Increased use/reliance on KM by top management
Knowledge Content (3 measures)
1.         Increased use or intention to use of knowledge content
2.         Increased identification of needed knowledge content and knowledge content sources
3.         Increased demand and/or searching for knowledge content
 
thanks....murray jennex, 
-----Original Message-----
From: Sachin Joshi <sachinjoshi.a@...>
To: main@SIKM.groups.io
Sent: Thu, Oct 15, 2020 7:08 am
Subject: [SIKM] KM ROI #question
Hello all, this is my first post on this very insightful forum/group.

I am KM consultant in IT organization. We implement and monitor lot of good KM practices in our teams. We also see good benefits of all these, unfortunately showcasing them as quantitative benefits is always a challenge (I am talking about benefits such as reduction in xxxx, improvement in xxxx and so on... not about quantitative KM dashboard data such as no. of SMEs, no. of posts, no. of assets, no. of sessions, etc.).

The question which is always asked on KM performance is, what is the benefit. Change in culture, improvement in collaborating, creation of assets, etc. is not an answer to this. since this is just an output but not measure of benefit/success. Can someone share, how we can correlate KM with business benefits OR how can we show return on investment in KM. Thank you.


Sreejith Menath
 
Edited

Hi Sachin,
Authors here have stated various quantifiable metrics with varied contextual relevance. They are important indeed, and need to be considered.
But here, my approach to this is 'pre-metric systematization.'
Even before, you think of relating a KM activity with an organizational strategic outcome and then create a metric around it.First you need to map meaningful clusters of KM activities/processes to functional processes of enterprise architecture. Both with regard to level of people/roles and the informational  pathways that relate them with one another.
Then, you proceed to update these clusters with along their labels and characterization to cross-functional - KM mapping .
In each level there need to be calibration with regard to alignment or mis-alignment with organizational goals. Again alignment not (only) in terms of mathematical measures, but more so on "semantic and contextual relevance"
Finally the cross-cluster mappings need to be integrated to single version of truth inter-connected thread, which has a one-shot representation of status quo, given any level of modularity required by the decision makers.
Based on the coherence or may be lack of it thereof, the top management can identify gaps and chalk out ways to measure and improve, using "meaningful" measures. These measures can now assume an auto-focus capability to reflect the required level of granularity rather than a disconnected one-off number, that do not say much about what happened behind the screen.

Hope this helps, with some pointers.
Open for discussion and critiques on this.

Thanks!


 

A lot of good conversation here already Sachin, so I will just make one small point.

Quite a few times I have seen KM hard to measure because evaluation was done as an exercise after the fact rather than done properly before and right through KM projects. As a result there is nothing agreed to measure to and the things you do pick, often Operations or Sales will say "That wasn't because of KM, that was because we worked hard!".  Good end to end evaluation (and if research is involved then good Knowledge Translation) helps keep the "Quantitative Or Nothing" managers on-side.

Stu French.


 

Well stated Stu!

 

From: main@SIKM.groups.io <main@SIKM.groups.io> On Behalf Of Stuart French via groups.io
Sent: Monday, October 26, 2020 20:12
To: main@SIKM.groups.io
Subject: Re: [SIKM] KM ROI #question #value

 

A lot of good conversation here already Sachin, so I will just make one small point.

Quite a few times I have seen KM hard to measure because evaluation was done as an exercise after the fact rather than done properly before and right through KM projects. As a result there is nothing agreed to measure to and the things you do pick, often Operations or Sales will say "That wasn't because of KM, that was because we worked hard!".  Good end to end evaluation (and if research is involved then good Knowledge Translation) helps keep the "Quantitative Or Nothing" managers on-side.

Stu French.


Murray Jennex
 

One thing I'd add to the end to end evaluation is to design the knowledge use to support organizational KPIs from the very beginning....murray jennex


-----Original Message-----
From: bill@... <bill@...>
To: main@SIKM.groups.io <main@SIKM.groups.io>
Sent: Mon, Oct 26, 2020 9:51 pm
Subject: Re: [SIKM] KM ROI #question #value

Well stated Stu!
 
From: main@SIKM.groups.io <main@SIKM.groups.io> On Behalf Of Stuart French via groups.io
Sent: Monday, October 26, 2020 20:12
To: main@SIKM.groups.io
Subject: Re: [SIKM] KM ROI #question #value
 
A lot of good conversation here already Sachin, so I will just make one small point.

Quite a few times I have seen KM hard to measure because evaluation was done as an exercise after the fact rather than done properly before and right through KM projects. As a result there is nothing agreed to measure to and the things you do pick, often Operations or Sales will say "That wasn't because of KM, that was because we worked hard!".  Good end to end evaluation (and if research is involved then good Knowledge Translation) helps keep the "Quantitative Or Nothing" managers on-side.

Stu French.


Dennis Thomas
 

Thanks Stu and others for this great thread.  I would like to add, as Stu says, "Quite a few times I have seen KM hard to measure because evaluation was done as an exercise after the fact rather than done properly before and right through KM projects." 

From a cognitive point of view, our brains are constantly asking questions about every situation and circumstance we are involved.   Given this, our approach is to ask the big questions before conducting a project.  The attached 10-Questions document defines questions that defined a project conducted by my mentor and business partner Dr. Richard L. Ballard (now deceased) and Martha Nawrocki, our Master Knowledge Engineer.   I have previously mentioned, but never explained these, but will now. 

This project addressed the need to down size the U.S. Science and Technology Laboratories (the “white hats”). The requirement was to determine what these laboratories were working on and how that knowledge could be used to reduce the time it was taking to define, design, develop, and test new warfare platforms.  Platforms are ships, aircraft, etc.  This project began by Admirals and Generals defining the 10-most important questions the knowledge base need to answer.  The top 10 questions typically answer 30-40 key questions overall.  (See attached). 

These questions require RATIONAL answers as opposed too INFORMATION answers.  Admirals and Generals make decisions where there are no right answers, only trade-offs.  (Choose your own devil - halloween lingo).   There were 70 SME documents that contained the knowledge required to answer these questions, plus the need to import data from disparate sources to keep the content updated.  The attached Modeler’s Map show the knowledge flow between the financial groups (Congress & the Commands), the S&T Labs, manufacturers, and end users.  The end result is that much of the S&T knowledge was not be adequately implemented because the Program Managers did not understand the SME language.  This means that knowledge systems need to interact with users using the language they know and understand. 

A much older knowledge system invented by Ballard was used to develop this project.  It took six months.  Our current system is a concept-driven cognitive system, now being updated for launch in the first quarter of 2021.  Again, we are looking for two U.S. commercial companies that want to be involve in our NSF Phase II projects (we already have a franchise operation that is ready to go).  Phase II comes with a $1,000,000 grant to help us to support those projects. 

Let’s talk about a new paradigm of computing that is PURE KM.  

Dennis L. Thomas
IQStrategix
(810) 662-5199

Leveraging Organizational Knowledge 


On October 27, 2020 at 1:36:28 AM, Murray Jennex via groups.io (murphjen@...) wrote:

One thing I'd add to the end to end evaluation is to design the knowledge use to support organizational KPIs from the very beginning....murray jennex


-----Original Message-----
From: bill@... <bill@...>
To: main@SIKM.groups.io <main@SIKM.groups.io>
Sent: Mon, Oct 26, 2020 9:51 pm
Subject: Re: [SIKM] KM ROI #question #value

Well stated Stu!
 
From: main@SIKM.groups.io <main@SIKM.groups.io> On Behalf Of Stuart French via groups.io
Sent: Monday, October 26, 2020 20:12
To: main@SIKM.groups.io
Subject: Re: [SIKM] KM ROI #question #value
 
A lot of good conversation here already Sachin, so I will just make one small point.

Quite a few times I have seen KM hard to measure because evaluation was done as an exercise after the fact rather than done properly before and right through KM projects. As a result there is nothing agreed to measure to and the things you do pick, often Operations or Sales will say "That wasn't because of KM, that was because we worked hard!".  Good end to end evaluation (and if research is involved then good Knowledge Translation) helps keep the "Quantitative Or Nothing" managers on-side.

Stu French.