Topics

Weighing the benefits of productivity improvements #value


Yao Ge
 

Hello KM peers, I am once again asked to justify what I am doing by associating my work in KM area to impact to business bottom lines such as hard-savings. This type of request must be common across KM and other collaboration related areas. It is a fair question but quite difficult to articulate.
 
Below is a short article I just wrote on how to measure the benefits of productivity improvements and I would like to hear what this group thinks on this topic (not only on workplace efficiency but also KM/Collaboration/Innovation etc). Thanks in advance!
 

One of the most significant challenges in today's knowledge work is to deal with explosion of "Information Overload". The saving of workers time in searching, accessing, and filtering the vast amount of information scattered across personal desktop, shared workspace, and enterprise repositories can be measured and potentially translated to hard-savings in workspace.

In the age of industry evaluation, we measure productivities by yields and through-put of the workers on the assembly lines. It is a bit harder to measure the productivities in the information age as knowledge work performed today as less mechanical and procedural in nature. Some type of work such as research and development can be measured by number of patents, number of design iterations, and product development lead times.

As an example, according to an IDC study, a knowledge work spends around an average of 9.5 hours per week searching for information. Based on the average salary plus benefits of $60,000/year from Department of Labor (2005), this would translate to $14,251.90 a year per worker. With improved enterprise search, there are two possible out-comes depends on the nature of the work. The knowledge workers can reduced the amount of the time spent on search for information. Or the they might actually spent the same or even more time on searching but discover more relevant information which results in improvements in the quality of the job deliverables. In the first case, a saving 10% of the time (1 hour per week) can be translated into potential saving of $1,425 a year per worker. If we only effected a 10% of the knowledge worker population at Ford, that would a saving of 3,000 x $1,425 = $4,275,000. Can this be translated to reduced workforce in term salary and benefits paid? Probably not, but we can safely say it would positively impacted the business bottom line indirectly through better work through-put, increased quality, and improved morale (reduced stressed at work). The second scenario is much harder to measure. If a research scientist can find a new connections among previously unknown facts due to improved search performance, the impact of productivity gain can be measured by new patents and innovate products that changes the competitive landscape in the industry. As we can tell, the benefits of various technologies on productivities can be measured directly through the amount of time saved. But the impacts are far more indirect and potentially very significant.

As you can see, there is a lot of "IF" statements. To a large degree, The numbers are anecdotal but yet very convincing to reflect the order of magnitude of the positive impact. Like other strategic investments, such as Knowledge Management, the returns are mixed with multitude of other internal and external factors when it comes to impact on bottom line. In a business environment of doing more with less, productivity improvement is not a "nice to have" but a basic element for survival.


Bill Dixon
 

Hello Yao,
 
I was asked a similar question this morning and have also been formulating a response. 
 
I like the general approach you have taken in the draft of your article.  Without getting in too much detail, you do a good job describing one of the potential benefits of KM.  In our environment, KM can also help mitigate risk.  By leveraging not only best practices but by avoiding worst practices, we avoid making potentially costly mistakes over an over again.  The impact on the business can be significant.
 
I am adapting Yelden and Albers (2004) (http://www.tlainc.com/articl69.htm) approach to establishing business cases for KM initiatives to our environment.  They advocate the following framework for developing business cases around KM initiatives:,

1.      Strategy assessment

2.      Knowledge audit

3.      Knowledge and business strategy alignment

4.      Opportunity identification

5.      Value, business benefits, and evaluation

6.      Risk reduction techniques

"In order to make a successful justification, it is necessary to clearly identify all the options available with the associated risks involved with each choice.  Furthermore, it is required to first identify and separate the benefits of a given initiative, then determine its value to the firm, before proceeding to infer an associated cost and expected return for undertaking the effort.  Clearly delineating the expected hard and soft benefits of each aspect of the initiative will greatly aid in effectively justifying its need."

I will send you a copy when I am finished. 

Thanks for sharing your draft.

Regards,

Bill Dixon,

Ernst & Young, LLP

 
 

----- Original Message -----
Sent: Wednesday, August 13, 2008 10:16 AM
Subject: [sikmleaders] Weighing the benefits of productivity improvements

Hello KM peers, I am once again asked to justify what I am doing by associating my work in KM area to impact to business bottom lines such as hard-savings. This type of request must be common across KM and other collaboration related areas. It is a fair question but quite difficult to articulate.
 
Below is a short article I just wrote on how to measure the benefits of productivity improvements and I would like to hear what this group thinks on this topic (not only on workplace efficiency but also KM/Collaboration/Innovation etc). Thanks in advance!
 

One of the most significant challenges in today's knowledge work is to deal with explosion of "Information Overload". The saving of workers time in searching, accessing, and filtering the vast amount of information scattered across personal desktop, shared workspace, and enterprise repositories can be measured and potentially translated to hard-savings in workspace.

In the age of industry evaluation, we measure productivities by yields and through-put of the workers on the assembly lines. It is a bit harder to measure the productivities in the information age as knowledge work performed today as less mechanical and procedural in nature. Some type of work such as research and development can be measured by number of patents, number of design iterations, and product development lead times.

As an example, according to an IDC study, a knowledge work spends around an average of 9.5 hours per week searching for information. Based on the average salary plus benefits of $60,000/year from Department of Labor (2005), this would translate to $14,251.90 a year per worker. With improved enterprise search, there are two possible out-comes depends on the nature of the work. The knowledge workers can reduced the amount of the time spent on search for information. Or the they might actually spent the same or even more time on searching but discover more relevant information which results in improvements in the quality of the job deliverables. In the first case, a saving 10% of the time (1 hour per week) can be translated into potential saving of $1,425 a year per worker. If we only effected a 10% of the knowledge worker population at Ford, that would a saving of 3,000 x $1,425 = $4,275,000. Can this be translated to reduced workforce in term salary and benefits paid? Probably not, but we can safely say it would positively impacted the business bottom line indirectly through better work through-put, increased quality, and improved morale (reduced stressed at work). The second scenario is much harder to measure. If a research scientist can find a new connections among previously unknown facts due to improved search performance, the impact of productivity gain can be measured by new patents and innovate products that changes the competitive landscape in the industry. As we can tell, the benefits of various technologies on productivities can be measured directly through the amount of time saved. But the impacts are far more indirect and potentially very significant.

As you can see, there is a lot of "IF" statements. To a large degree, The numbers are anecdotal but yet very convincing to reflect the order of magnitude of the positive impact. Like other strategic investments, such as Knowledge Management, the returns are mixed with multitude of other internal and external factors when it comes to impact on bottom line. In a business environment of doing more with less, productivity improvement is not a "nice to have" but a basic element for survival.


Andrew Gent
 

Yao and Bill,

With improved enterprise search ... knowledge workers can reduce the amount of the time spent on search ... a saving 10% of the time (1 hour per week) can be translated into potential saving of $1,425 a year per worker.

I am going to play devil's advocate here. Not because I don't like your arguments. I have used almost the exact same argument myself. But I have also had the following discussions with senior business managers. The conversation goes something like this:

  • We just spent $XXX thousand dollars last year on a new search engine. Where are the savings from that improvement?
  • 1 hour a week is 2.5% of a 40hr week. Show me where I am getting a 2.5% increase in production or performance? (Not theory but actual improvements in the bottom line.)
  • "... a saving of 3,000 x $1,425 = $4,275,000" $4 million return on an investment of what? -- say 4 people for a year, approximately $500K -- that's an ROI of 700% in one year. Who are you kidding? Real ROI for things like SAP are 200-400% over a 6 year period, not even breaking even for 2-3 years. I don't believe it.
Now having said all that, it occurs to me that there are two types of conversation like this. The justify-your-program discussion with your direct manager and the justify-your-expenditure discussion with upper management. What I have just described is the latter, which is why justifying KM programs is so difficult. Saved time and unmeasurable performance improvements simply don't go over in budget discussions.

However, for justifying your program to your direct manager, the argument concerning saved time is important, because usually that person is also managing the people whose performance you will impact. So I would suggest three things for that discussion:
  • Keep the argument about saving time. This has meaning to that manager (since they directly manage employees such as yourself)
  • Don't bother getting into the theoretical monetary savings, because it is meaningless. That manager is not going to see that money, so it is a waste of time.
  • As Stan has suggested before, use multiple arguments. Argue the saved time. Also provide supporting evidence -- emails, forum postings, performance review input -- from people who say your program has helped them save time or be more effective. It is best to  collect this information on an ongoing basis so you have it handy when you are asked to justify your program. It is hard-to-impossible to collect in a hurry at the last minute.
Best,

Andrew Gent
ajgent@...
(603) 888-0370


 


sjagannath@...
 


Hi Yao,

This is a wonderful topic. Around an year time back, while I developed the business case for instituting an enterprise search (certainly as part of KM initiative) I used same argument. Presented similar numbers of savings/year. But, with this calculation we are probably assuming the search would always provide relevant results...
My experience, it does not in many cases!! If in case it does capturing the same is almost an impossible task. How many of us have contributed information for formulating case study or rated an relevant article? Guess it is very difficult to have that discipline built within an organization to provide feedback/rating.

Andrew, Your thought process is absolutely right that a small search initiative cannot quantify million dollars to the bottom line... wonderfully substantiated with ERP example consuming more time than just 1 - 2 years for showing benefits.

Hence, am sailing in the same boat as Yao & looking forward for some more meaningful metrics, where dependency on feedback is relatively less.

Regards,
Srinivas P Jagannath



Peter Dorfman <pdorfman@...>
 

All of which suggests a simple best practice: Build in your internal marketing
for the initiative, whatever it is -- and this includes a constant commitment to
getting end users to speak up as advocates/references/case study subjects for
whatever you've given them -- from DAY 1. Do NOT expect the benefits to be
self-evident, and don't be stuck playing catch-up a month before the next budget
review.

Peter Dorfman

On Thu Aug 14 0:25 , Andrew Gent <ajgent@yahoo.com> sent:







Yao and Bill,

With improved enterprise
search ... knowledge workers can reduce the amount of the time spent on search
... a saving 10% of the
time (1 hour per week) can be translated into potential saving of $1,425 a
year per worker.

I am going to play devil's advocate here. Not because I don't like your
arguments. I have used almost the exact same argument myself. But I have also had
the following discussions with senior business managers. The conversation goes
something like this:

We just spent $XXX thousand dollars last year on a new search engine. Where are
the savings from that improvement?1 hour a week is 2.5% of a 40hr week. Show me
where I am getting a 2.5% increase in production or performance? (Not theory but
actual improvements in the bottom line.)"... a saving of 3,000 x $1,425 =
$4,275,000" $4 million return on an investment of what? -- say 4 people for a
year, approximately $500K -- that's an ROI of 700% in one year. Who are you
kidding? Real ROI for
things like SAP are 200-400% over a 6 year period, not even breaking even for
2-3 years. I don't believe it.
Now having said all that, it occurs to me that there are two types of
conversation like this. The justify-your-program discussion with your direct
manager and the justify-your-expenditure discussion with upper management. What I
have just described is the latter, which is why justifying KM programs is so
difficult. Saved time and unmeasurable performance improvements simply don't go
over in budget discussions.

However, for justifying your program to your direct manager, the argument
concerning saved time is important, because usually that person is also managing
the people whose performance you will impact. So I would suggest three things for
that discussion:
Keep the argument about saving time. This has meaning to that manager (since
they directly manage
employees such as yourself)Don't bother getting into the theoretical monetary
savings, because it is meaningless. That manager is not going to see that money,
so it is a waste of time.As Stan has suggested before, use multiple arguments.
Argue the saved time. Also provide supporting evidence -- emails, forum postings,
performance review input -- from people who say your program has helped them save
time or be more effective. It is best to collect this information on an ongoing
basis so you have it handy when you are asked to justify your program. It is
hard-to-impossible to collect in a hurry at the last minute.Best,

Andrew Gent
ajgent@yahoo.com
(603) 888-0370










Bill Ives <iveswilliam@...>
 

I agree with Andrew on the time savings ROI - It is meaningless. It does not go to the bottom line. What are the people going to do with the time savings? leave early?  It is more effective to tie the performance improvement to bottom line measures such as increased productivity on factor x or less money spent on factor Y.  The time savings ROI has been around for a long time and given KM a bad name.  I am surprised to see it still being used.  Sorry to not agree but this hits a hot button for me (it has been around for so long) and I am a big believer that real measurable savings can be found from KM or enterprise search. Finding it is a good test that we are employing the initiatives to address business issues.   Bill


On Aug 14, 2008, at 12:25 AM, Andrew Gent wrote:


Yao and Bill,

With improved enterprise search ... knowledge workers can reduce the amount of the time spent on search ... a saving 10% of the time (1 hour per week) can be translated into potential saving of $1,425 a year per worker.

I am going to play devil's advocate here. Not because I don't like your arguments. I have used almost the exact same argument myself. But I have also had the following discussions with senior business managers. The conversation goes something like this:

  • We just spent $XXX thousand dollars last year on a new search engine. Where are the savings from that improvement?
  • 1 hour a week is 2.5% of a 40hr week. Show me where I am getting a 2.5% increase in production or performance? (Not theory but actual improvements in the bottom line.)
  • "... a saving of 3,000 x $1,425 = $4,275,000" $4 million return on an investment of what? -- say 4 people for a year, approximately $500K -- that's an ROI of 700% in one year. Who are you kidding? Real ROI for things like SAP are 200-400% over a 6 year period, not even breaking even for 2-3 years. I don't believe it.
Now having said all that, it occurs to me that there are two types of conversation like this. The justify-your-program discussion with your direct manager and the justify-your-expenditure discussion with upper management. What I have just described is the latter, which is why justifying KM programs is so difficult. Saved time and unmeasurable performance improvements simply don't go over in budget discussions.

However, for justifying your program to your direct manager, the argument concerning saved time is important, because usually that person is also managing the people whose performance you will impact. So I would suggest three things for that discussion:
  • Keep the argument about saving time. This has meaning to that manager (since they directly manage employees such as yourself)
  • Don't bother getting into the theoretical monetary savings, because it is meaningless. That manager is not going to see that money, so it is a waste of time.
  • As Stan has suggested before, use multiple arguments. Argue the saved time. Also provide supporting evidence -- emails, forum postings, performance review input -- from people who say your program has helped them save time or be more effective. It is best to  collect this information on an ongoing basis so you have it handy when you are asked to justify your program. It is hard-to-impossible to collect in a hurry at the last minute.
Best,

Andrew Gent
ajgent@yahoo.com
(603) 888-0370


 




Anderson, Andrew (Tac Anderson) <tacanderson@...>
 

I have always enjoyed the debates here in this group but have seldom joined in since I’m more of a KM hobbyist than professional.

 

I’m going to take Bill’s approach and be the contrarian here. Using cost savings as a justification is a 0 sum game.  It becomes a race to the bottom and ultimately is as quantifiable as marketing. That is to say, not impossible but very difficult.

 

Enterprise search sucks. Even Googlers think their intranet search sucks. Internal search just doesn’t scale yet at the middle of the curve. It’s handling too much info for basic search and not enough for Web type search.

 

The real advantages of KM, IMO, is the expansive nature of the learning.  It’s about innovation and the ability to connecting the connectors.  What you should be focusing on is the quality of the information people are co-creating, sharing and discovering.  You should be focusing on the ability for people to find and discover other people across the organization based not on roles and job descriptions but on the knowledge they have and are willing to share.

 

I would look at things like # of non core team participants, cross collaboration, etc. Just like the challenges I face in Marketing if the measurements aren’t set up in advance they’re meaningless afterwards.  You need to be able to look at your power users and measure how their productivity has increased, because collectively, for every hour your power users save finding something your laggards will waste.

 

Management raises the level of productivity of an organization through hiring, motivating and firing the right people. KM raises the productivity of the individual and at best the work team.

 

I guess what I’m saying (speaking from an outsiders perspective) is that the benefits of KM are very difficult to measure in aggregate unless a base line is established in advance and every stakeholder buys off on KPI’s. KM is better measured on a granular level.

 

Tac Anderson
IPG WW Sales and Services
Web - CRM – Metrics
tacanderson@...
www.hp.com/blogs/marketing  

 

From: sikmleaders@... [mailto:sikmleaders@...] On Behalf Of Bill Ives
Sent: Thursday, August 14, 2008 7:04 AM
To: sikmleaders@...
Subject: Re: [sikmleaders] Weighing the benefits of productivity improvements

 

I agree with Andrew on the time savings ROI - It is meaningless. It does not go to the bottom line. What are the people going to do with the time savings? leave early?  It is more effective to tie the performance improvement to bottom line measures such as increased productivity on factor x or less money spent on factor Y.  The time savings ROI has been around for a long time and given KM a bad name.  I am surprised to see it still being used.  Sorry to not agree but this hits a hot button for me (it has been around for so long) and I am a big believer that real measurable savings can be found from KM or enterprise search. Finding it is a good test that we are employing the initiatives to address business issues.   Bill

 

 

On Aug 14, 2008, at 12:25 AM, Andrew Gent wrote:



 

Yao and Bill,

With improved enterprise search ... knowledge workers can reduce the amount of the time spent on search ... a saving 10% of the time (1 hour per week) can be translated into potential saving of $1,425 a year per worker.


I am going to play devil's advocate here. Not because I don't like your arguments. I have used almost the exact same argument myself. But I have also had the following discussions with senior business managers. The conversation goes something like this:

  • We just spent $XXX thousand dollars last year on a new search engine. Where are the savings from that improvement?
  • 1 hour a week is 2.5% of a 40hr week. Show me where I am getting a 2.5% increase in production or performance? (Not theory but actual improvements in the bottom line.)
  • "... a saving of 3,000 x $1,425 = $4,275,000" $4 million return on an investment of what? -- say 4 people for a year, approximately $500K -- that's an ROI of 700% in one year. Who are you kidding? Real ROI for things like SAP are 200-400% over a 6 year period, not even breaking even for 2-3 years. I don't believe it.

Now having said all that, it occurs to me that there are two types of conversation like this. The justify-your-program discussion with your direct manager and the justify-your-expenditure discussion with upper management. What I have just described is the latter, which is why justifying KM programs is so difficult. Saved time and unmeasurable performance improvements simply don't go over in budget discussions.

However, for justifying your program to your direct manager, the argument concerning saved time is important, because usually that person is also managing the people whose performance you will impact. So I would suggest three things for that discussion:

  • Keep the argument about saving time. This has meaning to that manager (since they directly manage employees such as yourself)
  • Don't bother getting in to the theoretical monetary savings, because it is meaningless. That manager is not going to see that money, so it is a waste of time.
  • As Stan has suggested before, use multiple arguments. Argue the saved time. Also provide supporting evidence -- emails, forum postings, performance review input -- from people who say your program has helped them save time or be more effective. It is best to  collect this information on an ongoing basis so you have it handy when you are asked to justify your program. It is hard-to-impossible to collect in a hurry at the last minute.

Best,

Andrew Gent
ajgent@...
(603) 888-0370

 

 

 

 


David Snowden <snowded@...>
 

I'd follow the general theme here.   Senior managers have seen too many promises of cost and time savings over the years to take them seriously unless they are very concrete.
I would focus instead on creating multiple small high impact KM inspired projects which link directly to current intractable business or organisational issues.  Then you can do a cost/benefit per project and as importantly can create anecdotal evidence to support your case



Dave Snowden
Founder & Chief Scientific Officer
Cognitive Edge Pte Ltd

Now blogging at www.cognitive-edge.com


On 14 Aug 2008, at 14:52, Anderson, Andrew (Tac Anderson) wrote:


I have always enjoyed the debates here in this group but have seldom joined in since I’m more of a KM hobbyist than professional.

 

I’m going to take Bill’s approach and be the contrarian here. Using cost savings as a justification is a 0 sum game.  It becomes a race to the bottom and ultimately is as quantifiable as marketing. That is to say, not impossible but very difficult.

 

Enterprise search sucks. Even Googlers think their intranet search sucks. Internal search just doesn’t scale yet at the middle of the curve. It’s handling too much info for basic search and not enough for Web type search.

 

The real advantages of KM, IMO, is the expansive nature of the learning.  It’s about innovation and the ability to connecting the connectors.  What you should be focusing on is the quality of the information people are co-creating, sharing and discovering.  You should be focusing on the ability for people to find and discover other people across the organization based not on roles and job descriptions but on the knowledge they have and are willing to share.

 

I would look at things like # of non core team participants, cross collaboration, etc. Just like the challenges I face in Marketing if the measurements aren’t set up in advance they’re meaningless afterwards.  You need to be able to look at your power users and measure how their productivity has increased, because collectively, for every hour your power users save finding something your laggards will waste.

 

Management raises the level of productivity of an organization through hiring, motivating and firing the right people. KM raises the productivity of the individual and at best the work team.

 

I guess what I’m saying (speaking from an outsiders perspective) is that the benefits of KM are very difficult to measure in aggregate unless a base line is established in advance and every stakeholder buys off on KPI’s. KM is better measured on a granular level.

 

Tac Anderson 
IPG WW Sales and Services 
Web - CRM – Metrics 
tacanderson@hp.com 
www.hp.com/blogs/marketing  

 

From: sikmleaders@yahoogroups.com [mailto:sikmleaders@yahoogroups.com]On Behalf Of Bill Ives
Sent: Thursday, August 14, 2008 7:04 AM
To: sikmleaders@yahoogroups.com
Subject: Re: [sikmleaders] Weighing the benefits of productivity improvements

 

I agree with Andrew on the time savings ROI - It is meaningless. It does not go to the bottom line. What are the people going to do with the time savings? leave early?  It is more effective to tie the performance improvement to bottom line measures such as increased productivity on factor x or less money spent on factor Y.  The time savings ROI has been around for a long time and given KM a bad name.  I am surprised to see it still being used.  Sorry to not agree but this hits a hot button for me (it has been around for so long) and I am a big believer that real measurable savings can be found from KM or enterprise search. Finding it is a good test that we are employing the initiatives to address business issues.   Bill

 

 

On Aug 14, 2008, at 12:25 AM, Andrew Gent wrote:



 

Yao and Bill,

With improved enterprise search ... knowledge workers can reduce the amount of the time spent on search ... a saving 10% of the time (1 hour per week) can be translated into potential saving of $1,425 a year per worker.


I am going to play devil's advocate here. Not because I don't like your arguments. I have used almost the exact same argument myself. But I have also had the following discussions with senior business managers. The conversation goes something like this:

  • We just spent $XXX thousand dollars last year on a new search engine. Where are the savings from that improvement?
  • 1 hour a week is 2.5% of a 40hr week. Show me where I am getting a 2.5% increase in production or performance? (Not theory but actual improvements in the bottom line.)
  • "... a saving of 3,000 x $1,425 = $4,275,000" $4 million return on an investment of what? -- say 4 people for a year, approximately $500K -- that's an ROI of 700% in one year. Who are you kidding? Real ROI for things like SAP are 200-400% over a 6 year period, not even breaking even for 2-3 years. I don't believe it.

Now having said all that, it occurs to me that there are two types of conversation like this. The justify-your-program discussion with your direct manager and the justify-your-expenditure discussion with upper management. What I have just described is the latter, which is why justifying KM programs is so difficult. Saved time and unmeasurable performance improvements simply don't go over in budget discussions.

However, for justifying your program to your direct manager, the argument concerning saved time is important, because usually that person is also managing the people whose performance you will impact. So I would suggest three things for that discussion:

  • Keep the argument about saving time. This has meaning to that manager (since they directly manage employees such as yourself)
  • Don't bother getting in to the theoretical monetary savings, because it is meaningless. That manager is not going to see that money, so it is a waste of time.
  • As Stan has suggested before, use multiple arguments. Argue the saved time. Also provide supporting evidence -- emails, forum postings, performance review input -- from people who say your program has helped them save time or be more effective. It is best to  collect this information on an ongoing basis so you have it handy when you are asked to justify your program. It is hard-to-impossible to collect in a hurry at the last minute.

Best,

Andrew Gent
ajgent@yahoo.com
(603) 888-0370

 

 

 

 




Yao Ge
 

Good point on search quality and effectiveness. We are carefully logging the search activities and I believe we will be able extract some indicators of value-adds of the tool. A few examples:
- Number of repeated users (users coming back to the search tool repeatedly if they like the results)
- Search activities over-time (good quality search have sustained usages)
- Number of zero hits returns and number of tries on similar queries for a given session (a signed a not able to provide relevant answer)
 
We can measure and analyze the logs. But I can not translate them to hard-savings.
 
-Yao



From: sikmleaders@... [mailto:sikmleaders@...] On Behalf Of sjagannath@...
Sent: Thursday, August 14, 2008 2:34 AM
To: sikmleaders@...
Subject: Re: [sikmleaders] Weighing the benefits of productivity improvements


Hi Yao,

This is a wonderful topic. Around an year time back, while I developed the business case for instituting an enterprise search (certainly as part of KM initiative) I used same argument. Presented similar numbers of savings/year. But, with this calculation we are probably assuming the search would always provide relevant results...
My experience, it does not in many cases!! If in case it does capturing the same is almost an impossible task. How many of us have contributed information for formulating case study or rated an relevant article? Guess it is very difficult to have that discipline built within an organization to provide feedback/rating.

Andrew, Your thought process is absolutely right that a small search initiative cannot quantify million dollars to the bottom line... wonderfully substantiated with ERP example consuming more time than just 1 - 2 years for showing benefits.

Hence, am sailing in the same boat as Yao & looking forward for some more meaningful metrics, where dependency on feedback is relatively less.

Regards,
Srinivas P Jagannath



Yao Ge
 

Dave,
 
Very good advice. Enterprise search and KM is such generic and fundamental that its impacts gets lost amount many other direct contributing factors. Beyond engaging early with high impact business projects. We are thinking about collecting anecdotal evidences through some focused survey questions.
 
-Yao


From: sikmleaders@... [mailto:sikmleaders@...] On Behalf Of David Snowden
Sent: Thursday, August 14, 2008 10:02 AM
To: sikmleaders@...
Subject: Re: [sikmleaders] Weighing the benefits of productivity improvements

I'd follow the general theme here.   Senior managers have seen too many promises of cost and time savings over the years to take them seriously unless they are very concrete.

I would focus instead on creating multiple small high impact KM inspired projects which link directly to current intractable business or organisational issues.  Then you can do a cost/benefit per project and as importantly can create anecdotal evidence to support your case



Dave Snowden
Founder & Chief Scientific Officer
Cognitive Edge Pte Ltd

Now blogging at www.cognitive-edge.com


On 14 Aug 2008, at 14:52, Anderson, Andrew (Tac Anderson) wrote:


I have always enjoyed the debates here in this group but have seldom joined in since I’m more of a KM hobbyist than professional.

I’m going to take Bill’s approach and be the contrarian here. Using cost savings as a justification is a 0 sum game.  It becomes a race to the bottom and ultimately is as quantifiable as marketing. That is to say, not impossible but very difficult.

Enterprise search sucks. Even Googlers think their intranet search sucks. Internal search just doesn’t scale yet at the middle of the curve. It’s handling too much info for basic search and not enough for Web type search.

The real advantages of KM, IMO, is the expansive nature of the learning.  It’s about innovation and the ability to connecting the connectors.  What you should be focusing on is the quality of the information people are co-creating, sharing and discovering.  You should be focusing on the ability for people to find and discover other people across the organization based not on roles and job descriptions but on the knowledge they have and are willing to share.

I would look at things like # of non core team participants, cross collaboration, etc. Just like the challenges I face in Marketing if the measurements aren’t set up in advance they’re meaningless afterwards.  You need to be able to look at your power users and measure how their productivity has increased, because collectively, for every hour your power users save finding something your laggards will waste.

Management raises the level of productivity of an organization through hiring, motivating and firing the right people. KM raises the productivity of the individual and at best the work team.

I guess what I’m saying (speaking from an outsiders perspective) is that the benefits of KM are very difficult to measure in aggregate unless a base line is established in advance and every stakeholder buys off on KPI’s. KM is better measured on a granular level.

Tac Anderson 
IPG WW Sales and Services 
Web - CRM – Metrics 
tacanderson@hp.com 
www.hp.com/blogs/marketing  

From: sikmleaders@yahoogroups.com [mailto:sikmleaders@yahoogroups.com]On Behalf Of Bill Ives
Sent: Thursday, August 14, 2008 7:04 AM
To: sikmleaders@yahoogroups.com
Subject: Re: [sikmleaders] Weighing the benefits of productivity improvements

I agree with Andrew on the time savings ROI - It is meaningless. It does not go to the bottom line. What are the people going to do with the time savings? leave early?  It is more effective to tie the performance improvement to bottom line measures such as increased productivity on factor x or less money spent on factor Y.  The time savings ROI has been around for a long time and given KM a bad name.  I am surprised to see it still being used.  Sorry to not agree but this hits a hot button for me (it has been around for so long) and I am a big believer that real measurable savings can be found from KM or enterprise search. Finding it is a good test that we are employing the initiatives to address business issues.   Bill

On Aug 14, 2008, at 12:25 AM, Andrew Gent wrote:



Yao and Bill,

With improved enterprise search ... knowledge workers can reduce the amount of the time spent on search ... a saving 10% of the time (1 hour per week) can be translated into potential saving of $1,425 a year per worker.


I am going to play devil's advocate here. Not because I don't like your arguments. I have used almost the exact same argument myself. But I have also had the following discussions with senior business managers. The conversation goes something like this:

  • We just spent $XXX thousand dollars last year on a new search engine. Where are the savings from that improvement?
  • 1 hour a week is 2.5% of a 40hr week. Show me where I am getting a 2.5% increase in production or performance? (Not theory but actual improvements in the bottom line.)
  • "... a saving of 3,000 x $1,425 = $4,275,000" $4 million return on an investment of what? -- say 4 people for a year, approximately $500K -- that's an ROI of 700% in one year. Who are you kidding? Real ROI for things like SAP are 200-400% over a 6 year period, not even breaking even for 2-3 years. I don't believe it.

Now having said all that, it occurs to me that there are two types of conversation like this. The justify-your-program discussion with your direct manager and the justify-your-expenditure discussion with upper management. What I have just described is the latter, which is why justifying KM programs is so difficult. Saved time and unmeasurable performance improvements simply don't go over in budget discussions.

However, for justifying your program to your direct manager, the argument concerning saved time is important, because usually that person is also managing the people whose performance you will impact. So I would suggest three things for that discussion:

  • Keep the argument about saving time. This has meaning to that manager (since they directly manage employees such as yourself)
  • Don't bother getting in to the theoretical monetary savings, because it is meaningless. That manager is not going to see that money, so it is a waste of time.
  • As Stan has suggested before, use multiple arguments. Argue the saved time. Also provide supporting evidence -- emails, forum postings, performance review input -- from people who say your program has helped them save time or be more effective. It is best to  collect this information on an ongoing basis so you have it handy when you are asked to justify your program. It is hard-to-impossible to collect in a hurry at the last minute.

Best,

Andrew Gent
ajgent@yahoo.com
(603) 888-0370




David Snowden <snowded@...>
 

Survey questions can be challenged but setting up a simple web site to gather anecdotes can help
The real value is when you can combine quantitative and qualitative data to measure impact (ie combine numbers with anecdotes).  I need to declare a commercial interest there however as that is one of the function of our software (stand at KM World if anyone is interested).  Happy to take that up off line if you want



Dave Snowden
Founder & Chief Scientific Officer
Cognitive Edge Pte Ltd

Now blogging at www.cognitive-edge.com


On 14 Aug 2008, at 15:24, Ge, Yao (Y.) wrote:


Dave,
 
Very good advice. Enterprise search and KM is such generic and fundamental that its impacts gets lost amount many other direct contributing factors. Beyond engaging early with high impact business projects. We are thinking about collecting anecdotal evidences through some focused survey questions.
 
-Yao


From: sikmleaders@yahoogroups.com [mailto:sikmleaders@yahoogroups.com] On Behalf Of David Snowden
Sent: Thursday, August 14, 2008 10:02 AM
To: sikmleaders@yahoogroups.com
Subject: Re: [sikmleaders] Weighing the benefits of productivity improvements

I'd follow the general theme here.   Senior managers have seen too many promises of cost and time savings over the years to take them seriously unless they are very concrete.

I would focus instead on creating multiple small high impact KM inspired projects which link directly to current intractable business or organisational issues.  Then you can do a cost/benefit per project and as importantly can create anecdotal evidence to support your case



Dave Snowden
Founder & Chief Scientific Officer
Cognitive Edge Pte Ltd

Now blogging at www.cognitive-edge.com


On 14 Aug 2008, at 14:52, Anderson, Andrew (Tac Anderson) wrote:


I have always enjoyed the debates here in this group but have seldom joined in since I’m more of a KM hobbyist than professional.

I’m going to take Bill’s approach and be the contrarian here. Using cost savings as a justification is a 0 sum game.  It becomes a race to the bottom and ultimately is as quantifiable as marketing. That is to say, not impossible but very difficult.

Enterprise search sucks. Even Googlers think their intranet search sucks. Internal search just doesn’t scale yet at the middle of the curve. It’s handling too much info for basic search and not enough for Web type search.

The real advantages of KM, IMO, is the expansive nature of the learning.  It’s about innovation and the ability to connecting the connectors.  What you should be focusing on is the quality of the information people are co-creating, sharing and discovering.  You should be focusing on the ability for people to find and discover other people across the organization based not on roles and job descriptions but on the knowledge they have and are willing to share.

I would look at things like # of non core team participants, cross collaboration, etc. Just like the challenges I face in Marketing if the measurements aren’t set up in advance they’re meaningless afterwards.  You need to be able to look at your power users and measure how their productivity has increased, because collectively, for every hour your power users save finding something your laggards will waste.

Management raises the level of productivity of an organization through hiring, motivating and firing the right people. KM raises the productivity of the individual and at best the work team.

I guess what I’m saying (speaking from an outsiders perspective) is that the benefits of KM are very difficult to measure in aggregate unless a base line is established in advance and every stakeholder buys off on KPI’s. KM is better measured on a granular level.

Tac Anderson 
IPG WW Sales and Services 
Web - CRM – Metrics 
tacanderson@hp.com 
www.hp.com/blogs/marketing  

From: sikmleaders@yahoogroups.com [mailto:sikmleaders@yahoogroups.com]On Behalf Of Bill Ives
Sent: Thursday, August 14, 2008 7:04 AM
To: sikmleaders@yahoogroups.com
Subject: Re: [sikmleaders] Weighing the benefits of productivity improvements

I agree with Andrew on the time savings ROI - It is meaningless. It does not go to the bottom line. What are the people going to do with the time savings? leave early?  It is more effective to tie the performance improvement to bottom line measures such as increased productivity on factor x or less money spent on factor Y.  The time savings ROI has been around for a long time and given KM a bad name.  I am surprised to see it still being used.  Sorry to not agree but this hits a hot button for me (it has been around for so long) and I am a big believer that real measurable savings can be found from KM or enterprise search. Finding it is a good test that we are employing the initiatives to address business issues.   Bill

On Aug 14, 2008, at 12:25 AM, Andrew Gent wrote:



Yao and Bill,

With improved enterprise search ... knowledge workers can reduce the amount of the time spent on search ... a saving 10% of the time (1 hour per week) can be translated into potential saving of $1,425 a year per worker.


I am going to play devil's advocate here. Not because I don't like your arguments. I have used almost the exact same argument myself. But I have also had the following discussions with senior business managers. The conversation goes something like this:

  • We just spent $XXX thousand dollars last year on a new search engine. Where are the savings from that improvement?
  • 1 hour a week is 2.5% of a 40hr week. Show me where I am getting a 2.5% increase in production or performance? (Not theory but actual improvements in the bottom line.)
  • "... a saving of 3,000 x $1,425 = $4,275,000" $4 million return on an investment of what? -- say 4 people for a year, approximately $500K -- that's an ROI of 700% in one year. Who are you kidding? Real ROI for things like SAP are 200-400% over a 6 year period, not even breaking even for 2-3 years. I don't believe it.

Now having said all that, it occurs to me that there are two types of conversation like this. The justify-your-program discussion with your direct manager and the justify-your-expenditure discussion with upper management. What I have just described is the latter, which is why justifying KM programs is so difficult. Saved time and unmeasurable performance improvements simply don't go over in budget discussions.

However, for justifying your program to your direct manager, the argument concerning saved time is important, because usually that person is also managing the people whose performance you will impact. So I would suggest three things for that discussion:

  • Keep the argument about saving time. This has meaning to that manager (since they directly manage employees such as yourself)
  • Don't bother getting in to the theoretical monetary savings, because it is meaningless. That manager is not going to see that money, so it is a waste of time.
  • As Stan has suggested before, use multiple arguments. Argue the saved time. Also provide supporting evidence -- emails, forum postings, performance review input -- from people who say your program has helped them save time or be more effective. It is best to  collect this information on an ongoing basis so you have it handy when you are asked to justify your program. It is hard-to-impossible to collect in a hurry at the last minute.

Best,

Andrew Gent
ajgent@yahoo.com
(603) 888-0370







sswarup44 <sswarup44@...>
 

I am enjoying the debate so far.

I suggest a broader approach for the value that KM can provide to an
organization.

One way is to use the Balance Score Card Approach. Leadership will be
a lot more receptive to these macro factors:

KM Macro metrics:
Finance Metrics: Increased market share, sales
Customer metrics: On-time delivery
Internal Processes metrics: Cycle time, unit costs,
Innovation and Learning metrics: Number of new products introduced,
time to develop next generation of products

Micro factors are also worth considering.
KM micro metrics

KM Micro metrics
KM metrics from Finance and Accounting perspective: Reduced unit
costs
KM metrics from Human Resource Accounting perspective: Reduced
employee turnover, since employees feel valued for their knowledge
KM metrics from IT Perspective: Quality of search results
KM metrics from Organizational Development perspective: Testimonials
from individuals who have benefited from Lessons Learned data base,
or Communities of Practice

During my 7 years with KM application called Best Practice
Replication at Ford Motor Company, we profusely used both the micro
as well macro metrics.

Chapter 7 of "What They Didn't Tell You About Knowledge Management"
by Jay Liebowitz can further help develop KM metrics.

Delighted to have an off line discussion on value metrics.

Sanjay Swarup
Program Manager KM
ManTech International
301.866.4315


--- In sikmleaders@yahoogroups.com, David Snowden <snowded@...> wrote:

Survey questions can be challenged but setting up a simple web site
to
gather anecdotes can help
The real value is when you can combine quantitative and
qualitative
data to measure impact (ie combine numbers with anecdotes). I need
to
declare a commercial interest there however as that is one of the
function of our software (stand at KM World if anyone is
interested).
Happy to take that up off line if you want



Dave Snowden
Founder & Chief Scientific Officer
Cognitive Edge Pte Ltd

Now blogging at www.cognitive-edge.com


On 14 Aug 2008, at 15:24, Ge, Yao (Y.) wrote:


Dave,

Very good advice. Enterprise search and KM is such generic and
fundamental that its impacts gets lost amount many other direct
contributing factors. Beyond engaging early with high impact
business projects. We are thinking about collecting anecdotal
evidences through some focused survey questions.

-Yao

From: sikmleaders@yahoogroups.com
[mailto:sikmleaders@yahoogroups.com] On Behalf Of David Snowden
Sent: Thursday, August 14, 2008 10:02 AM
To: sikmleaders@yahoogroups.com
Subject: Re: [sikmleaders] Weighing the benefits of productivity
improvements

I'd follow the general theme here. Senior managers have seen
too
many promises of cost and time savings over the years to take
them
seriously unless they are very concrete.

I would focus instead on creating multiple small high impact KM
inspired projects which link directly to current intractable
business or organisational issues. Then you can do a
cost/benefit
per project and as importantly can create anecdotal evidence to
support your case



Dave Snowden
Founder & Chief Scientific Officer
Cognitive Edge Pte Ltd

Now blogging at www.cognitive-edge.com


On 14 Aug 2008, at 14:52, Anderson, Andrew (Tac Anderson) wrote:


I have always enjoyed the debates here in this group but have
seldom joined in since I'm more of a KM hobbyist than
professional.


I'm going to take Bill's approach and be the contrarian here.
Using
cost savings as a justification is a 0 sum game. It becomes a
race
to the bottom and ultimately is as quantifiable as marketing.
That
is to say, not impossible but very difficult.


Enterprise search sucks. Even Googlers think their intranet
search
sucks. Internal search just doesn't scale yet at the middle of
the
curve. It's handling too much info for basic search and not
enough
for Web type search.


The real advantages of KM, IMO, is the expansive nature of the
learning. It's about innovation and the ability to connecting
the
connectors. What you should be focusing on is the quality of
the
information people are co-creating, sharing and discovering.
You
should be focusing on the ability for people to find and
discover
other people across the organization based not on roles and job
descriptions but on the knowledge they have and are willing to
share.


I would look at things like # of non core team participants,
cross
collaboration, etc. Just like the challenges I face in Marketing
if
the measurements aren't set up in advance they're meaningless
afterwards. You need to be able to look at your power users
and
measure how their productivity has increased, because
collectively,
for every hour your power users save finding something your
laggards will waste.


Management raises the level of productivity of an organization
through hiring, motivating and firing the right people. KM
raises
the productivity of the individual and at best the work team.


I guess what I'm saying (speaking from an outsiders perspective)
is
that the benefits of KM are very difficult to measure in
aggregate
unless a base line is established in advance and every
stakeholder
buys off on KPI's. KM is better measured on a granular level.


Tac Anderson
IPG WW Sales and Services
Web - CRM – Metrics
tacanderson@...
www.hp.com/blogs/marketing


From: sikmleaders@yahoogroups.com
[mailto:sikmleaders@yahoogroups.com]On Behalf Of Bill Ives
Sent: Thursday, August 14, 2008 7:04 AM
To: sikmleaders@yahoogroups.com
Subject: Re: [sikmleaders] Weighing the benefits of
productivity
improvements


I agree with Andrew on the time savings ROI - It is meaningless.
It
does not go to the bottom line. What are the people going to do
with the time savings? leave early? It is more effective to
tie
the performance improvement to bottom line measures such as
increased productivity on factor x or less money spent on
factor
Y. The time savings ROI has been around for a long time and
given
KM a bad name. I am surprised to see it still being used.
Sorry
to not agree but this hits a hot button for me (it has been
around
for so long) and I am a big believer that real measurable
savings
can be found from KM or enterprise search. Finding it is a good
test that we are employing the initiatives to address business
issues. Bill



On Aug 14, 2008, at 12:25 AM, Andrew Gent wrote:





Yao and Bill,

With improved enterprise search ... knowledge workers can
reduce
the amount of the time spent on search ... a saving 10% of the
time
(1 hour per week) can be translated into potential saving of
$1,425
a year per worker.


I am going to play devil's advocate here. Not because I don't
like
your arguments. I have used almost the exact same argument
myself.
But I have also had the following discussions with senior
business
managers. The conversation goes something like this:

We just spent $XXX thousand dollars last year on a new search
engine. Where are the savings from that improvement?
1 hour a week is 2.5% of a 40hr week. Show me where I am getting
a
2.5% increase in production or performance? (Not theory but
actual
improvements in the bottom line.)
"... a saving of 3,000 x $1,425 = $4,275,000" $4 million return
on
an investment of what? -- say 4 people for a year,
approximately
$500K -- that's an ROI of 700% in one year. Who are you
kidding?
Real ROI for things like SAP are 200-400% over a 6 year period,
not
even breaking even for 2-3 years. I don't believe it.
Now having said all that, it occurs to me that there are two
types
of conversation like this. The justify-your-program discussion
with
your direct manager and the justify-your-expenditure discussion
with upper management. What I have just described is the
latter,
which is why justifying KM programs is so difficult. Saved time
and
unmeasurable performance improvements simply don't go over in
budget discussions.

However, for justifying your program to your direct manager,
the
argument concerning saved time is important, because usually
that
person is also managing the people whose performance you will
impact. So I would suggest three things for that discussion:

Keep the argument about saving time. This has meaning to that
manager (since they directly manage employees such as yourself)
Don't bother getting in to the theoretical monetary savings,
because it is meaningless. That manager is not going to see
that
money, so it is a waste of time.
As Stan has suggested before, use multiple arguments. Argue the
saved time. Also provide supporting evidence -- emails, forum
postings, performance review input -- from people who say your
program has helped them save time or be more effective. It is
best
to collect this information on an ongoing basis so you have it
handy when you are asked to justify your program. It is hard-to-
impossible to collect in a hurry at the last minute.
Best,

Andrew Gent
ajgent@...
(603) 888-0370










john_mcquary <john.mcquary@...>
 

My experience was that different executives required different
messages and the value / cost savings message needs to change as your
KM program matures. Our best success, and what we use today, is
sharing success stories coupled with usage statistics. The stories
show tangible examples of cost savings or customer value add, and the
statistics let the audience extrapolate their interpretation of the
enterprise-wide benefits. We don't try to correlate usage statistics
directly to cost savings. Usage statistics measure usage. They
don't measure value.

Early in our KM journey, we convinced a few strictly cost focused
executives that by merging everything into one enterprise-wide KM
tool, our annual cost was less than the annual costs for separate
practice and procedure databases, lessons learned database,
innovation database, discussion forums, expertise locators, etc.
This is a lousy way to prove value for KM, but if the executive is
cost focused, it is a convincing argument.

Another early method to show benefits was to create a spreadsheet of
early success examples with dollar amounts. We extrapolated how many
similar examples we expected in the next year, and added a column
called "Your Choice" where the executive could fill in what they
believed would happen. Some executives would take exception to some
of our examples, and zero out or reduce that item, but the bottom
line was always positive, and it was their number. This was time
consuming and we only did it with a few executives, but always had
good results.

One note on success stories... An executive pulled me away once, and
said, "The group executives are tired of hearing the XYZ story. You
need new stories." Funny thing is we only shared the XYZ story
once. It was picked up and repeated by others. But, this shows the
need to always be prepared with several examples of success, and the
power of storytelling (that is another thread).

A final point on value vs. cost savings. These are different
measures, especially when you consider customer value. Executives,
particularly if they have a Sales background, are often easier to be
convinced on the value of KM when your examples take into account the
value the customer received.


vs_shenoy <vs_shenoy@...>
 

Yao,

It does seem from your post that you are in the process of
understanding Enterprise Search for a particular application. I've
seen a similar situation at a previous employer and can't say I was
seeing it the right way because of a few assumptions made. I've summed
these up as a debate and wondered how to justify benefits:

1. Discovery vs. Security - In a truly collaborative environment, all
content would be findable. While there is a savings in terms of
finding expertise, isn't there an equal loss of time spent in debating
how to share content because the content could be sensitive (audience,
time etc). While this could be the exception rather than the norm,
content may or may not be "made" available in terms of being really
useful to all users.

2. Quantity vs. Quality - It is true when metrics are introduced
around producing more content, the quality does take a side seat.
Also, most content creators are not involved in the discovery
discussion. In such an situation, one has to make the assumption that
all content is created equally. Can we truly say that a less detailed
article that has a 1000 hits is more useful than a detailed article
that has 600 hits assuming they are tagged the same way? Not all users
are intent on truly revising their search beyond a few tries.

I've worked in multiple environments often in heavy content creation
environments and haven't been convinced of simple metrics as being the
best measure of content's usefulness. I also would like to know what
the best approach was in large ECM environments and the best way to
leverage Search, rather than examine the processes involved in the
content life cycle as a start to evaluating the right search strategy.

Sorry if I complicated the discussion.

Thanks,
Vinod

--- In sikmleaders@yahoogroups.com, "Ge, Yao (Y.)" <yge@...> wrote:

Good point on search quality and effectiveness. We are carefully logging
the search activities and I believe we will be able extract some
indicators of value-adds of the tool. A few examples:
- Number of repeated users (users coming back to the search tool
repeatedly if they like the results)
- Search activities over-time (good quality search have sustained
usages)
- Number of zero hits returns and number of tries on similar queries for
a given session (a signed a not able to provide relevant answer)

We can measure and analyze the logs. But I can not translate them to
hard-savings.

-Yao

________________________________

From: sikmleaders@yahoogroups.com [mailto:sikmleaders@yahoogroups.com]
On Behalf Of sjagannath@...
Sent: Thursday, August 14, 2008 2:34 AM
To: sikmleaders@yahoogroups.com
Subject: Re: [sikmleaders] Weighing the benefits of productivity
improvements




Hi Yao,

This is a wonderful topic. Around an year time back, while I developed
the business case for instituting an enterprise search (certainly as
part of KM initiative) I used same argument. Presented similar numbers
of savings/year. But, with this calculation we are probably assuming the
search would always provide relevant results...
My experience, it does not in many cases!! If in case it does capturing
the same is almost an impossible task. How many of us have contributed
information for formulating case study or rated an relevant article?
Guess it is very difficult to have that discipline built within an
organization to provide feedback/rating.

Andrew, Your thought process is absolutely right that a small search
initiative cannot quantify million dollars to the bottom line...
wonderfully substantiated with ERP example consuming more time than just
1 - 2 years for showing benefits.

Hence, am sailing in the same boat as Yao & looking forward for some
more meaningful metrics, where dependency on feedback is relatively
less.

Regards,
Srinivas P Jagannath


Yao Ge
 

Vinod,
 
Our search application augment Google Search Appliance but provide a couple features that address the issues you brought up here.
 
On "Discovery vs. Security", we recognize not all documents are created equal. Our corporate information policy requires user to have legitimate business needs to access sensitive information. This "needs-to-know" has become a barrier for information discovery as people "don't know what they don't know". Our search application allows custom display of search results on documents users not yet have access to. We can mask the more detailed information such as snippets and content cache yet inform users the existences of the information in the repositories. Of course there are some contents data source owner don't even want to acknowledge the existence of records to non-users. But for many case, the data owners are willing to share but don't know how advertise the existence of information without breaking the security policy. Our solution did that with custom layer of code runs on-top-of GSA.
 
On "Quantity vs. Quality" - The search relevancy score in GSA is primarily computed based on document content (words) and the keywords user entered. The Enterprise PageRank is a no-factor or a weak factor due to the lack of link structure in the enterprise information space. The number of hits (not links) to the article is not a factor for search algorithm (at least for Google). In our case, we augemented the search interface by allowing user to bookmark and tag the results. These user actions on search results can act as "votes" to the search results and as indicators to search quality. However based our experience on the few months after the launch, very few users actually bother to take the time to tag results. But I don't believe it is a indication of poor search returns...
 
Enterprise Search is a common struggles for many companies. We believe it can be improved if you understand the nature of the enterprise knowledge stores and information access barriers. I can share some more on our experiences if many of you are interested.
 
-Yao


From: sikmleaders@... [mailto:sikmleaders@...] On Behalf Of vs_shenoy
Sent: Thursday, August 14, 2008 3:41 PM
To: sikmleaders@...
Subject: [sikmleaders] Re: Weighing the benefits of productivity improvements

Yao,

It does seem from your post that you are in the process of
understanding Enterprise Search for a particular application. I've
seen a similar situation at a previous employer and can't say I was
seeing it the right way because of a few assumptions made. I've summed
these up as a debate and wondered how to justify benefits:

1. Discovery vs. Security - In a truly collaborative environment, all
content would be findable. While there is a savings in terms of
finding expertise, isn't there an equal loss of time spent in debating
how to share content because the content could be sensitive (audience,
time etc). While this could be the exception rather than the norm,
content may or may not be "made" available in terms of being really
useful to all users.

2. Quantity vs. Quality - It is true when metrics are introduced
around producing more content, the quality does take a side seat.
Also, most content creators are not involved in the discovery
discussion. In such an situation, one has to make the assumption that
all content is created equally. Can we truly say that a less detailed
article that has a 1000 hits is more useful than a detailed article
that has 600 hits assuming they are tagged the same way? Not all users
are intent on truly revising their search beyond a few tries.

I've worked in multiple environments often in heavy content creation
environments and haven't been convinced of simple metrics as being the
best measure of content's usefulness. I also would like to know what
the best approach was in large ECM environments and the best way to
leverage Search, rather than examine the processes involved in the
content life cycle as a start to evaluating the right search strategy.

Sorry if I complicated the discussion.

Thanks,
Vinod


--- In sikmleaders@yahoogroups.com, "Ge, Yao (Y.)" wrote:
>
> Good point on search quality and effectiveness. We are carefully logging
> the search activities and I believe we will be able extract some
> indicators of value-adds of the tool. A few examples:
> - Number of repeated users (users coming back to the search tool
> repeatedly if they like the results)
> - Search activities over-time (good quality search have sustained
> usages)
> - Number of zero hits returns and number of tries on similar queries for
> a given session (a signed a not able to provide relevant answer)
>
> We can measure and analyze the logs. But I can not translate them to
> hard-savings.
>
> -Yao
>
> ________________________________
>
> From: sikmleaders@yahoogroups.com [mailto:sikmleaders@yahoogroups.com]
> On Behalf Of sjagannath@...
> Sent: Thursday, August 14, 2008 2:34 AM
> To: sikmleaders@yahoogroups.com
> Subject: Re: [sikmleaders] Weighing the benefits of productivity
> improvements
>
>
>
>
> Hi Yao,
>
> This is a wonderful topic. Around an year time back, while I developed
> the business case for instituting an enterprise search (certainly as
> part of KM initiative) I used same argument. Presented similar numbers
> of savings/year. But, with this calculation we are probably assuming the
> search would always provide relevant results...
> My experience, it does not in many cases!! If in case it does capturing
> the same is almost an impossible task. How many of us have contributed
> information for formulating case study or rated an relevant article?
> Guess it is very difficult to have that discipline built within an
> organization to provide feedback/rating.
>
> Andrew, Your thought process is absolutely right that a small search
> initiative cannot quantify million dollars to the bottom line...
> wonderfully substantiated with ERP example consuming more time than just
> 1 - 2 years for showing benefits.
>
> Hence, am sailing in the same boat as Yao & looking forward for some
> more meaningful metrics, where dependency on feedback is relatively
> less.
>
> Regards,
> Srinivas P Jagannath
>


Michael Brown
 

Very interesting debate. May I suggest another approach to demonstrate KM enabling characteristics to engerprise performance? We are researhing, designing, and establishing evalutation criteria for a KM system that seeks the user vs the user seeking the system. Predicitive information/KM support, in context, to the right person at the right time (point of execution) is the thesis of our effort.
 
This approach may not work but wanted to share the thought with the group. 
 

Michael L. Brown, Founder & CEO

Vice Chair IEEE LTSC
SkillsNET Enterprises
3295 North Hwy 77
Waxahachie, Texas 75165



O-972-923-2950
c-972-987-4599


SkillObjects: The OBJECT of work analysis


----- Original Message ----
From: john_mcquary
To: sikmleaders@...
Sent: Thursday, August 14, 2008 10:36:01 AM
Subject: [sikmleaders] Re: Weighing the benefits of productivity improvements

My experience was that different executives required different
messages and the value / cost savings message needs to change as your
KM program matures. Our best success, and what we use today, is
sharing success stories coupled with usage statistics. The stories
show tangible examples of cost savings or customer value add, and the
statistics let the audience extrapolate their interpretation of the
enterprise-wide benefits. We don't try to correlate usage statistics
directly to cost savings. Usage statistics measure usage. They
don't measure value.

Early in our KM journey, we convinced a few strictly cost focused
executives that by merging everything into one enterprise-wide KM
tool, our annual cost was less than the annual costs for separate
practice and procedure databases, lessons learned database,
innovation database, discussion forums, expertise locators, etc.
This is a lousy way to prove value for KM, but if the executive is
cost focused, it is a convincing argument.

Another early method to show benefits was to create a spreadsheet of
early success examples with dollar amounts. We extrapolated how many
similar examples we expected in the next year, and added a column
called "Your Choice" where the executive could fill in what they
believed would happen. Some executives would take exception to some
of our examples, and zero out or reduce that item, but the bottom
line was always positive, and it was their number. This was time
consuming and we only did it with a few executives, but always had
good results.

One note on success stories... An executive pulled me away once, and
said, "The group executives are tired of hearing the XYZ story. You
need new stories." Funny thing is we only shared the XYZ story
once. It was picked up and repeated by others. But, this shows the
need to always be prepared with several examples of success, and the
power of storytelling (that is another thread).

A final point on value vs. cost savings. These are different
measures, especially when you consider customer value. Executives,
particularly if they have a Sales background, are often easier to be
convinced on the value of KM when your examples take into account the
value the customer received.


Matt Moore <laalgadger@...>
 

Hello,

I think there is no one way to justify KM initiatives (and I'm a bit dismayed that we seem to have reduced KM to "implementing search").

How you justify your KM initiative depends on: i. what you are doing & ii. who you have to persuade.

I think you have to remember that these kinds of justifications are a form of quantitative rhetoric. If your audience is concerned with reductions in time then go for that. However at the CEO level they probably have a different set of concerns. There you are going to have to show how KM has supported the strategic objectives of the organisation. Or at the very least not hindered them.

Some observations here: http://www.slideshare.net/engineerswithoutfears/showing-the-value-of-km/

Aristotle reminds us that rhetoric depends not just on our logos (i.e. the plausibility of our arguments and the awesomeness of our spreadsheets) but also pathos (the stories people tell about our KM activities) and ethos (our standing in the eyes of our audience). Let's not forget these.

Cheers,

Matt


Bill Dixon
 

Yao and Colleagues,
 
To close the loop on this, I have attached our initial draft of "A General Business Case for KM" we'll be using as a starting point for discussion with selected audiences.  What resonates most often with my management teams is KM as a mechanism by which we mediate risk. The other attributes of KM are nice, but saving a few minutes when looking for something doesn't directly impact the bottom line.  I've heard from more than one leader, increasing the time available for coffee breaks does not make a strong case for investing real dollars in any initative.
 
Yao will recongize his original post is woven throughout this document.  And Matt Moore will recongize his reference to "Value Drivers".  I appreciate having the opportunity to build on everyones ideas.
 
Regards,
 
Bill Dixon
Ernst & Young, LLP
 
 

----- Original Message -----
From: Bill Dixon
Sent: Wednesday, August 13, 2008 8:59 PM
Subject: Re: [sikmleaders] Weighing the benefits of productivity improvements

Hello Yao,
 
I was asked a similar question this morning and have also been formulating a response. 
 
I like the general approach you have taken in the draft of your article.  Without getting in too much detail, you do a good job describing one of the potential benefits of KM.  In our environment, KM can also help mitigate risk.  By leveraging not only best practices but by avoiding worst practices, we avoid making potentially costly mistakes over an over again.  The impact on the business can be significant.
 
I am adapting Yelden and Albers (2004) (http://www.tlainc.com/articl69.htm) approach to establishing business cases for KM initiatives to our environment.  They advocate the following framework for developing business cases around KM initiatives:,

1.      Strategy assessment

2.      Knowledge audit

3.      Knowledge and business strategy alignment

4.      Opportunity identification

5.      Value, business benefits, and evaluation

6.      Risk reduction techniques

"In order to make a successful justification, it is necessary to clearly identify all the options available with the associated risks involved with each choice.  Furthermore, it is required to first identify and separate the benefits of a given initiative, then determine its value to the firm, before proceeding to infer an associated cost and expected return for undertaking the effort.  Clearly delineating the expected hard and soft benefits of each aspect of the initiative will greatly aid in effectively justifying its need."

I will send you a copy when I am finished. 

Thanks for sharing your draft.

Regards,

Bill Dixon,

Ernst & Young, LLP

 
 
----- Original Message -----
Sent: Wednesday, August 13, 2008 10:16 AM
Subject: [sikmleaders] Weighing the benefits of productivity improvements

Hello KM peers, I am once again asked to justify what I am doing by associating my work in KM area to impact to business bottom lines such as hard-savings. This type of request must be common across KM and other collaboration related areas. It is a fair question but quite difficult to articulate.
 
Below is a short article I just wrote on how to measure the benefits of productivity improvements and I would like to hear what this group thinks on this topic (not only on workplace efficiency but also KM/Collaboration/Innovation etc). Thanks in advance!
 

One of the most significant challenges in today's knowledge work is to deal with explosion of "Information Overload". The saving of workers time in searching, accessing, and filtering the vast amount of information scattered across personal desktop, shared workspace, and enterprise repositories can be measured and potentially translated to hard-savings in workspace.

In the age of industry evaluation, we measure productivities by yields and through-put of the workers on the assembly lines. It is a bit harder to measure the productivities in the information age as knowledge work performed today as less mechanical and procedural in nature. Some type of work such as research and development can be measured by number of patents, number of design iterations, and product development lead times.

As an example, according to an IDC study, a knowledge work spends around an average of 9.5 hours per week searching for information. Based on the average salary plus benefits of $60,000/year from Department of Labor (2005), this would translate to $14,251.90 a year per worker. With improved enterprise search, there are two possible out-comes depends on the nature of the work. The knowledge workers can reduced the amount of the time spent on search for information. Or the they might actually spent the same or even more time on searching but discover more relevant information which results in improvements in the quality of the job deliverables. In the first case, a saving 10% of the time (1 hour per week) can be translated into potential saving of $1,425 a year per worker. If we only effected a 10% of the knowledge worker population at Ford, that would a saving of 3,000 x $1,425 = $4,275,000. Can this be translated to reduced workforce in term salary and benefits paid? Probably not, but we can safely say it would positively impacted the business bottom line indirectly through better work through-put, increased quality, and improved morale (reduced stressed at work). The second scenario is much harder to measure. If a research scientist can find a new connections among previously unknown facts due to improved search performance, the impact of productivity gain can be measured by new patents and innovate products that changes the competitive landscape in the industry. As we can tell, the benefits of various technologies on productivities can be measured directly through the amount of time saved. But the impacts are far more indirect and potentially very significant.

As you can see, there is a lot of "IF" statements. To a large degree, The numbers are anecdotal but yet very convincing to reflect the order of magnitude of the positive impact. Like other strategic investments, such as Knowledge Management, the returns are mixed with multitude of other internal and external factors when it comes to impact on bottom line. In a business environment of doing more with less, productivity improvement is not a "nice to have" but a basic element for survival.


Mark May
 

Yao and fellow SIKM leaders,

In the not too recent past (2003), APQC completed a best practice
study on Measuring the Impact of KM. It describes several case
studies from Caterpiller, Ford, Halliburton, IBM and Schlumberger. If
you or your firm is an APQC member, you can download it for free.

http://tinyurl.com/66xama

If you don't have access, I can summarize the case studies is a few
words - best practices in measurement included selective sampling and
heavy anecdotes that linked KM activities to business outcomes that
resonated with management (cost savings, cost avoidance, etc.).

Best regards,
Mark May
IBM Global Technology Services


Mark D Neff <mneff@...>
 


Mark,

Please do. Share what you can Mark. Five years is a lot in web time but maybe not so much in KM time . The more people understand what we do and the value it provides the better it will be for everyone. Learning to put what we do in terms others can understand and demonstrate value in ways they value, the easier it will be to do more KM work. Interesting following this discussion.

Mark Neff




"Mark"
Sent by: sikmleaders@...

08/18/2008 06:19 PM

Please respond to
sikmleaders@...

To
sikmleaders@...
cc
Subject
[sikmleaders] Re: Weighing the benefits of productivity improvements





Yao and fellow SIKM leaders,

In the not too recent past (2003), APQC completed a best practice
study on Measuring the Impact of KM.  It describes several case
studies from Caterpiller, Ford, Halliburton, IBM and Schlumberger.  If
you or your firm is an APQC member, you can download it for free.

http://tinyurl.com/66xama

If you don't have access, I can summarize the case studies is a few
words - best practices in measurement included selective sampling and
heavy anecdotes that linked KM activities to business outcomes that
resonated with management (cost savings, cost avoidance, etc.).

Best regards,
Mark May
IBM Global Technology Services


------------------------------------

Yahoo! Groups Links

<*> To visit your group on the web, go to:
   http://groups.yahoo.com/group/sikmleaders/

<*> Your email settings:
   Individual Email | Traditional

<*> To change settings online go to:
   http://groups.yahoo.com/group/sikmleaders/join
   (Yahoo! ID required)

<*> To change settings via email:
   mailto:sikmleaders-digest@...
   mailto:sikmleaders-fullfeatured@...

<*> To unsubscribe from this group, send an email to:
   sikmleaders-unsubscribe@...

<*> Your use of Yahoo! Groups is subject to:
   http://docs.yahoo.com/info/terms/