Date   

Re: Assessing Use of Knowledge: Feedback and a Couple of Questions #metrics #knowledge-reuse

Ivan Butina
 

Dan, Tom, Fred, and Daan,

Thanks so much for sharing your inputs. 

@Dan: Thanks for making the connection to taxonomy and community and for your premise at the end. Since we have some excellent digital communities and an excellent taxonomist (who's also here in SIKM), we'll definitely consult with them as well. What you shared provides some useful food for thought in that direction.

@Tom and @Fred: excellent questions from both of you. So, thanks for taking the time to share them! A few of them are easier do answer (what knowledge, what user audience, what outcome is expected - at least on paper), while others certainly need more reflection. And this is not only demonstrating value but also about helping teams doing KM focus on what works in getting the knowledge that is needed where it is needed (how it's packaged, how it's shared, how it's accessed). Since in our case usually, KM teams have few resources, it's important to focus on what creates value for the end-users. 

@Daan: Lots to look into and then digest from what you shared: thanks a lot! I especially appreciate the categorization of different types of knowledge products and services. And I'll let Ian know, although he might read this directly as well :) 

I will keep you posted over the next weeks/months. In the meantime, if anyone else has something to share, please, do so. 

@Katrina Pugh and @Nancy Dixon: I might even volunteer for a peer assist at some point :)

Best,
Ivan

On Thu, Jun 18, 2020 at 9:42 PM Daan Boom <daanboom@...> wrote:
Dear Ivan:

Trust you are doing fine. Thank you for sharing the concept paper on using knowledge. It has been a subject of my attention and study and practiced developing framework to measure effectiveness of knowledge and learning in profit and non-profit organizations. For some organizations developing pop-up windows requesting a user or visitor of the website and downloaded a publication. It provides some qualitative information in the format of stories. From my research in a dedicated global research institute I also noted that information can be shelved for a long time before it was used and applied. I recall one example that information on a physics subject was known and documented but at that time shelved because it was to fundamental. About 30 years later, and through conversations (not through a search/browse process in the internal database) someone remembered the research from way back. Thirty years later the information could be applied because other missing pieces had filled the knowledge gap. To stay practical for your framework the following. First I would recommend to take notice of the Frascati manual (OECD, 2015). The manual defines concepts on data and research e.g. knowledge. Second, I noticed in your article that you define information as knowledge vice versa (Unicef produces a lot of knowledge products, including technical guidance, webinars and research reports). You may need to define more precisely what distinct an UNICEF knowledge product. You may wish to take a fresh look at a definition I used in a KM strategy paper:
  
Knowledge Products and Services: Knowledge-rich content and activities that enable, facilitate or support decisions or actions by intended users.
Knowledge Products: publications that 
showcase the organization thought leadership through forecasts, research, and major data and trend analytics in the economic, sector and thematic areas.technical studies, working papers, policy briefs, op-eds, case studies, toolkits, learning modules, technical blog posts, technical notes, websites, videos, infographics, peer reviewed journal articles, impact evaluations, Q&As/FAQs, guides
Knowledge Services: conferences, workshops, seminars and other similar events, learning events, peer reviews, knowledge partnerships and networks, capacity development events, policy advice and dialogues. 

Thirdly, in one of the upcoming KM4Dev journal an article an article from me and two other associates in the CCLFI organization will appear on ‘framework for measuring the impact of knowledge management solutions applied to wok processes'. It can be downloaded. In this article we are proposing a framework (Kirkpatrick model) to measure use and application of knowledge. This Kirkpatrick model also provided input to a methodology PLATO, a Practical Learning Assessment Tool for Organizations, we use in organization to assess and monitor organizational capacities to uptake, share and use knowledge. You find information on PLATO in the product and services segment of CCLFI website. This model uses an algorithm to assess the perceived value of knowledge. 

Hope this information provides you further guidance.

Daan

PS: regards to Ian!


 
On Jun 19, 2020, at 01:43, Ivan Butina <ivan.butina@...> wrote:

<Concept Note - Monitoring utilization of knowledge (CP pilot).docx>


Re: Challenge: Identify the source of the high volume of replays for the Feb. 18, 2020 monthly call recording #monthly-call

Murray Jennex
 

True, but since we aren't talking large numbers (a couple of downloads vs 40) and which are only significant when compared to each other (but not when compared to overall Internet traffic), I think the simpler explanation is probably the most likely.


-----Original Message-----
From: Luis Suarez <elesar1@...>
To: SIKM@groups.io
Sent: Fri, Jun 19, 2020 4:52 am
Subject: Re: [SIKM] Challenge: Identify the source of the high volume of replays for the Feb. 18, 2020 monthly call recording #challenge

That could well be it, Murray, but then again similar traffic should be happening with other recordings from other months and from what I understand from Stan that's not happening at the moment. So I guess the mystery continues, but we shall see what else we can find out ...

Thanks for the feedback!


Re: Challenge: Identify the source of the high volume of replays for the Feb. 18, 2020 monthly call recording #monthly-call

Christopher Johannessen
 

You could reach out to the call hosting provider and ask them to pull a page traffic report for you with a channel / source breakdown to see where the traffic is coming from. I imagine they probably have Google Analytics running on their website. Doesn't hurt to ask.

I do quite a bit of work in data analytics for websites and communities, as well as process mining lately for "internal digital analytics" - happy to look at any report you manage to get access to.


Chris Johannessen
https://www.linkedin.com/in/chrisjohannessen/


Re: Challenge: Identify the source of the high volume of replays for the Feb. 18, 2020 monthly call recording #monthly-call

Stan Garfield
 

I assume there is another cause for two main reasons:
  1. This is the only recording out of many that receives a high number of plays.
  2. This is happening every single day.
Today's report showed 17 plays for Luis's call yesterday, compared to 2 plays for Alice's call which was held this past Tuesday and would normally be the only recording receiving any plays.

My theory is that there is a presentation, site, brochure, training course, podcast, video, or article that mentions Luis's blog post and/or presentation recording and provides a link to it.  This happened once with a LinkedIn article that I had posted, and I eventually tracked the source to a brochure published by Salesforce.com that included a URL but no associated text.  This brochure was viewed by millions of customers and prospects, and many of them followed the link to my article. As of today, there have been 131,896 views of that article.


Re: Challenge: Identify the source of the high volume of replays for the Feb. 18, 2020 monthly call recording #monthly-call

 

That could well be it, Murray, but then again similar traffic should be happening with other recordings from other months and from what I understand from Stan that's not happening at the moment. So I guess the mystery continues, but we shall see what else we can find out ...

Thanks for the feedback!


Re: Challenge: Identify the source of the high volume of replays for the Feb. 18, 2020 monthly call recording #monthly-call

Murray Jennex
 

people catching up?  there was a lot of work that needed to be done fast in February and March....murray


-----Original Message-----
From: Stan Garfield <stangarfield@...>
To: SIKM@groups.io
Sent: Thu, Jun 18, 2020 6:11 am
Subject: Re: [SIKM] Challenge: Identify the source of the high volume of replays for the Feb. 18, 2020 monthly call recording #challenge

Today was a new high: 40 plays for the February call, as compared to 6 plays for the June call held earlier this week. Does anyone have any ideas as to how and why this is happening?

Conference - Ref# 45 - Recorded: Feb 18, 2020 10:52:14 AM EST - Duration: 68m:46s - 40 Plays
June 17, 2020 - 40 Plays

Conference - Ref# 49 - Recorded: Jun 16, 2020 10:49:48 AM EDT - Duration: 70m:52s - 6 Plays
June 17, 2020 - 6 Plays


Knowledge Organisation Specialist - Contract Position in Singapore #jobs

Patrick Lambe
 

The Monetary Authority of Singapore (the de facto Central Bank and regulator) has recently established a new Enterprise Knowledge Department and has been building a dynamic and diverse team over the past months.

They are seeking to recruit a Knowledge Organisation Specialist to work at the interface of taxonomy/metadata, search, information architecture, and auto-classification. This is a contract position, advertised as 1 year, but they have told me that they could consider a 2 year contract for a strong international candidate.

This role is meant to establish internal capabilities and to carry forward a current taxonomy + search + IA project. The new KM department is a result of the progress of this project and the recognition of senior management of the importance of this aspect of KM.

In my view this is a challenging, but interesting role, working with excellent, detail-driven people, and an opportunity to work the interfaces between search, taxonomy + metadata, and auto classification. They have acquired a taxonomy management system. This is going to need to be a person with discipline, and very good communication skills as well as technical skills.

The job advertisement can be reached via the link below.


P

Patrick Lambe
Partner
+65 62210383



twitter: @plambesg

Knowledge mapping made easy: www.aithinsoftware.com


Re: Assessing Use of Knowledge: Feedback and a Couple of Questions #metrics #knowledge-reuse

Daan Boom
 

Dear Ivan:

Trust you are doing fine. Thank you for sharing the concept paper on using knowledge. It has been a subject of my attention and study and practiced developing framework to measure effectiveness of knowledge and learning in profit and non-profit organizations. For some organizations developing pop-up windows requesting a user or visitor of the website and downloaded a publication. It provides some qualitative information in the format of stories. From my research in a dedicated global research institute I also noted that information can be shelved for a long time before it was used and applied. I recall one example that information on a physics subject was known and documented but at that time shelved because it was to fundamental. About 30 years later, and through conversations (not through a search/browse process in the internal database) someone remembered the research from way back. Thirty years later the information could be applied because other missing pieces had filled the knowledge gap. To stay practical for your framework the following. First I would recommend to take notice of the Frascati manual (OECD, 2015). The manual defines concepts on data and research e.g. knowledge. Second, I noticed in your article that you define information as knowledge vice versa (Unicef produces a lot of knowledge products, including technical guidance, webinars and research reports). You may need to define more precisely what distinct an UNICEF knowledge product. You may wish to take a fresh look at a definition I used in a KM strategy paper:
  
Knowledge Products and Services: Knowledge-rich content and activities that enable, facilitate or support decisions or actions by intended users.
Knowledge Products: publications that 
showcase the organization thought leadership through forecasts, research, and major data and trend analytics in the economic, sector and thematic areas.technical studies, working papers, policy briefs, op-eds, case studies, toolkits, learning modules, technical blog posts, technical notes, websites, videos, infographics, peer reviewed journal articles, impact evaluations, Q&As/FAQs, guides
Knowledge Services: conferences, workshops, seminars and other similar events, learning events, peer reviews, knowledge partnerships and networks, capacity development events, policy advice and dialogues. 

Thirdly, in one of the upcoming KM4Dev journal an article an article from me and two other associates in the CCLFI organization will appear on ‘framework for measuring the impact of knowledge management solutions applied to wok processes'. It can be downloaded. In this article we are proposing a framework (Kirkpatrick model) to measure use and application of knowledge. This Kirkpatrick model also provided input to a methodology PLATO, a Practical Learning Assessment Tool for Organizations, we use in organization to assess and monitor organizational capacities to uptake, share and use knowledge. You find information on PLATO in the product and services segment of CCLFI website. This model uses an algorithm to assess the perceived value of knowledge. 

Hope this information provides you further guidance.

Daan

PS: regards to Ian!


 

On Jun 19, 2020, at 01:43, Ivan Butina <ivan.butina@...> wrote:

<Concept Note - Monitoring utilization of knowledge (CP pilot).docx>


Re: Assessing Use of Knowledge: Feedback and a Couple of Questions #metrics #knowledge-reuse

Fred Nickols
 

I rarely respond because most of what I read seems way out of my league. However, I have been asked to “look into” various matters over the years, and I can easily envision a client asking me to “look into” the use of knowledge.

As with all my “investigations” I begin with questions of clarification. In this case, I would want to know at least the following:

What knowledge are we talking about?
What kind of knowledge is it?
How is it used or applied?
By whom?
To what ends?

Those are for starters.

Fred Nickols
Solution Engineer & Chief Toolmaker
Distance Consulting LLC


On Jun 18, 2020, at 3:12 PM, Tom Short <tshortconsulting@...> wrote:

Ivan - I’d start by laying out what a “win” looks like. Consider answers to questions like:
- who is the intended audience for the content/knowledge products you are managing?
- what good outcome is expected if that content is used effectively by your target audience?
- are there other ancillary good outcomes that might come about as a result of this collection of stuff, either for the producers who codified it or other types of consumers/users of it?
- what is the value of these good outcomes, in terms of time, cost, revenue or quality/effectiveness?

You might also consider the “anti-case” - who would be upset if this collection suddenly became unavailable? Or what would be the risk of not having it? 

To me, these are ways that I would explore to help me get at the business value that your work is generating. It is a very different approach than simply measuring transactions against artifacts. And it is much more likely to attract the attention of a wider audience of stakeholders - if you find a way to link it to business metrics that matter. 
--
-Tom
--

Tom Short Consulting
TSC
+1 415 300 7457

All of my previous SIKM Posts


Re: Assessing Use of Knowledge: Feedback and a Couple of Questions #metrics #knowledge-reuse

 

Ivan - I’d start by laying out what a “win” looks like. Consider answers to questions like:
- who is the intended audience for the content/knowledge products you are managing?
- what good outcome is expected if that content is used effectively by your target audience?
- are there other ancillary good outcomes that might come about as a result of this collection of stuff, either for the producers who codified it or other types of consumers/users of it?
- what is the value of these good outcomes, in terms of time, cost, revenue or quality/effectiveness?

You might also consider the “anti-case” - who would be upset if this collection suddenly became unavailable? Or what would be the risk of not having it? 

To me, these are ways that I would explore to help me get at the business value that your work is generating. It is a very different approach than simply measuring transactions against artifacts. And it is much more likely to attract the attention of a wider audience of stakeholders - if you find a way to link it to business metrics that matter. 
--
-Tom
--

Tom Short Consulting
TSC
+1 415 300 7457

All of my previous SIKM Posts


Re: Assessing Use of Knowledge: Feedback and a Couple of Questions #metrics #knowledge-reuse

Dan Ranta
 

Hello Ivan - I believe that a big part of the opportunity is related to effective taxonomy.  When you use, for example, a community-based taxonomy that is created and curated by folks with expertise in the domain, you are in a position to assess utilization of Knowledge.  What I have done in the past is show by taxonomy term all the blogs, discussions, content (including wiki articles), expertise, events, that are using that specific term.  At that point, you can delve further to assess utilization / re-use of the different types of knowledge.  Your taxonomy is the circulatory system in this case that helps you discover more.  One particularly savvy team member (Michael) created for us a real-time taxonomy coverage radial graph - to analyze the usage of each community taxonomy - showing gaps and opportunities.  An example of what it showed for every topic and subtopic was how long it took any problem-solving question to be answered that used a particular taxonomy.  It also showed blank areas in the radial graph where terms were not being used (maybe they were never needed).  The graph works by hovering your cursor over any part of the graph (representing terms) and in the table on the right, you see knowledge utilization results.  Again, real-time and available for all to see.

My Premise:  Taxonomy opens the door for more effective knowledge assessment of utilization, etc..  

image.png

On Thu, Jun 18, 2020 at 11:43 AM Ivan Butina <ivan.butina@...> wrote:
Fellow KMers, 

I hope that I can tap into your experience and expertise to address one of the areas of KM that I've been wanting to address in a while: assessing the use of knowledge. In my experience, KM work often focuses on the generation, capturing, curation, and sharing of knowledge leaving the use of knowledge as an after-thought. To be more specific, by this I mean the ability to assess (track, monitoring, measure) the extent to which knowledge is being (re)used. This is crucial to be able to show the value of KM work but also to be able to improve KM work (which approaches lead to the use of knowledge and which ones don't, and why). 

Recently, I came across colleagues in my organization (UNICEF) who are willing to undertake this work and I have started working with them on creating a pilot that we would test within their area of work (child protection) to then create a framework that could be used by the whole organization. 

I'm attaching here the first draft of a concept note, in case anyone would like to provide feedback, and sharing two questions that I offered to share with KM colleagues outside our "UN bubble."

 

  • What are your experiences and good examples of assessing utilization of knowledge, beyond quantitative numbers/proxies (e.g. views, downloads, online community interactions such as comments, likes etc.)?
  • Do you use any computerized programmes/algorithms to capture the utilization of knowledge internally or externally?


The first question is on how to go from typical measurements like views, downloads etc. to what can actually tell us whether a knowledge product is being used.


The second question is about the possibility to use tech to automate any component of what to us looks like a pretty labor-intensive exercise. For example, I came across a software that's used in academia to scan documents for specific terms. We could use that to see whether the program/project design documents that UNICEF teams produce incorporate references to existing knowledge products (guidance, briefs, good practices lessons learned, evaluations, research etc.).


Best,

Ivan


Assessing Use of Knowledge: Feedback and a Couple of Questions #metrics #knowledge-reuse

Ivan Butina
 

Fellow KMers, 

I hope that I can tap into your experience and expertise to address one of the areas of KM that I've been wanting to address in a while: assessing the use of knowledge. In my experience, KM work often focuses on the generation, capturing, curation, and sharing of knowledge leaving the use of knowledge as an after-thought. To be more specific, by this I mean the ability to assess (track, monitoring, measure) the extent to which knowledge is being (re)used. This is crucial to be able to show the value of KM work but also to be able to improve KM work (which approaches lead to the use of knowledge and which ones don't, and why). 

Recently, I came across colleagues in my organization (UNICEF) who are willing to undertake this work and I have started working with them on creating a pilot that we would test within their area of work (child protection) to then create a framework that could be used by the whole organization. 

I'm attaching here the first draft of a concept note, in case anyone would like to provide feedback, and sharing two questions that I offered to share with KM colleagues outside our "UN bubble."

 

  • What are your experiences and good examples of assessing utilization of knowledge, beyond quantitative numbers/proxies (e.g. views, downloads, online community interactions such as comments, likes etc.)?
  • Do you use any computerized programmes/algorithms to capture the utilization of knowledge internally or externally?


The first question is on how to go from typical measurements like views, downloads etc. to what can actually tell us whether a knowledge product is being used.


The second question is about the possibility to use tech to automate any component of what to us looks like a pretty labor-intensive exercise. For example, I came across a software that's used in academia to scan documents for specific terms. We could use that to see whether the program/project design documents that UNICEF teams produce incorporate references to existing knowledge products (guidance, briefs, good practices lessons learned, evaluations, research etc.).


Best,

Ivan


Re: Challenge: Identify the source of the high volume of replays for the Feb. 18, 2020 monthly call recording #monthly-call

 

Hi Lesley, thanks a lot for the feedback and for the info details. This is, indeed, what has baffled both Stan and myself. I don't use Google Analytics but Jetpack. When I look into the Jetpack analytics I can see some of the traffic to that blog post, but in no way to the numbers Stan is seeing in the recording itself. When checking out the Referrers I see some traffic, but most of that is coming from Google Search into that blog post. Again, in lower numbers than Stan reports from the recording.  And that's why are intrigued by that particular unknown source of the extra traffic created, because we just can't track what it is. That's why Stan decided to ask the community to see if anyone may be able to shed some light from similar experiences they may have encountered in the past.

I don't know where else to look to find that source that brings all of that traffic. But many thanks, anyway, for the hints and for the advice. I very much appreciate it.


Re: Challenge: Identify the source of the high volume of replays for the Feb. 18, 2020 monthly call recording #monthly-call

 

Does Luis have google analytics installed on his blog? if so, you could go in google analytics to Behavior->Site Content->Landing Pages and then look for the past 30 days/etc. or different time period at how many sessions/day that page is seeing, and if you look at the Source or the "Referral Path" under secondary dimensions it will tell you what url those referrals are coming from. 


Re: Knowledge recognition technologies #tools

Katrina Pugh
 

Hi, all
I would add to the CoP platforms Influent Solutions ( http://Influentsolutions.ch)  Influent works with UN, WHO, and large cross-organizational networks). (Community Cloud is the name of the product).

(We’re looking at this for a plastic waste nonprofit http://bit.ly/2X4PP20.)

Thanks
Kate

Katrina Pugh
AlignConsulting | Collaboration, Analytics and Strategy
Columbia University | Information and Knowledge Strategy Master of Science Program

On Jun 18, 2020, at 12:24 AM, Stan Garfield <stangarfield@...> wrote:

For forums and Q&A, I suggest evaluating these major vendors of Enterprise Social Networks (ESNs):
  1. Atlassian Confluence
  2. Aurea Jive
  3. Facebook Workplace
  4. Google Currents
  5. HCL Connections
  6. Limeade (formerly Sitrion)
  7. Microsoft Yammer
  8. Salesforce Chatter
  9. SAP Jam
  10. TIBCO tibbr


Re: Challenge: Identify the source of the high volume of replays for the Feb. 18, 2020 monthly call recording #monthly-call

Stan Garfield
 

Today was a new high: 40 plays for the February call, as compared to 6 plays for the June call held earlier this week. Does anyone have any ideas as to how and why this is happening?

Conference - Ref# 45 - Recorded: Feb 18, 2020 10:52:14 AM EST - Duration: 68m:46s - 40 Plays

June 17, 2020 - 40 Plays


Conference - Ref# 49 - Recorded: Jun 16, 2020 10:49:48 AM EDT - Duration: 70m:52s - 6 Plays

June 17, 2020 - 6 Plays


Re: Knowledge recognition technologies #tools

Stan Garfield
 

For forums and Q&A, I suggest evaluating these major vendors of Enterprise Social Networks (ESNs):
  1. Atlassian Confluence
  2. Aurea Jive
  3. Facebook Workplace
  4. Google Currents
  5. HCL Connections
  6. Limeade (formerly Sitrion)
  7. Microsoft Yammer
  8. Salesforce Chatter
  9. SAP Jam
  10. TIBCO tibbr


Re: Knowledge recognition technologies #tools

Tom Barfield
 

Rahul – Years ago one of my teams explored an open source Q&A system that was similar to Stack Exchange.  We determined it wasn’t a great fit for us – but could be good for you.  I don’t have details but I just did a search that you might find gives you a whole bunch of ideas to consider.

 

https://www.google.com/search?ei=3NPqXo-qJM7QtQbk2KbICw&q=open+source+stack+overflow+alternative&oq=open+source+stack+overflow+alternative&gs_lcp=CgZwc3ktYWIQAzICCAA6BAgAEA06BggAEAcQHjoICAAQDRAFEB46CAgAEAgQDRAeOgUIIRCrAlCrpAFYo7kBYO-6AWgBcAB4AIABaogBoAaSAQM5LjGYAQCgAQGqAQdnd3Mtd2l6&sclient=psy-ab&ved=0ahUKEwiPiIP6qorqAhVOaM0KHWSsCbkQ4dUDCAw&uact=5

 

Tom

 

From: SIKM@groups.io <SIKM@groups.io> On Behalf Of Rahul Lama
Sent: Wednesday, June 17, 2020 10:57 AM
To: SIKM@groups.io
Subject: Re: [SIKM] Knowledge recognition technologies

 

Hi Aprill,

We got in touch with the Guru team and their enterprise version is very expensive for my organization. 

We are thinking of the Discourse platform for piloting. Any platforms we can use for as Forums or for Q&A for the org where everyone can ask questions, seek people and connect? Would Groups.io be a good fit for that?

Regards,
Rahul


Re: COP Charters or Templates #CoP

Tom Barfield
 

There have been great resources shared in this thread.  I have had added the following to the KM Collection

- SIKM Charter (Kate)
- ABC Charter Examples (Dan)
- NASA example (Doug)
- Stan's post with several links (Stan)

Here is a public link to the Community of Practice section of the KM Collection.
Anyone who has registered on the KM Collection and has installed the browser extension will be able to easily find these materials when searching on the internet.

Tom


Re: Knowledge recognition technologies #tools

Aprill Allen
 

Hi Rahul,

Hmm. OK. Good to know re the enterprise cost. 
Discourse and Vanilla are well regarded for forums and Q&A. 
Groups.io is not ideal. There are limitations that don't make it great for large organisations. It's really just a slightly fancy mailing list. 

--

Aprill Allen
Founder and Managing Director | Knowledge Bird
KM Consulting & KCS Training
M: +61 (0)400 101 961
knowledgebird.com

1461 - 1480 of 9248