"Big data" - does it have anything to do with knowledge management? #data-science
|
Matt Moore <innotecture@...>
Hi,
Over the last 18 months, I've been unable to avoid the phrase 'big data". It's a buzz term that covers a lot of different activities and tools. It's vague but it's not meaningless and I'm deliberately not going to define it in this email. I've written an introductory article that provides on the topic that should come out in a few months (ah, the joys of the traditional publishing process). I'm thinking of following it up with an article that features some specific examples and also some quotes from people that have interesting opinions on the topic. My angle is very much from an information management / knowledge management perspective rather than, say, a pure data analytics perspective. So my questions are: Does the term "Big Data" mean anything to you? And does it have anything to do with knowledge management? Thanks in advance. Matt Moore BTW For anyone I do quote in the article, I will always ask your permission to do so first. |
|
|
|
Joe Raimondo
Matt:
toggle quoted message
Show quoted text
With regard to "bid data" and KM, I share your consternation and confusion. Using the term data, for example, conjures for me flowchart symbols for databases, or a SQL Server Admin interface. And big--it conjures mainly hype, as you noted. That said, I have been engaged in a technology project that involves exploring some new technology approaches to KM--particularly in the realm of adopting NoSQL databases and MapReduce techniques. This is an example where we have been exploring application of some of these emerging technologies and how they engulf some of our common conceptions about how and where data are stored and accessed. Moreover, it has an impact on some of the basic models that people concerned with KM uses , e.g.,search and findability, structuring, sorting, and tagging, etc. A whole new world of capabilities is coming on lone. It's very exciting in terms of potential. but understand that the technology is still raw and it reveals its secrets reluctantly. As noted, when I see the word data, I'm thinking of databases as a single thing. I'm not thinking so much the processes that create the data, the interactions the data represents, or the means by which the data is transformed into knowledge or insight--something I now know as a result of arranging the content of the data and displaying it in a particular way that I didn't know before. That said, I think it's useful to tease apart the three main phases of a "bid data" model: - Input. - Rendering and storage - Access, analysis and output Again, from a grunt-level technology perspective, you can see how this is vital. But from a KM perspective, we can now drop sensors into many more parts of the organization, from the log file level on up, to start to get some insight. The key notion from "big data" is actually the analytics part--the analysis and output component. Yo can envision systems that provide near-real-time indicators of a variety of signal from low levels that can be statistically analyzed to show demonstrable and valid lift in performance. You can then use these indicators as a means to dive into the knowledge base of that group--everything is stored and tracked and accessible and render-able. It changes the perspective on the performance management game. = Joe Raimondo = Ontologique --- In sikmleaders@..., Matt Moore wrote:
|
|
|
|
Steve Ardire <sardire@...>
> So my questions are: Does the term "Big Data" mean anything to you? And does it have anything to do with knowledge management?
toggle quoted message
Show quoted text
Yes and Yes and for more see We don’t need more data scientists — just make big data easier to use http://gigaom.com/2012/12/22/we-dont-need-more-data-scientists-just-simpler-ways-to-use-big-data/
|
|
|
|
gordonvalawebb <gvalawebb@...>
"Big data" is most certainly related to KM - although not many "big data" talkers think about it from a K perspective.
toggle quoted message
Show quoted text
Dion Hinchcliff wrote a posting a couple of years ago which I think nailed it. The title says it all: "How social media and big data will unleash what we know" http://www.zdnet.com/blog/hinchcliffe/how-social-media-and-big-data-will-unleash-what-we-know/1533 For me, KM is fundamentally about trying to help people within an organization made better decisions / judgments. And data (big or otherwise) are important as an input into certain kinds of decisions. So Big Data is part of KM. Hope that helps, G --- In sikmleaders@..., Matt Moore wrote:
|
|
|
|
Steve Ardire <sardire@...>
This Humanizing Big Data Infographic should help and I have no ties to alteryx
toggle quoted message
Show quoted text
On Mon, Jan 7, 2013 at 6:03 AM, Steve Ardire <sardire@...> wrote:
|
|
|
|
Simard, Albert <albert.simard@...>
IMHO:
“Big Data” is related to KM, but to me, it is in the sense of integrating IT, DM, IM, and KM to support knowledge work and the organization’s business. I find that a majority of organizational work requires more than one of the above categories of service and to distinguish between them (above architectural and functional levels) is somewhat artificial from a business perspective. For example, organizations might want to monitor their environment, produce actionable intelligence, integrate content, and/or mobilize knowledge to address complicated or complex problems and issues. These are conceptually similar processes (acquire, analyse, apply) with variations in methods.
We need IT systems to access and process petabytes of Web-based content. The question is how do we classify services such as a Web server, search engine, AI report generator, and content repository, all of which are necessary for environmental monitoring? We need data management to efficiently search for and organize patterns in the flod of data (I prefer content) to find something interesting. So, we bring in pattern recognition and statistical apps, metadata, content taxonomies, and more repositories. Then we need analysts to analyse or synthesize and interpret the patterns in the context of the organization’s needs and business. Although this is pure knowledge work (e.g., thinking is required), it is normally supported with directories of expertise, sharing sites, Cop’s and collaboration sites to corroborate and/or validate what may have been found. Then we need someone to write and submit an intelligence report including recommended action, using office application software, work processes, and an understanding of the organization’s cultural context. But we’re not done yet. That report must finally be considered by decision makers and approval given (or not) for action to be taken.
So, yes, Big Data is connected to KM, in the same sense that, as John Muir stated in 1911:
"When we try to pick out anything by itself, we find it hitched to everything else in the Universe."
I could go all the way back to the Corpus Hermiticum, (Egypt, 30th century BC) for a similar observation on connectedness, but John Muir is more recognizable.
Al Simard
From: sikmleaders@... [mailto:sikmleaders@...] On Behalf Of JoeR
Matt: |
|
|
|
Patrick Lambe
A few tangential naturalistic/political remarks:
toggle quoted message
Show quoted text
I think what makes "big data" sexy is the opacity around the methods used to extract insight from large datasets - and often large datasets from diverse sources. This has the dual effect of attracting talent (you've got to be technically skilled in statistical science and smart in analytics to identify potentially meaningful and useful patterns) as well as hucksters (nobody has the time or access to check your methodology, especially when some of the datasets are proprietary/ not subject to external scrutiny). This lack of transparency is not just a problem in business, it's also a big issue in scientific research, where there's a movement - meeting all kinds of political and logistical resistance - to publish datasets that underpin research findings. As a taxonomist, what is not clear to me are the means by which meaning is resolved across diverse datasets, and then how that resolution is validated. Take no-fly lists. We know how often the wrong people are barred from flying because they have the "wrong" names. We also know how people slip through social protection nets because medical/social service records are scattered across name variations so that records around the same case are never resolved to build bigger pictures (Vivian Alvarez case in Australia comes to mind). Names of people should be among the simpler things to resolve. Much harder are terms for concepts, activities, ages vs age ranges, subjective terms (eg side effects of medications, degree of pain/nausea etc). The lack of transparency around such methods makes "big data" a bit of a black art. Talent comes up with a few really exciting examples (Target and pregnancy comes to mind) which we cherry pick to whip up excitement, the hucksters waste a deal of time and money on producing rotten apples which degrade the perceptions of the field over time, the technologists come up with black box "solutions" that sidestep the high costs of real talent, but produce manifestly unreliable results that you only discover are unreliable after a couple of million bucks, and it slowly merges back into the background as more robust methodologies emerge. KM used to be a lot like that, and still is in pockets, but KM is now mostly just hard work - like IM. I think there is some genuine novelty around "big data" - pattern-seeking across diverse datasets in particular - but the challenges are non-trivial, and the proclaimed benefits over-simplistically presented. P Patrick Lambe Partner Tel: 62210383 weblog: www.greenchameleon.com Have you seen our new KM Planning Toolkit? On Jan 6, 2013, at 6:06 PM, Matt Moore wrote:
|
|
|
|
Tom Reamy <tomr@...>
Does Big Data have anything to do with KM? I’d say not directly but lots of implications for KM. Big Data is usually “defined” by the three V’s – Volume, Velocity, and Variety. Volume and Velocity are what is really different about Big Data – so much content coming in at such a scale that traditional data bases can’t handle them and so you get new technologies (Hadoop, etc.) and of course IT defines movements and “revolutions” based on the technology. And this technology has little to do with KM.
However, what does impact KM is what can be done with all that content – especially social media but also an increased ability to gain insights into groups of people, both outside and inside the organization, by utilizing new statistical and analytical techniques that are being developed. But what is needed to bring in social media in any intelligent way is better methods for analyzing unstructured text -- the scope of which still makes big data small in comparison, IOW, Big Text is way bigger than Big Data. So it seems to me that text analytics will have a bigger impact on both KM and the uses of Big Data.
I just posted an announcement in a number of KM forums about an upcoming conference, Text Analytics World, that looks at text analytics, Big Data, and KM among other topics. In the interest of disclosure, I’m the program chair of the conference, so I’m biased, but I strongly think that text analytics will end up having a bigger impact on KM than Big Data.
And I wouldn’t worry about this being mere data rather than information and knowledge – I agree with a speaker at KMWorld in November – the DIKW hierarchy must die!
Tom
Tom Reamy Chief Knowledge Architect KAPS Group, LLC 510-530-8270 (O) 510-530-8272 (Fax) 510-333-2458 (M)
Text Analytics World San Francisco: April 17-18, 2013
From: sikmleaders@... [mailto:sikmleaders@...] On Behalf Of Simard, Albert
Sent: Tuesday, January 08, 2013 10:37 AM To: sikmleaders@... Subject: RE: [sikmleaders] Re: "Big data" - does it have anything to do with knowledge management?
IMHO:
“Big Data” is related to KM, but to me, it is in the sense of integrating IT, DM, IM, and KM to support knowledge work and the organization’s business. I find that a majority of organizational work requires more than one of the above categories of service and to distinguish between them (above architectural and functional levels) is somewhat artificial from a business perspective. For example, organizations might want to monitor their environment, produce actionable intelligence, integrate content, and/or mobilize knowledge to address complicated or complex problems and issues. These are conceptually similar processes (acquire, analyse, apply) with variations in! methods.
We need IT systems to access and process petabytes of Web-based content. The question is how do we classify services such as a Web server, search engine, AI report generator, and content repository, all of which are necessary for environmental monitoring? We need data management to efficiently search for and organize patterns in the flod of data (I prefer content) to find something interesting. So, we bring in pattern recognition and statistical apps, metadata, content taxonomies, and more repositories. Then we need analysts to analyse or synthesize and interpret the patterns in the context of the organization’s needs and business. Although this is pure knowledge work (e.g., thinking is required), it is normally supported with directories of expertise, shar! ing sites, Cop’s and collaboration sites to corroborate and/or v alidate what may have been found. Then we need someone to write and submit an intelligence report including recommended action, using office application software, work processes, and an understanding of the organization’s cultural context. But we’re not done yet. That report must finally be considered by decision makers and approval given (or not) for action to be taken.
So, yes, Big Data is connected to KM, in the same sense that, as John Muir stated in 1911:
"When we try to pick out anything by itself, we find it hitched to everything else in the Universe."
I could go all the way back to the Corpus Hermiticum, (Egypt, 30th century BC) for a similar observation on connectedness, but John Muir is more recognizable.
Al Simard
From: sikmleaders@... [mailto:sikmleaders@...] On! Behalf Of JoeR
Matt: |
|
|
|
This might be of interest. Deloitte Debates: Does Big Data Still Need the Human Touch? |
|
|
|
Sameer Bhide
Very interesting to see some of the emails related to Big Data & KM ! After spending 17-18 years as a KM practitioner, iam getting involved now in Big Data & Analytics area with my clients ! My initial take on Big Data projects etc is that many organizations are looking only at the "Infrastructure and Technology" side
of Big Data NOT "business outcomes". The challenge was, is and will be not only to derive "insights" from Big Data, but what "actions" an organization can take based on those insights and how those actions affect the bottom line. I don't think corporate America did well in that aspect whether as a part of KM or BI or DW initiatives. Hopefully we all will learn from the past decade and focus on it during Big Data initiatives....otherwise it will be one more corporate buzzword ! Regards Sameer Bhide Senior Manager, Global Consulting Practice Tata Consultancy Services Cell:- 7036063039 Mailto:
s.bhide@... Website: http://www.tcs.com From: Tom Reamy To: sikmleaders@... Sent: Wednesday, January 9, 2013 5:37 PM Subject: RE: [sikmleaders] Re: "Big data" - does it have anything to do with knowledge management? Does Big Data have anything to do with KM? I’d say not directly but lots of implications for KM. Big Data is usually “defined” by the three V’s – Volume, Velocity, and Variety. Volume and Velocity are what is really different about Big Data – so much content coming in at such a scale that traditional data bases can’t handle them and so you get new technologies (Hadoop, etc.) and of course IT defines movements and “revolutions” based on the technology. And this technology has little to do with KM. However, what does impact KM is what can be done with all that content – especially social media but also an increased ability to
gain insights into groups of people, both outside and inside the organization, by utilizing new statistical and analytical techniques that are being developed. But what is needed to bring in social media in any intelligent way is better methods for analyzing unstructured text -- the scope of which still makes big data small in comparison, IOW, Big Text is way bigger than Big Data. So it seems to me that text analytics will have a bigger impact on both KM and the uses of Big Data. I just posted an announcement in a number of KM forums about an upcoming conference, Text Analytics World, that looks at
text analytics, Big Data, and KM among other topics. In the interest of disclosure, I’m the program chair of the conference, so I’m biased, but I strongly think that text analytics will end up having a bigger impact on KM than Big Data. And I wouldn’t worry about this being mere data rather than information and knowledge – I agree with a speaker at KMWorld in November – the DIKW hierarchy must die! Tom Tom Reamy Chief Knowledge Architect KAPS Group, LLC 510-530-8270 (O) 510-530-8272 (Fax) 510-333-2458 (M) Text Analytics World San Francisco: April 17-18, 2013 From: sikmleaders@... [mailto:sikmleaders@...] On Behalf Of Simard, Albert Sent: Tuesday, January 08, 2013 10:37 AM To: sikmleaders@... Subject: RE: [sikmleaders] Re: "Big data" - does it have anything to do with knowledge management? IMHO: “Big Data” is related to KM, but to me, it is in the sense of integrating IT, DM, IM, and KM to support knowledge work and the organization’s business. I find that a majority of organizational work requires more than one of the above categories of service and to distinguish between them (above architectural and
functional levels) is somewhat artificial from a business perspective. For example, organizations might want to monitor their environment, produce actionable intelligence, integrate content, and/or mobilize knowledge to address complicated or complex problems and issues. These are conceptually similar processes (acquire, analyse, apply) with variations in! methods. We need IT systems to access and process petabytes of Web-based content. The question is how do we classify services such as a Web server, search engine, AI report generator, and content repository, all of which are necessary for environmental monitoring? We need data management to efficiently search for and organize patterns in the flod of data (I prefer content) to find
something interesting. So, we bring in pattern recognition and statistical apps, metadata, content taxonomies, and more repositories. Then we need analysts to analyse or synthesize and interpret the patterns in the context of the organization’s needs and business. Although this is pure knowledge work (e.g., thinking is required), it is normally supported with directories of expertise, shar! ing sites, Cop’s and collaboration sites to corroborate and/or v alidate what may have been found. Then we need someone to write and submit an intelligence report including recommended action, using office application software, work processes, and an understanding of the organization’s cultural context. But we’re not done yet. That report must finally be considered by decision makers and approval given (or not) for action to be taken. So, yes, Big Data is connected to KM, in the same sense that, as John Muir stated in 1911: "When we try to pick out anything by itself, we find it hitched to everything else in the Universe." I could go all the way back to the Corpus Hermiticum, (Egypt, 30th century BC) for a similar observation on connectedness, but John Muir is more recognizable. Al Simard From: sikmleaders@... [mailto:sikmleaders@...] On! Behalf Of JoeR Sent: January-07-13 8:44 AM To: sikmleaders@... Subject: [sikmleaders] Re: "Big data" - does it have anything to do with knowledge management? Matt: With regard to "bid data" and KM, I share your consternation and confusion. Using the term data, for example, conjures for me flowchart symbols for databases, or a SQL Server Admin interface. And big--it conjures mainly hype, as you noted. That said, I have been engaged in a technology project that involves exploring some new technology approaches to KM--particularly in the realm of adopting NoSQL databases and MapReduce techniques. This is an example where we have been exploring application of some of these emerging technologies and how they engulf some of our common conceptions about how and where data are stored and accessed. Moreover, it ha! s an impact on some of the basic models that people concerned with KM uses , e.g.,search and findability, structuring, sorting, and tagging, etc. A whole new world of capabilities is coming on lone. It's very exciting in terms of potential. but understand that the technology is still raw and it reveals its secrets reluctantly. As noted, when I see the word data, I'm thinking of databases as a single thing. I'm not thinking so much the processes that create the data, the interactions the data represents, or the means by which the data is transformed into knowledge or insight--something I now know as a result of arranging the content of the data and displaying it in a particular way that I didn't know before. That said, I think it's useful to tease apart the three main phases of a "bid data" model: - Input. - Rendering and storage - Access, analysis and output Again, from a grunt-level technology perspective, you can see how this is vital. But from ! a KM perspective, we can now drop sensors into many more parts of the organization, from the log file level on up, to start to get some insight. The key notion from "big data" is actually the analytics part--the analysis and output component. Yo can envision systems that provide near-real-time indicators of a variety of signal from low levels that can be statistically analyzed to show demonstrable and valid lift in performance. You can then use these indicators as a means to dive into the knowledge base of that group--everything is stored and tracked and accessible and render-able. It changes the perspective on the performance management game. = Joe Raimondo = Ontologique --- In sikmleaders@..., Matt Moore wrote: > > Hi, > > Over the last 18 months, I've been unable to avoid the phrase 'big data". It's a buzz term that covers a lot of different activities and tools. It's vague but it's not meaningless and I'm deliberately not ! going to define it in this email. I've written an introductory article that provides on the topic that should come out in a few months (ah, the joys of the traditional publishing process). I'm thinking of following it up with an article that features some specific examples and also some quotes from people that have interesting opinions on the topic. My angle is very much from an information management / knowledge management perspective rather than, say, a pure data analytics perspective. > > So my questions are: Does the term "Big Data" mean anything to you? And does it have anything to do with knowledge management? > > Thanks in advance. > > Matt Moore > > BTW For anyone I do quote in the article, I will always ask your permission to do so first. > |
|
|
|
Simard, Albert <albert.simard@...>
I can answer that question without thinking. Without the human element, big data is little more than an exercise in massive data processing and advanced statistics. And everyone knows what Disraeli said about statistics: “There are three kinds of lies: lies, damned lies, and statistics”
From: sikmleaders@... [mailto:sikmleaders@...] On Behalf Of StanGarfield
Sent: January-10-13 9:32 AM To: sikmleaders@... Subject: [sikmleaders] Re: "Big data" - does it have anything to do with knowledge management?
This might be of interest.
Deloitte Debates: Does Big Data Still Need the Human Touch? |
|
|
|
McKinsey outlined the field and the business opportunities fairly comprehensively in this 140+ whitepaper a year or so ago, but still a good reference on defining the space and market oppty. and skills gaps
toggle quoted message
Show quoted text
Also here is a round up of viewpoints of "Big Data in 2013" I'm intrigued by the 'data in motion' challenge - i.e., as the IFTF phrases it - massive increases in sensors and processing power a programmable system."
I agree re: the human touch --- Catherine Shinners @cathshinners
On Thu, Jan 10, 2013 at 7:21 AM, Simard, Albert <albert.simard@...> wrote:
--
Merced Group Silicon Valley, USA +1-650-704-3889 Email: catherineshinners@...
Twitter: @catshinners | Skype: CatherinePaloAlto
Social Business Strategic Consulting and Enterprise 2.0 Services |
|
|
|
Stephen Bounds
Going right back to Matt's original question about relevance to KM:
On 11/01/2013 2:06 AM, Sameer Bhide wrote: My initial take on Big Data projects etc is that many organizationsHow often have we heard a variant on this statement? * Sharepoint: often implemented in organisations as a technology project by ICT without any people with skills in information or knowledge management getting involved * Business process reengineering (BPR): "... attractive to traditional managers for several reasons. For one thing, it was a technology fix: managers didn’t need to change their behavior. They could sit back while technology solved the problem." http://stevedenning.typepad.com/steve_denning/2010/07/what-hbr-wont-say-why-bpr-failed.html * Government/Bureaucracy: "... a lot of taxpayers’ money is being spent without considerations to the final outcomes, especially to the people of the community it says it’s supporting ..." http://blogs.windsorstar.com/2012/10/23/74021/ What links these is that a solution is being implemented as a fix which doesn't consider the full context in which it needs to work. Therefore ANY change initiative, including Big Data, that isn't complemented by knowledge management is at greater risk of being poorly implemented or failing outright. Remember, organisations with good knowledge management: * correctly identify problems that need solving as they occur * optimise individual decision making * embrace effective knowledge creation processes * share and integrate knowledge as it is created across the whole of the organisation If Big Data can solve a legitimate organisational problem, it makes sense to use it. But the positive and negative consequences to an organisation and its stakeholders need to be carefully considered, and particularly whether alternative solutions may produce better outcomes. Cheers, -- Stephen. |
|
|
|
Simard, Albert <albert.simard@...>
Stephen - Right ON! In fact, I’d take it further. Any knowledge services architecture that doesn’t include both structured (technology) and unstructured (social) aspects of KM will achieve only a shadow of it’s potential.
Al Simard
From: sikmleaders@... [mailto:sikmleaders@...] On Behalf Of Stephen Bounds
Sent: January-10-13 3:24 PM To: sikmleaders@... Subject: Re: [sikmleaders] Re: "Big data" - does it have anything to do with knowledge management?
Going right back to Matt's original question about relevance to KM: |
|
|