Enterprise document rating systems - do they work? #ratings


Matt Moore <innotecture@...>
 

Hi,

I was looking at a list of content / collaboration functionality today and it mentioned "document rating". The initial model was Amazon (stars out of 5) but Twitter/Facebook ("like") is probably more prevalent now. This allows end users to rate what they think of a document, page, etc. Now I've seen this appear n a lot of functional requirements over the years but when it's implemented, it's not heavily used.

I'm curious to hear other people's experience. Are there situations where people have been rating content and others have been consuming that?

Regards,

Matt


Dennis Pearce <dennis.pearce@...>
 

We use Jive for our internal ESN and it has both likes and ratings (5 star).  The likes are very heavily used but the ratings hardly ever.  Ratings might be useful for an external community but I think there are several problems when they are used internally:
  1. A low rating of a document can be interpreted as a slam against a co-worker, and many people are reluctant to do that.
  2. Internal rating systems (unless the organization is really large) don't have the statistical power of an Amazon or eBay.  If you only have a handful or ratings, it doesn't mean much.  This is especially true when the average is displayed.  If in a company of thousands of employees you find a document that has a couple of 5-star and a couple of 1-star ratings for an average of 3, it's all pretty worthless.
  3. There is no provision for intended audiences.  So for example if I post a technical document that is pretty basic, it might get high ratings from novices but low ratings from experts.  Conversely, if I post a highly esoteric document on some arcane topic, the ratings might be reversed.
I think ratings work much better for products than for information, but if they are used for information then it probably should be clear who the intended audience is meant to be, and that audience should be large enough that an overall rating represents a consensus view.


Nancy Dixon
 

Very good advice. Your example of it being read by a novice vs an expert is right on. 

Many companies have found it useful to use "curated" vs "non-curated".  Curated means a small group of experts have read it and found it an accurate document that explains a critical step in a process. Non-curated documents are needed as well, and don't necessarily need to be labeled as "non-curated". They are often thought provoking and provide useful information. So it is important to support both.
Nancy

On Tue, Sep 6, 2016 at 8:29 AM, Dennis Pearce dennis.pearce@... [sikmleaders] <sikmleaders@...> wrote:
 

We use Jive for our internal ESN and it has both likes and ratings (5 star).  The likes are very heavily used but the ratings hardly ever.  Ratings might be useful for an external community but I think there are several problems when they are used internally:
  1. A low rating of a document can be interpreted as a slam against a co-worker, and many people are reluctant to do that.
  2. Internal rating systems (unless the organization is really large) don't have the statistical power of an Amazon or eBay.  If you only have a handful or ratings, it doesn't mean much.  This is especially true when the average is displayed.  If in a company of thousands of employees you find a document that has a couple of 5-star and a couple of 1-star ratings for an average of 3, it's all pretty worthless.
  3. There is no provision for intended audiences.  So for example if I post a technical document that is pretty basic, it might get high ratings from novices but low ratings from experts.  Conversely, if I post a highly esoteric document on some arcane topic, the ratings might be reversed.
I think ratings work much better for products than for information, but if they are used for information then it probably should be clear who the intended audience is meant to be, and that audience should be large enough that an overall rating represents a consensus view.



Stan Garfield
 

Matt, these may be helpful.

1. Content rating is different behind the firewall than it is on the Internet

2. Improving enterprise search results: Why don't you just tell me what you need?

 

3. Are you content with your content?


Regards,
Stan


Michael Koffman
 

Not used at Oracle in Health Sciences where I work - Millenium Pharmacy had a taxonomy of key terms that docs could be tagged with...I think that's more useful. Knowing something is a "research result" vs. an "opinion paper" would make a difference to me


On Sep 1, 2016, at 8:06 PM, Matt Moore innotecture@... [sikmleaders] <sikmleaders@...> wrote:

 

Hi,

I was looking at a list of content / collaboration functionality today and it mentioned "document rating". The initial model was Amazon (stars out of 5) but Twitter/Facebook ("like") is probably more prevalent now. This allows end users to rate what they think of a document, page, etc. Now I've seen this appear n a lot of functional requirements over the years but when it's implemented, it's not heavily used.

I'm curious to hear other people's experience. Are there situations where people have been rating content and others have been consuming that?

Regards,

Matt