A new MIT study was published discrediting some of what we may believe in “the wisdom of the crowds.” The idea is that a small group of users that share a large number of reviews can distort the quality of the reviews. I haven’t read The Wisdom of Crowds (though I just downloaded the book through Audible to listen later). I also have Crowdsourcing (saved for later listening) where the book discusses that innovation happens by opening data and resources to a large group of people.
I’m interested in the idea of crowdsourcing (or even the collective wisdom of a large group), but I immediately see its flaws. We’re often easily influenced by the reviews on Amazon or other sites, but how many of us actually go back and post our own later? I know I don’t do it very often and don’t have the time. I’ve started adding a few reviews to Yelp or TripAdvisor, but to add a quality review does take some thought and feels like work (actually I cheat and use the same review to post at both, Yelp is my favorite and TripAdvisor seems to be most popular).
Another flaw to the crowd based review system is that many people normally don’t contribute unless they really like something or really hate something. In most cases people probably post reviews based on negative experiences and not so much on positive ones. We are not really motivated to post a review of something we feel sort of neutral on. When this happens what we then see is the extreme ends of the review. This may not always be beneficial to somebody reading a review if the reviewer is showing any amount of bias through their thoughts. We know from statistics that outliers exist in every situation, this really isn’t any different for online recommender systems.
I think there’s some slight nuances between the “wisdom of crowds” and crowdsourcing though. A good example of a “crowdsourced” phenomenon is Wikipedia. A couple of posts have suggested that only about 1%-2% of the users are actually editing articles. The same sort of thing is going on here, a small number of users are contributing a very large amount of content. What’s the real difference then? Wikipedia is a community in itself, and it attracts passionate users. Since when have we ever thought of the reviews on Amazon contributed by passionate community members? Wikipedia is actually harnessing the “wisdom” of its users (even if they are a few). The community is also managed in a way to eliminate bias (at least the best it can… a librarian like myself will say don’t use Wikipedia as your only source, but I stand by what it tries to accomplish). There is actually quality stuff in Wikipedia!
Both Wikipedia and Amazon rely on crowdsourcing for content, they both are integral in explaining the Long Tail, except that where passionate users on Wikipedia can be found editing in Wikipedia, the niche markets around products sold in Amazon exist elsewhere on the web. Let’s say you’re into a band, then maybe you’re posting your thoughts and reviews on their MySpace or Facebook pages instead. Conversations are happening on another platform, and then they go to Amazon to purchase (or iTunes, or wherever). What’s left on Amazon is a filtered view in a way. Niche markets like niche tools, and Amazon is too mainstream… besides, if we really like something, we don’t want to tell the world, we just want to tell our friends.
There have been some attempts by a few services to focus on improving user reviews. Some companies have thought to remove the outlier reviews from their recommender systems (remove the best and the worst and focus on the bulk in the middle). Others have been fine tuning their systems to focus on the user and how they compare to other users with the same interests. Both are a step in the right direction, but I think they’re also missing a key component and that is the network.
There’s a lot of great services out there that people just don’t use. For example, I’m on Yelp (and do use it), but I only have a few “friends” on there. When I’m really stuck on where to go I usually turn to Twitter for help, this is where my network is. Other people may do the same but go to Facebook, or whatever service their network mostly communicates. If I’m going to proclaim that I really really like something I’m going to do it on Twitter… what if Yelp (or some other relevant service) could parse from my Twitter stream that I was at a certain place (which they could tell because I checked in via Brightkite), and then that I thought the food was really good, but the service was incredibly slow (of course based on my excessive complaining about service and praise of meal in some succession of tweets). Boom, instant review.
Ok, I realize I’m now in the fuzzy, abstract world of the Semantic Web, but it could happen. Actually I believe it will happen (that’s another blog post a few years down the road, I hope). We’re sharing the information, but we’re doing it the way we’ve always done it, in conversation, word of mouth, and with our friends. There’s wisdom out there, in large groups and small groups, I think you just have to know where to look for it.