Every time I present my research that looks at how to induce greater contributions to social computing systems, one of the first questions I get is “But what about quality?” How do we make sure that the contributions are actually worthwhile? This is an extremely important question for real social computing systems, and a number of real systems have failed not beause they didn’t have e nough contributions but because the contributions they did get weren’t very good. While none of my work has explicitly been about quality, it is closely related.
Quality of contributions is a tricky property to improve. In the end, the quality of the contributions is really up to the users who are contributing. If all of the user contributions are really low quality (e.g. trolls) then there is not much the system can do with it. There are two main methods of dealing with quality:
First, you can attempt to filter high-quality contributions from low-quality contributions. This filtering can be programatically as long as there is some method of measuring quality automatically. Or, you can have users rate others’ contributions and use those ratings to identify high-quality contributions. Amazon.com does this with its 5-star ratings; users rate each review with 1 to 5 stars, Amazon averages all these ratings and provides an overall quality rating. It then uses these ratings to better suggest new items. Slashdot also uses user-supplied ratings in its comment moderation system. Users are given a certain number of “moderator” points that they can use to vote certain comments either up or down. But user ratings also add a new user-contribution problem complete with its own contribution and quality issues; Slashdot has had to introduce “meta-moderation” where users can vote on whether other users have been using their moderator points well.
Second, rather than trying to identify the quality of contributions, it may be possible to provide an incentive for users to increase the quality of their contributions. Both the side effect mechanism and the minimum threshold mechanism can be used to encourage higher quality contributions if they are designed properly. And there might be other mechanisms that more directly deal with quality issues. This is an interesting open area for research. Certainly, any solution of the first type that tries to identify high-quality contributions and then promotes those contributions might also, indirectly, induce users to try to produce higher-quality contributions. Wikipedia allows users to give “barn stars” to other users for high-quality contributions; the rationale for these is that as a public display they encourage higher quality article writing and editing. It is interesting to see if there are any direct quality improvement mechanisms that do not also work for quantity.
Post a Comment