Commentary

Crowdsourcing: Building a Comprehensive Database with User Contributions

by Keyvan Mohajer, founder, president and chief executive officer, MELODIS

“Crowdsourcing” may be a relatively new word, but the concept has already proven itself in the Web 2.0 world. Crowdsourcing is defined as "the act of a company or institution taking a function once performed by employees and outsourcing it to an undefined (and generally large) network of people in the form of an open call." In essence it means leveraging the power of the masses, which is exactly what user-generated web sites like Wikipedia have done to great acclaim.

But crowdsourcing also has implications for another core technology of the Web 2.0 world: search engines. The ability to find information quickly and easily is key for users, yet many once-popular search engines are now in danger of falling by the wayside. What is causing their decline when user demand for effective search engine technology remains so high?

The answer lies in a fundamental oversight made by many search engine developers. These companies typically focus on the technical challenges of designing their search engines. Often they enable their products with sticky features and functions that should guarantee a loyal user base. Yet this approach doesn’t seem to work in the real world. Why? Because without a complementary database that contains the information users want to find, search engines by themselves are not useful.

Building a comprehensive database the traditional way usually means relying on a handful of highly paid experts, an expensive, time-consuming process that invariably leaves gaps in the knowledge base. Crowdsourcing offers search engine developers and other companies a faster, more efficient and cost-effective way to do the very same thing.

Instead of hiring a small number of experts, companies can now use the Internet to create large pools of knowledge by soliciting contributions from millions of people around the world. Although each individual user contributes only a little bit, the accumulation of millions of such contributions can yield a bigger, better, more reliable result than the traditional model does. Just consider the success of user-generated Wikipedia, whose search results rival those of professionally developed encyclopedias like Britannica.com.

Industry giant Google is another crowdsourcing success story. Users did not actually create Google’s enormous searchable database; however, they do contribute to it by creating web page links that allow Google to determine which search results are most relevant. And the ability to return relevant search results is what makes Google the most popular search engine in the world.

So does this mean companies that need to create large, searchable databases should simply send out an open call to the masses? Unfortunately it’s not that easy. Like individuals, crowds are inherently self-interested. In order for them to contribute to—and stay engaged with—a searchable database, they need incentives. What kind of incentives? That depends on the characteristics of the contributors you want to attract.

At MELODIS, for example, we realized that many contributors to our database of searchable music were karaoke enthusiasts who enjoyed sharing their voices or aspiring singers hoping to be discovered through our web site, www.midomi.com. To keep these users engaged, we developed a rating system that lets our users identify “star” performers on our site. We also provide social networking features that allow them to comment on other users’ performances, post photos and bios of themselves, share songs with friends, and send private messages over our network. The result is that MELODIS now has the largest database of searchable music in the world.

The first lesson, then, is that companies cannot succeed on the strength of their search engines alone. Search engines must be combined with comprehensive databases so that users can find what they need easily. Otherwise, they won’t keep coming back.

The second lesson is that in today’s Web 2.0 world, the best way to build a large database is to leverage the knowledge of the crowd. Through crowdsourcing, companies can use millions of user contributions to quickly create pools of knowledge that far exceed the scope of databases developed by subject matter experts. The trick is to offer incentives that ensure you attract the right crowd.