Tag: wikis

We recently spoke with Jack Herrick, founder of wikiHow, about how wikiHow’s quality differs from a site like eHow, which is often labeled a content farm with suspect quality. Herrick’s insight on this is particularly interesting, given that he used to run eHow, before selling to Demand Media. 

Herrick has now shared more information with us about the quality control process at wikiHow. We’ll let you be the judge on how effective it is. 

"One of the apparent mysteries of wikis like wikiHow is how they can produce such high quality content while simultaneously allowing anyone at any time to edit almost any page," says Herrick. "You can go to wikiHow or Wikipedia right now, press edit, remove everything and write ‘I like cheese’ and press save and that is what will be published, at least for a short while.

He even invited me to give it a try. So I did, at 10:02am. 

"Why doesn’t ‘I like cheese’ stay on the page for days and days? The answer is twofold," he says. "First of all, there is a group of passionate people working hard day in and day out to make sure that every part of wikiHow is the best that it can be. Secondly, the wiki technology provides them with powerful tools, including a system of checks and balances, which helps them ensure that every edit that stays on wikiHow improves its overall quality."

"The wikiHow community thinks of wikiHow as a second home," Herrick tells us. "Within wikiHow they accomplish tasks, organize, clear lists of things to do, and socialize with like minded people. The people who contribute on our site pride themselves deeply in the work they do. It is hard for some people to imagine why people would dedicate so much time and do so much work when they are not being compensated financially. But, it is often financial compensation that muddies people’s motivations to accomplish goals. (See Daniel Pink’s book ‘Drive’ if you are curious to learn more about how money can pollute motivation.) Each individual in our community completes tasks because it satisfies them on a deep personal level. As an analogy, every individual prides himself on something different –  like taking great care of their dogs, fine tuning their snowboarding skills, learning to play an instrument, or being artistic. The people on wikiHow pride themselves on building and maintaining the best online how to manual they can. As a result, they take quality very seriously."

"The wiki itself is a busy place, sort of like a busy city, where people are all doing different jobs that make the city function," Herrick continues. "There are tons of activities that people can do, some of which are quite social, all which help the wiki become better. Some examples are: categorizing articles, curating images, selecting videos, expanding articles, clearing out lists of articles that need attention, fact checking, answering how-to requests, writing articles, copyediting, formatting, reviewing edits, etc."

"One of these jobs is wikiHow’s first layer of defense against bad edits," he says. "Typically within an hour of an edit being made, a volunteer wikiHow editor reviews the edit in an app called Recent Changes Patrol. Think of Recent Changes Patrol as a social app that helps volunteer editors review edits made by other contributors."

Herrick says volunteers review edits one at a time, and that they take one of the following four actions:

1. Improve an edit: Editors make quick changes to fix grammar errors or polish obtuse text.

2. Rollback vandalism and nonsense. Open editing attracts a fair amount of ludicrous edits. Fortunately, volunteer editors typically revert such nonsense within minutes. This is where the ‘I like cheese’ edit would get addressed.

3. Mark the edit as patrolled without changing it.

4. Leave a message for the editor asking for further improvements or clarifications, or complimenting them on what a great job they did.

Here is what the Recent Patrol Patrol app looks like: 

"In the case of the edit you see here, this edit which smells like link building would almost certainly get rolled back," says Herrick. 

"While wikiHow has 300,000 registered users editing on the site, most of them are not active recent changes patrollers," he says. "This is probably a good thing! Typically on any given week less than 50 very dedicated patrollers will do the majority of the work. The power patrollers do a heck of a lot though: the top patrollers will review thousands of edits each week. They get a lot of satisfaction out of the work they do. And, as you can imagine, wikiHow also has a system to patrol the patrollers to make sure they are doing a good job. If not, they will receive coaching."

There are still more layers of quality control. A second layer, he says is a watchlist system that notifies people via email (and on wikiHow) when an article they are following gets changed. 

"When a person starts an article, they’re given the option to get notified whenever it gets edited," he explains. "On top of that, anyone can watch any page on wikiHow, whether they wrote it or not. So, when a page gets edited poorly, it’s likely that someone subscribing to watchlist emails will fix the page when they check their inbox."

"A third layer of quality control is a system called accuracy patrol," he says. "Like many websites, we ask our readers to flag articles for accuracy. When we get a certain ratio of reader flags, we place the problem articles on a list for further review and improvement from editors."

While plenty of details here given here, Herrick says all of this explanation really only scratches the surface. "There is some mystery and magic to wikis that is difficult to capture in a brief description," he says. 

Do you think the system i working? DuckDuckGo, obviously thinks it works pretty well, considering they’re using wikiHow content as instant answers in their search engines. More on that from DuckDuckGo founder Gabriel Weinberg here.

By the way, at 11:17, as I’m about to click publish on this story, the page I edited still says, "I like Cheese."

Update at 11:45: It’s now back to normal as of sometime in the last few minutes. Under two hours. Not too bad. It’s hard to say with any certainty that this article didn’t play a role, however.