Knol - thinking about authority

A knol is an authoritative article about a specific topic.

http://knol.google.com/

So by its own assertion, right at the top of the Knol page, Knols are about authority.
But they've made an interesting choice, which is to attach authority to authorship.
This is a model that we understand quite well, since it is the classic Britannica model.

So if you want to organise knowledge this way, it's quite easy, you get Learned Persons to write articles in their areas of expertise.  There are, however, multiple problems with this approach:

1. You actually have to get them to write.
2. You get a tremendous management problem as you try to scale out from a handful of articles to millions.
3. Just because someone is very learned in one area (e.g. Ph.D. in Physics with accompanying publications) doesn't necessarily mean that he or she has any expertise whatsoever in another area (e.g. climate change).
4. We only have "authority structures" for a small number of areas (related to issue #2 above).  You can probably find an authority in Evidence-Based Medicine (and indeed the Knol on that topic is quite learned).  But who is the authority on say... Gnolls, obscure mythical creatures?

So Google has created an odd hybrid system.  It's seeded with some articles mostly from medicine, but it's open to anyone.  A system that claims authoritativeness, with no mechanism to verify this, other than a weak name-verification procedure.

Uniquely identifying authors (and producing a global authority file, to use library terminology) would have been an interesting problem to address, but it is not dealt with.

The actual problem they appear to be trying to address is not one of authority, but of certification.  They're focusing on credentials, when the focus should be on proof.  As best I can deduce the argument, it goes something like "on Wikipedia anyone can author, anyone can make changes, and anyone can challenge anything".

Part of this is simply a cultural issue.  There is a culture war in America, and partisans on either side use sophistry to try to prop up their arguments, leading to endless rounds of dispute.  However Wikipedia has at its core the most elegant of mechanisms to address this:

{fact}

This tiny bit of wikicode is so foundational that I think it perhaps gets overlooked.  What it does is put in a link that says "citation needed".  This little piece takes Wikipedia from an open brawl, to one of the most powerful engines promoting scientific thinking in our time.  It says simply: "I don't care who you are, demonstrate with evidence that your statement is true".

To retreat from this is to retreat from reasoned discourse.

Why?  Well, I have to turn to Wikipedia

When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong.

Clarke's First Law - Wikipedia: Clarke's three laws

(You can also amuse yourself with Wikiquote: Incorrect predictions.)

To put it another way: authority is a weak predictor of truth.  Only evidence is a strong indication of truth.

This brings me to the another flaw in Knol - the ability to cite evidence is laughably weak.
The Knol idea of a reference is... a URL with a title, that is autonumbered.  That's it.
Let's compare: Wikipedia has an entire page on Citing sources, and an entire page of metadata-rich Citation templates.  Knol has

Search Results:
No results found for citation

or more generously, it has a help page on Adding References with the example

Are you kidding me?  It's 2008 and scientists are enthusiastic about creating a semantically-rich web, and you provide a box to construct a numbered A HREF?  No DOI input?  No bibliographic import or awareness of bibliographic metadata whatsoever?  Where's the authority in that?

Even worse, rather than the single-article Wikipedia model, where everyone must work towards a consensus, Knol encourages a marketplace of competing isolated viewpoints:

What if someone else has already written an article on that subject?

No problem, you can still write your own article. In fact, the Knol project is a forum for encouraging individual voices and perspectives on topics. As mentioned, no one else can edit your knol (unless you permit it) or mandate how you write about a topic. If you do a search on a topic, you may very well see more than one knol in the search results. Of course, people are free to disagree with you, to write their own knols, to post comments and ratings.

http://knol.google.com/k/knol/knol/Help

Great, so all the people with over a century of evidence supporting the benefits of immunization can balkanize off in their own Knol, while the nutjobs who think immunization is some government conspiracy can have their Knol, and each community can rate their respective Knol highly.  How does that help anything?

The problem is that Knol has focused on authorship, and not certification.  What demonstrates the law of gravity is true is a preponderance of evidence, not some sort of individual gravitas.

Now there certainly is something in ranking and commenting that helps us to get at certification, but the mechanism provided is simply too weak.  Certification is some combination of general review, authoritative review, ranking, and ranking of the rankers.  Certification must work on a consensus item.  You may think this is abstract, but for example in astrophysics one camp of researchers put the Hubble Constant at a much higher value than other group, and each research "camp" would consistently release research supporting their number.  I can tell you this in part, not because I have a perfect memory, but because I can extract

The value of the Hubble constant was the topic of a long and rather bitter controversy between Gérard de Vaucouleurs who claimed the value was around 100 and Allan Sandage who claimed the value was near 50.

from the Disputes over Hubble's constant section of the consensus article on Wikipedia about Hubble's Law

Now tell me, what do you think you would get if instead, each of these groups had created its own Knol about the Hubble constant?

SIDEBAR: Problems with Knol

• There is no way to permalink to a comment.  If you have lots of comments, you will have to page through them to find what you want.
• There is no way to see all the comments that a person has posted, that is, to get from their profile to their comments.
• Despite it supposedly being all about you, the author, there is no way to get any page hit stats (which would tell you where you were being linked from, and what search terms were reaching your page, amongst other things).

SIDEBAR 2: Amusing Knol front page

"Who needs a search engine? Ctrl+F" line on main Knol page (i.e. telling you to use your browser's in-page search to find something... which I should mention is actually Splat-F on the Mac)

END SIDEBARS

To wrap up this discussion, let's talk about Gnolls.  (I have to admit, much of this posting was prompted simply because I was amused by the idea of doing a Knol about Gnolls.)

Search Results:
No results found for gnoll

http://knol.google.com/k/knol/system/knol/pages/Search?q=gnoll&restrict=general#

So, who is the authority about Gnolls?  Are we to search the publications of the Academy of Gnoll Studies?  And if you are the authority about Gnolls, what benefit do you get from putting up a Knol, rather than demonstrating your knowledge through blog and forum postings on sites that you control (or that are part of your community of interest), or perhaps writing a book?

Similarly, if you know some tidbit about Gnolls, where are you going to contribute that sentence?  Do you want to put it in front of millions of Wikipedia eyeballs, or do you want to attach it to a Knol that you hope will get rated and linked and searched enough that it will rise above the other Gnoll Knols so that people actually see it?

Whither the Gnoll, that sadly neglected second-tier monster, lost in the shadows of its better known competitors, Goblins and Gnomes.  A pretty obscure topic, one would think.  Yet there is the Wikipedia page

http://en.wikipedia.org/wiki/Gnoll

with paragraph after paragraph of information about Gnolls from the historical to the modern, from Lord Dunsany to World of Warcraft.  (If you want further reading, Dunsany's story is online: How Nuth Would Have Practised His Art Upon the Gnoles.) 

So this is how Wikipedia beats Knols:

• it provides a platform to develop consensus in a single article about a topic, with a focus on citations
• its citations have rich metadata
• it provides a platform to share "microknowledge"
• it provides a big enough Crowd so that you get some Wisdom
• it encourages cross-linking of topics, so you begin to see a web of knowledge

In this case, I think the Wisdom of the Crowd is going to beat the authority of the individual.

I hope that Google will learn from this and refocus to more interesting areas including:

• author identification, impact, reputation, and peer networks - and how to make those all open and portable
• much more powerful systems for ranking, rating, and discussing any web page (including a Knol) - as well as ranking the rankers
• knowledge certification
• creation of semantically-rich knowledge bases

Previously/Related:

July 23, 2008  probably the 1000th Knol post you've seen today

June 22, 2007  Community peer review in Wiki environment - Christine Chichester - June 22 - ICSTI 2007 Nancy

June 22, 2007  my presentation at ICSTI 2007 Nancy - Web tools for peer reviewers and everyone.  Also see video of this presentation (RealVideo).

In general all of the ICSTI 2007 presentations.

March 02, 2007  soft peer review

June 21, 2006  my article on peer review for Nature