Science Library Pad

Google Scholar adds personal profiles displaying citations

I guess the "make everything social" decree went out to Google Scholar as well.

There is now essentially a scholarly social network built in - individual user profiles displaying citations, links to colleagues.

Today we’re introducing Google Scholar Citations: a simple way for you to compute your citation metrics and track them over time.

...

You can enable automatic addition of your newly published articles to your profile. This would instruct the Google Scholar indexing system to update your profile as it discovers new articles that are likely yours. And you can, of course, manually update your profile by adding missing articles, fixing bibliographic errors, and merging duplicate entries.

You can also create a public profile with your articles and citation metrics

Google Scholar Blog - Google Scholar Citations - July 20, 2011

There's more information at http://scholar.google.com/intl/en/scholar/citations.html

Citations is in limited release, you have to sign up on a waiting list, usual procedures.

Here is the profile for Anurag Acharya: http://scholar.google.com/citations?user=nGEWZbkAAAAJ&hl=en

Previously:
November 20, 2004  Google Scholar  (my first mention of Scholar, shortly after it was launched)

G+ Circles: pseudo-control over your push channels

G+ at least gives us lots of opportunities to talk about G+.

Summary

Google+ Circles don't come anywhere near modelling the complexity of our online interactions.

Details

One of the main topics is Circles, which people seem to have difficulty thinking through - either in creating them or understanding how they work.

This interest in circles is a bit surprising because Facebook has had Friend Lists (which are basically the exact same thing is circles) for a long time but almost no one talks about using Friend Lists.

So I have to give Google kudos for drawing attention to this type of feature, although I'm a bit puzzled why it gets so much attention since Facebook is the same, even to the extent of a per-post pulldown at the bottom where you can select the lists that will see your Facebook post.

Facebook is actually placing more emphasis on Groups now, perhaps because G+ doesn't have groups (yet anyway).

It's important to understand that Circles & Friend Lists are filtering-at-source, outbound filtering, content "channel" preselection.  There are two common scenarios for outbound pre-filtering:

1) The "NOT" scenario for personal content, e.g. I want my friends to see this picture but I do NOT want my family seeing it

2) The "I want to signal a topic, content type or urgency" scenario, e.g. I will put these photos in the Ottawa buildings group

First, it's important to understand the "NOT" scenario is a fantasy, it's pseudo-control over your content.  Basically as long as the content exists in some recorded, viewable (or hearable) format somewhere, it can be made public.  If you want private, I suggest spy-type park bench conversations with a noise screen and check your companion for a wire first.  Other than that there is no private.  There's only pseudo-private.

So for example you can push content into your "Super Top Secret" Circle with "sharing disabled" on the post, but anyone in that circle can just copy and paste or screen capture or take a photo of the screen or any of dozens of other methods of duplicating recorded content (some more work than others). 

(As a side note, this is why DRM is a fantasy, which doesn't stop attempts such as Microsoft's Information Rights Management.)

So putting your content anywhere, whether it be in an SMS, an email, a Facebook posting... you should expect it could become public at any time.  This is why I would suggest against say, creating two Google+ Circles called "Circle where I pretend I love (task)" and "Circle where I tell people I trust that I actually hate (task)".  That's just asking for trouble.

There are lots of ways people currently use entire sites or communication types as different types of channels.

Sometimes you use a channel to reach a particular group, sometimes you use it because it provides pseudo-privacy, sometimes because it signals urgency or lack thereof.  Sometimes something that seems like a very simple channel is actually very complex (the most notable example is Twitter).

For example, I use

* email: rapid ad-hoc bi-drectional "private" communication - also if I want to be sure you will see it (the TCP of messaging)

* SMS (text messaging): realtime 1-1 bi-directional "private" communication; urgent; expectation that message will interrupt someone

* Facebook: either "private" sharing with a group (friends) or sharing-because-it's-where-everyone-is for e.g. work people who don't use other channels - but with no real expectation everyone will see it (the UDP of messaging)

* Yammer: in theory for sharing with closed work group of people just at my department (I say in theory because there is not enough use to see if it works well)

* Twitter: public stuff of interest with no expectation everyone will see it (UDP) plus signals of topics (hashtags) AND signals of per-individual interest (@ someone is public way of signalling, DM says this is both important and "private") AND accounts for different types of audiences (e.g. some people use different accounts for regular tweeting and livetweeting, or different accounts for different topics)

* blog: a mix of long-form ideas (with no expectation they will be read) and link sharing (for people who don't follow other channels)

* Topic or participant-specific groups: many tools (Facebook, LinkedIn etc. etc.) provide this feature where people with a common interest or some other factor in common can have a shared space.

There are channels that I never use, just due to the particular group of people I communicate with and history (e.g. use of instant messaging grew long after I was in the habit of emailing friends).

The short overview above is just to make the point that there is a lot more nuance to "the subtlety and substance of real-world interactions" as Google puts it than just controlling what channel you push into.  For work content, public with at-destination filtering (e.g. publish & subscribe) may make more sense as I usually don't know a priori per-individual what content may be of interest, and I'm usually not engaged in pseudo-hiding content from one group or another.

In summary, at the moment Circles only makes sense to me as a way to NOT share content with selected individuals, which is more of a Facebook use case, but be mindful that at best this is just the appearance of not sharing content; it can be copied and made public at any time.  The single most notable absence in Google+ is the lack of groups.  Since Circles are not just per-source filtering, but fully or partially hidden per-source filtering (you can't see who is in someone else's groups, or even their group names), Circles doesn't help people form interest groups.  This is kind of weird since one of the aspects of the Internet is that, far from being the anti-social medium portrayed sometimes in the media, the Internet leads almost naturally, organically, to people forming discussions around common topics of interest.  Google does of course have an entire Groups product, called, well, Google Groups, which is an inheritor of the USENET tradition of granular per-topic discussions.

There are a lot of ways that Google could make this experience easier:

* first and foremost Google+ needs to provide Group functionality - just a next-generation version of Google Groups would probably do this

* use GMail data to suggest groups (this technology has already been built by Google, it will suggest possible "missing people" when you compose an email, based on people you usually email together)

- similarly use GReader and GBuzz data

* partner with LinkedIn to suggest groups - either based on connections (see my post LinkedIn maps my connections on LinkedIn clustering) and/or based on existing LinkedIn groups

* extract communication patterns from Twitter to suggest groups, also allow import/conversion of Twitter lists

* alllow various location-based groupings including: location groups based on profile location (or past location) information; location groups based on location detection in browser; location groups based on topic declarations (for example I would like to be able to share information just of local interest to Ottawa residents, but there's no way I'm going through every single person who follows me and figuring out who lives in the Ottawa area)

* allow recipient (destination) filtering by content type and content source, so I can e.g. choose to see all your posted external links but none of your photos or status messages (Facebook provides this to a limited extent and Friendfeed provides this for every separate content source it knows of)

* statistics so you can discover who is using & sharing and "+1ing" your content and can adjust your channel models accordingly

* pull channels (e.g. pull in content from Twitter, perhaps filtered by hashtag)

Ultimately the point is that our interactions are much richer than a flat communication model - but Google has provided only one small aspect, pre-filtering at source, out of a much much larger set of source, topic, and urgency filters.  Pre-filtering at source is mainly use for (an illusion of) personal information control, not for work sharing.  Google+ does not come anywhere near modelling the way people currently signal public/private, topics and urgency by their choice of channels.

There is a great post that explains this much better than I have above: Dave Gray - Sharing universe.

Google+ is social signals data for search

I think it's a mistake to think of Google+ as a Facebook replacement or even as an attempt to replace Facebook.

Google has a basic problem: they make their money from search ads.  Mostly from search ads when people are looking to buy something.  So a search like "best 2009 used cars" represents a ton of money for them.  A Facebook posting "hey guys, can anyone recommend a good 2009 used car?" is a disaster for Google.  Not only is it probably invisible to their indexing engine, it connects people to information without search intermediation that you can attach ads to.

So Google has two problems: 1) people are moving out of its search-driven search space into one-to-many questions to selected groups of people 2) Google doesn't have a lot of rich information about the social context of people who are searching.  Things Google might want to know: how connected is the searcher, how influential, how wealthy, what interests does she have, what content does she share...

Some people stay logged in to Google (GMail) all day, but this trackable identity can be lost if e.g. the user is reading GMail using iPad mail app, or if like me they only login to GMail web for a few minutes, check email, and then log out.

Google would much rather have a single, perpetual, always-logged-in identity.  A social network with a notification bar encourages people to stay logged in, lest they miss a notifcation.

Google is a data-driven organisation.  So let us imagine that through Google+ "all" Google gets is a complete map of the social graph of all the top tech influencers in the world, along with the kinds of things they like to share, and their interests (their "sparks" in Google+ terminology).  And along with that, Google gets social graphs for a few million other people.  Even if G+ then only limps along, Google now has much better data to analyse for its core search business.

So I think Google+ is mainly about providing Google with enormous amounts of data that it can analyse to determine social signals for search, to understand Q&A social behaviour, to find out how people are grouped and interconnected, and to have data to drive social driven re-ranking and display.  But most importantly by far, it gives Google the beginnings of data to optimise ads for the social graph.  Would you rather pay 1 cent to display your ad to someone who has zero tech influence, or $1000 to get your ad in front of the eyeballs of a tech influencer whose posts are reposted and retweeted thousands of times?  Would you rather pay 1 cent to display an ad to random people based on search keywords, or $100 to display the ad to a "circle" of people who have demonstrated a sustained interconnected interest in your particular topic?

This post inspired by GigaOm: Why Google+ won’t hurt Facebook, but Skype will hate it found via @nicholemcgill

Previously:
July 1, 2011  Google Plus

Google and the web of structured data

Google has announced it will be using (some) microformats and RDFa to enrich search results.  They call this Rich Snippets.

To display Rich Snippets, Google looks for markup formats (microformats and RDFa) that you can easily add to your own web pages. In most cases, it's as quick as wrapping the existing data on your web pages with some additional tags.

Official Google Webmaster Blog - Introducing Rich Snippets - May 12, 2009

O'Reilly Radar says

Moving toward the Semantic Web will allow our searching technologies to become more intelligent and will set the stage for the next revolution in which computing systems can become more aware of the "meaningfulness of data".

We've already seen a shift toward "semantic search": Google has already been augmenting search results with Google Maps, limited catalog searches, and more recent entries into the search market such as Amazon's A9 and the yet to be released Wolfram Alpha differentiate themselves by the structured data and content that can be extracted from a search result. We have yet to a see a compelling reason for web masters to place RDFa or microformats into a site to enable this semantic data to be mined until today, until Google provided a social incentive for site designers.

O'Reilly Radar - Google Announces Support for Microformats and RDFa - May 12, 2009

Google has a help file to get you started: Marking up structured data.

Incidentally, if you're thinking, "why didn't someone tell me this structured data thing was coming?" I should mention that actually I did try to tell people, whether it was in my presentation to Allen Press in 2007 (where I talked about the need for microformats and semantic enrichment) or in my keynote to NISO Discovery last year (where I talked specifically about Yahoo SearchMonkey using semantic information).

Previously:
September 8, 2008  semantic search thoughts

Wolfram Alpha and the web of structured data

The good news is, WolframAlpha can understand a query like "unemployment in Ottawa".

The bad news is, as you can see in the Result, that while it knows what data it needs to compute the answer, "(data not available)".

This is as clear an example of why we need open data as I can think of.  If you want your computers to start providing you with smarter answers, you have to give them better information to work with.

WolframAlpha provides two channels to try to address this problem: Contribute Structured Data and Suggest Data Sources.  You can also contribute individual facts, but that won't scale very well.

in which I realise Twitter can make money

(I was going to say something about "monetization" but that term is getting a bit old.  Can I make it through the entire post with out using it?  Let's see...)

Ok, social networks are hard places to make money.  Sure you know a lot about the users, but they're busy flirting with one another or posting pictures of their cats or digging up embarassing images of people passed out last night.  They're not searching for information, they're interacting with people.  Trying to wedge commerce into friendship, as Facebook is trying to do, trying to make money from social interactions - just doesn't work well.  Wrong mode.  See previous posting the difficulty of (making money from) Web 2.0 eyeballs.

So I thought Twitter was probably hopeless in terms of a business.
But then I realized that Twitter is also real-time search.
And search we do know how to make money on.  Boy howdy do we.  To the tune of billions of dollars.

So Twitter actually has it easy.

1. Do the usual adsense matching ads to keywords thing on search.twitter.com
2. When you can't parse the keywords well (as may be the case with hashtags, e.g. #lotf09), build a metadata directory to figure out what the hashtag maps to and then match the ads to that - in which case, ideas like tagalus are going to be worth lots of money - you heard it here first

So Twitter should be fine.  Should be big dollars fine, in fact, as real-time search grows in popularity.

This is also a very simple lesson to all you Twitter-wannabes out there (are you listening, Facebook and FriendFeed?) : you want to make money from the real-time web dynamic whatever - fine, just do it in a separate search interface, not in your main user interface.  When they're in separate search, you know they have intentionality.  Intentionality, you can make money from.  When people are expressing a desire, you can use ads to meet that desire.  That's like, probably the whole underlying concept of ads (yeah, I watch Mad Men).

Whereas ads while people are socially sharing links, pictures, and whatever, not so much intentional seeking, not so much any opportunity to make money.

UPDATE 2009-04-07: I should also mention a current approach that Twitter is experimenting with

3. Subject-specific portals

Since Twitter provides aggregated real-time conversations, you can try to find various ways to slice the conversations up by topic or other  attribute.  The two approaches I am aware of so far are:

* to slice by "authority" or identity - ExecTweets is a portal to "executive" Tweeters (which I think is a bit nonsense, in an age in which authority emanates from your ideas, not your position) - the portal is sponsored by Microsoft - there are other portals that slice out particular celebrity twits etc.

* to slice by content - MarchTweetness/TitleTweets is a portal around the US March Madness basketball thing, sponsored by AT&T - one can imagine a similar soccer-oriented site for the international audience

So one can imagine e.g. a site built around the launch of the SF series Caprica, that captures all the #caprica tweets and attempts to target people interested in science fiction, selling BSG merchandise etc.

I personally think this idea is pretty weak overall, because subject-specific portals generally fail.  Google beat Yahoo.  People jump from topic to topic as they explore the web, as the impulse strikes them.  No matter how much info your portal pulls in, there's always going to be some rival portal on the same topic that people want to check as well.  You cannot hold people on a specific website.  Also, unless you do this in an automated fashion for all topics (in which case it will look generic), it is a lot of work to do a custom siloed portal for each and every particular topic you want to make money from.

I can imagine though they will continue the TitleTweets trend of searching for the "big eyeball" sweet spots like the SuperBowl for custom Twitter portals.

Twitter modes: real-time news and event reporting, topic rendezvous, mobile and power interfaces

There is one core use-case for Twitter which I have discussed before, that of using Twitter as a "social network maintenance" tool, to engage in an ongoing conversation with friends and colleagues, across distance and time (some of the people I tweet with are in England, others in California).

In discussion on FriendFeed, one of those far-away contacts, Deepak, reminds me that I should discuss some of the other modes, the other use-cases for Twitter.

1) Real-time news: if something happens, you should be able to pick out reports about it (with the right keywords) by using search.twitter.com

2) Similarly, through the use of hashtags, you can get reports from events and about topics.

For example, if you want to know what people are saying at the UK Serials Group conference, a search of #uksg09 will help (I don't know what you will find there, as at the time of this writing, the March 30 event hasn't started yet - right now you just see people doing their final arrangements to get there).

If you want to see a view post-event, try e.g. #jisc09 to see what people had to say during and after the annual JISC conference.

You can also see an example of this in my previous posting, about the Twittering of Obama's first visit to Canada.

And it was by watching event reporting on the #ali social media conference in Ottawa on Twitter that I got connected into the community that is now working on #changecampottawa.  In fact, a lot of the ChangeCamp Ottawa planning and information sharing takes place through Twitter.

Likewise, if you're interested in a particular topic, say #nasa space programs, a search will lead you to other interested people and links about that topic.

You can see a small list of Ottawa social media tags in my posting connecting to Ottawa social media community through Twitter.

3) Mobile and the constraints it brings

A big deal is made out of "only 140 characters".  Well, keep in mind that it's possible the desktop computer may one day be considered as obsolete as the deskset telephone.  For many people, their entire telephone experience is mobile, and it may be that our main computer interface in the US and Canada becomes mobile as well, as it already is in other places in the world.  Mobile means two things: small screens and terrible keyboards.  As anyone who is unskilled in cellphone text entry can attest, 140 characters starts to look like a lot when you're trying to use a numeric keyboard to enter words.  As well, trying to read more than a sentence or two at a time on many cellphone screens is simply not possible.

The reason both Twitter and Facebook have short status messages, is that this supports SMS as transport, and that many of the users only ever see those messages through the tiny window of a cellphone screen; they never use big screens on desktop computers for their social networking.

4) Other interfaces

The Twitter web interface is pretty basic.  There are better clients on mobile phones and on desktops.  In particular, TweetDeck is one popular power-user Twitter interface.  Here's a selected view showing some live searches (you can click through to Flickr then select All Sizes to see it at original size):

Using Twitter in this way is really a qualitatively different experience.  Not only is it much easier to do some things (such as retweet), you also can get a big picture of what is going on - incoming filtered tweets next to your incoming @ replies, next to direct messages, with the option to have people sorted into columns by group (work, personal, etc.) as well as to be continuously running realtime searches on topics of interest.  This is a very powerful window on the world of information, if you want to be grand about it, it's a sort of realtime world dashboard.

UPDATE: 5) Search and the Twitter ecosystem

I should mention a couple things:

First about search -

* Twitter search is (mostly) an exact match.  If you search e.g. social, you won't see hits for socialize.

* If you use Twitter search from the home page (or add &source=serp to the search URL), Twitter will show matching user accounts as well, and those are sliding matches.  e.g. here is a search on #gov that shows matching accounts for various US Governors.

Secondly, because Twitter was smart enough to provide an API, a huge ecosystem has grown up around Twitter - this includes tools I have mentioned before, but many many more.  See e.g. this giant list.

Change-tweets come to Ottawa

(A post rescued from draft, somewhat dated but helps to explain a bit about the aspect of Twitter for news and event reporting.)

This graph from TwitScoop shows the activity on Twitter for the hashtag #obamawa - the tag people used for President Obama's visit to Ottawa on February 19, 2009.  Graph captured around 4 PM.  You can see a peak as he arrives in Ottawa, specifically when he gets to Parliament Hill, and a second peak as the joint press conference takes place.

The use of hashtags and the powerful near-realtime search provided by search.twitter.com is a powerful engine for connecting people together who would otherwise never communicate, and for connecting across many different communities.  If you haven't been paying attention to Twitter, you may be surprised at a sampling of the news and government organisations that made postings ("tweets"):

• the US embassy in Ottawa - yes, the real US embassy officially live-tweeted the visit, including the visit to Byward Market
  
• CBC
• TVO's The Agenda

as well of course, many Canadian citizens just going about their lives in Ottawa and elsewhere posted their thoughts.  Whatever you may think about some of the trivialities that people post on Twitter (which are to be expected, since some of Twitter is small talk, which is usually not exactly groundshaking), I think you will agree that the first visit of President Obama to Canada was an event worth people's time and online words.

Here's what the scoop looked like at 5:30 PM that day

If we're talking about laments for democracy and citizenship and news reporting - what this means is people were engaged, they were excited, and they were participating in a shared experience as equals with major news organisations. This is what democracy is all about. I'm not saying that means they have an equal level of expertise, I'm saying the technology enables them to have the same level of participation, and to share their own observations.  Even if we wanted citizens to be passive consumers of broadcast news (and surely we didn't?) that age is gone.  The new era is about the newsmakers, the newsmedia, and the newscitizenry all engaging in a discussion together.  Surely that's a more profound engagement in our own democracy than sitting on a couch watching Peter Mansbridge, or skimming the Globe over morning coffee and then tossing it in the recycling bin.

CISTI in Google Scholar and Facebook - official news

I always have to walk a fine line because it takes a while for official messages to be approved in the Government of Canada. We've got some approved releases, so therefore I'm happy to now be able to safely enter

<marketing-mode>

Ottawa, ON, March 23, 2009 – NRC Canada Institute for Scientific and Technical Information (NRC-CISTI) is meeting the research community where they work through our new collaboration with Google Scholar.

CISTI Media Room - Google Scholar

Ottawa, ON, March 23, 2009 – In an effort to generate awareness about the products and services offered by NRC Canada Institute for Scientific and Technical Information (NRC-CISTI), NRC-CISTI is now on Facebook.

CISTI Media Room - NRC-CISTI on Facebook

</marketing-mode>

Libraries in Computers - March 31, 2009

I was thinking about the post title and it occurs to me - no offense to InfoToday - that "Computers in Libraries" is kind of an old way of thinking about things.  The reality today is Libraries in Computers.
This is not a new idea - Mark Weiser was writing about ubiquitous computing in Scientific American in 1991.

The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.

Is your library becoming a part of your patrons' everyday life?  All this to introduce CIL 2009 session

D203: Embedding Services: Go Where the Client Is

coming up on March 31, featuring my colleague Natalie Collins.  She'll talk about the process that got us to "allowing discovery to happen in the user’s regular work environment".  If you want more details, a previous post about a thousand words may give a hint.

a thousand words?

http://scholar.google.ca/scholar?q=a+thousand+words&hl=en&lr=

Note: AFAIK you must use either scholar.google.ca or be in Canada to see all aspects of this result.

UPDATE: If you want a search that will take you more directly to the PDF option http://scholar.google.ca/scholar?q=Simulations+of+the+2.5D+inviscid+primitive+equations+in+a+limited+domain&hl=en&lr=&btnG=Search

UPDATE 2009-02-27: If you want to see a third type of result, try http://scholar.google.ca/scholar?q=Biochemistry+and+molecular+cell+biology+of+diabetic+complications&hl=en&lr=

10 years of Google

TVO's The Agenda with Steve Paikin had a good segment on 10 years of Google.

You can watch Flash video of it on the site,
they also have podcasts in audio and video flavours.

You can also follow them on Twitter

twitter.com/theagenda

twitter.com/spaikin

semantic search thoughts

Thomas Tague has asked for input to help better define semantic search.

So, let’s deconstruct semantic search into it’s constituent components and talk a bit about how and whether semantic technologies might actually make it better.  The results of the dissection are here on the table….

   1. What kinds of questions can we ask? Can we embed logic in our questions? Do we expect inference in our results?
   2. How can we ask them – keywords, natural language and all that jazz.
   3. Generating the “right” result set for the query.
   4. Displaying the result set in the most effective manner
   5. Making money from doing all that

Here at CISTI we see Semantic Search as a key advanced technology to help the library support researchers, and in particular to help researchers get at the meaning locked up in our millions of locally-loaded journal articles.

We think of it as three pieces:

1. Domain-specific semantic extraction, in which the different types of scientific information are identified and indexed for search (e.g. chemical names, mathematical formulas, genes, etc.)
2. Straightforward search frontends, which use basic keyword mapping to provide an enhanced discovery interface (e.g. you search for water and you get any documents with the various possible representations for water)
3. Semantic search frontends (possibly also domain-specific) in which a researcher enters a search in terminology that's meaningful to them - for example drawing a chemical structure, or asking a complex question (e.g. what X are affected by Y when conditions A and B hold)

For all of these areas we see partnering as a key, as no single organisation is going to have the domain-specific expertise necessary, or the technology expertise necessary.

I'd be happy to hear other opinions on semantic search, particularly as it relates to science.
Should we separate it into several subareas?  Perhaps

• Semantic embedding (the document is already semantically enriched)
• semantic extraction (terms are extracted from a document that isn't semantically encoded)
• semantic indexing (constructing search indexes using semantic information)
• semantic search (search engines that use semantic information)

I see I don't even have a category for Semantic Web, so I guess it's time to make one.

in which net searching gets doubly disrupted

Internet searching is by no means a stable space.
Remember simple old Google?  Type in keyword, get a link?

Well now it's type in keyword, get suggested related searches, a link, a bunch of subpages of the link, and (sometimes) a site-specific search box.

As if that wasn't enough, along comes Firefox 3, with character matching in both URLs and page titles, when you type in the main URL bar.

Is Firefox 3 matching going to mean less searches sent to Google, as people navigate within their own set of URLs?  Is it going to change the page titles we use, so that we get better matches?  Why does it rank VALA ahead of ALA in the pattern match?

availability, discovery, and delivery - redux

• Availability does not equal accessibility: researchers’ top concern about scholarly communication is that they cannot access all the content they wish to access
• Researchers tend to use tried-and-tested discovery tools, or those which their library specifically trains them to use. Google and other web search engines remain the most-used search tools for work-related information. The main problem with discovery is coming up against an access barrier
• Researchers do not always know how to seek out a freely-available copy of an article that they want and which they have discovered behind a toll barrier

Key concerns within the scholarly communication process: report to the JISC Scholarly Communication Group, March 2008 [Word document]

via Lorcan Dempsey

I find it interesting that the focus of concerns is around delivery, not discovery (perhaps this is how the questions were framed).

I think the academic library faces two challenges:

1. Ensure that your researchers can always get from their chosen discovery environment easily to
1. Get to your licensed resources
2. Get to free copies, if licensed versions aren't available
3. Get to purchase options, if free copies aren't available
2. Ensure that as many of your resources as possible can actually be discovered

I'm not convinced that we're doing a particularly good job of addressing these fundamental challenges even after years of working on proxies, federated search, link resolvers, and "live in your environment" plugins and external website settings.

It seems to me that librarians were so focused on trying to control the discovery experience, trying to make people discover resources "properly", following established librarian search protocols, that the simple challenges above were not addressed.

I think we need to spend a lot of time with researchers letting them search however they want, and seeing whether they dead-end either by not being able to get to a resource at all, or by landing at a paywall for a resource we license or can get to for free.  Fix that first.

And I'm not entirely convinced we have all the tools we need to fix that problem right now.

Then once you have addressed consistent delivery, work on improving discovery.  I think discovery is much harder to fix.  And to some extent, there should be vendor pushback.  I don't care how rich or comprehensive a licensed resource is, if my users can never discover it, then the message to the vendor should be "enable easy ways for my user to discover your resources within their preferred discovery environments, or next year we're not licensing your content".

Previously:
Lorcan and I had a bit of a back-and-forth about discovery and delivery in 2006.

August 08, 2006  Discovery and disclosure - Lorcan Dempsey
August 09, 2006  online library role in discovering and delivering - Science Library Pad

Yahoo - the SearchMonkey cometh

Well, as long as it isn't a flying monkey, ok.

Enter the Yahoo Open Strategy (YOS). ...

There’s a massive, latent social network within Yahoo, and we’re going to bring it to the surface. We’re making Yahoo more social, but we’re not building yet another social network. We already have an incredible social network… we just need to unlock it.

...

A first taste of our strategy is SearchMonkey, which will let developers mash up helpful data with our search engine results.  ... launch party May 15 [2008]

http://ycorpblog.com/2008/04/24/developer-welcome-mat/
(Exclamations removed because I refuse to put exclamations all over the place, including inside of acronyms.)

SearchMonkey is what I previously described as "semantically-enriched search results".

General story of Yahoo Open Strategy very widely reported, e.g. Globe - Associated Press - Yahoo plans social makeover.  SearchMonkey bit via Search Engine Watch blog.

Building SkyNet for Science - presentation for NISO Discovery Tools Forum

My presentation is available at

http://www.slideshare.net/scilib/building-skynet-for-science-discovering-new-frontiers-using-embedded-knowledge/

A lot of it is conceptual, so you may want to wait until the audio is available from the http://www.niso.org/news/events_workshops/discovery08/ NISO site - http://www.niso.org/news/events/2008/discovery08/ (hopefully next week).

UPDATE 2009-04-16: I don't think audio was made available.  ENDUPDATE

UPDATE 2008-03-28: I forgot to mention that all some of the supporting links for the presentation (will be) are available at http://www.connotea.org/user/scilib/tag/nisodiscovery2008  ENDUPDATE

I thought it went well, although as first speaker up there is a disadvantage of not seeing how other people set up.  I was in a bit of a rush to get started so that I would finish on time, so I didn't do a great job of attaching my mike and I just held the wireless transmitter in my hand.  With the transmitter in one hand and my laser pointer gripped in the other for the entire 50 minutes, it's possible I looked a bit of a prat.

I usually try to remember to keep my hands free for presentations so that I can use more natural body language, anyway lesson learned.

I also forgot to say "The future is not set.  There's no fate but what we make for ourselves." before the last slide.

There are some common themes emerging from the presentations, I'm always amazed when a bunch of people develop presentations in isolation and then they actually all fit together when presented.

I've posted some photos of the presenters in my Flickr under nisodiscovery2008 (my cameraphone can upload directly to Flickr over WiFi), they also show up because of the machine tag linkage on the Upcoming page.  No pics of Chapel Hill yet as I don't have a car and it turns out while we're only about 4km from the town, the most direct route for me to get there I think would be to walk beside a six-lane divided highway, which is not too appealing.

UPDATE 2008-03-28: The carbon offset for my flights from myclimate.org (including the trip to Open Repositories) was about C$118.

Google Book Search API

The Google Book Search Book Viewability API enables developers to:

                   

• Link to Books in Google Book Search using ISBNs, LCCNs, and OCLC numbers
• Know whether Google Book Search has a specific title and what the viewability of that title is
• Generate links to a thumbnail of the cover of a book
• Generate links to an informational page about a book
• Generate links to a preview of a book

http://code.google.com/apis/books/

via LibraryThing blog - Google Books in LibraryThing - March 13, 2008

We need APIs everywhere for everything.

Semantically-enriched search results coming from Yahoo

In an upcoming talk I will be continuing a theme I started at Allen Press, calling for more semantic enrichment of scientific information online (I am of course, only one of many making such calls).

It is therefore timely to see Yahoo offering an open platform for harvesting and returning semantically-enhanced search.

There was a pre-announcement on TechCrunch, followed by the official word on the Yahoo Search Blog

In the coming weeks, we'll be releasing more detailed specifications that will describe our support of semantic web standards. Initially, we plan to support a number of microformats, including hCard, hCalendar, hReview, hAtom, and XFN. Yahoo! Search will work with the web community to evolve the vocabulary framework for embedding structured data. For starters, we plan to support vocabulary components from Dublin Core, Creative Commons, FOAF, GeoRSS, MediaRSS, and others based on feedback. And, we will support RDFa and eRDF markup to embed these into existing HTML pages. Finally, we are announcing support for the OpenSearch specification, with extensions for structured queries to deep web data sources.

Yahoo Search Blog - The Yahoo! Search Open Ecosystem - March 13, 2008

You can sign up for more information at

http://tools.search.yahoo.com/newsearch/open.html

So what would an appropriate set of semantic information be for a scientific article, what would your ideal search display include?  # of citations?  Impact Factor?  Chemical and gene sequences?  Price?  (Sometimes information wants to be expensive...)  How much can we fit into a couple of lines that will help to select one article over another in results?

UPDATE: And Yahoo is just one player in this space, as Paul Miller indicates in his posting Looking for a dominant Semantic Web search engine.

via Twitter mostly

Next Generation Discovery - NISO Forum - March 2008

http://www.niso.org/news/events_workshops/discovery08/ Next Generation Discovery: New Tools, Aging Standards - http://www.niso.org/news/events/2008/discovery08/
March 27-28, 2008
Chapel Hill, NC

Discovering scholarly information and data is essential for research and use of the content that the information community is producing and making available. The development of knowledge bases, web systems, repositories, and other sources for this information brings the need for effective discovery -- search-driven discovery and network (or browse) driven discovery -- tools to the forefront. With new tools and systems emerging, however, are standards keeping pace with the next generation of tools?

I will be presenting.
I also (after fighting with Yahoo accounts) managed to make an Upcoming event.

Other confirmed presenters include Peter Murray, Karen Hawkins (talking about scitopia.org), and Eric Schnell & Dave Munger (talking about researchblogging.org and the BPR3 initiative).

I'm proposing a tag: nisodiscovery2008

UPDATE 2008-03-10: And a Twitter Hashtag #ndf08

Retrevo - searching for reviews

I know the search engine experts out there can probably list dozens of similar sites, but I liked the execution of this one: Retrevo.

If you search Google, you get a flat information space, which you have to narrow with keywords.  I like Google a lot for bringing relevant results to the top - but if it doesn't know why you're searching, of course it has problems.  In particular, if you search for just a particular part number, it will often list page after page of sites selling that item, with very few links to reviews or blog postings.

Retrevo specifically separates its results out into "Reviews & Articles", "Forums & Blogs", and "Shopping".  Nicely done.  Of course, the fact it ranks me first doesn't hurt...

http://www.retrevo.com/s/dpl700?sub.x=0&sub.y=0&sub=Search

Internet Archive 20th Century Search - DLF developers' preconference - Nov 3, 2007

Before the main event there was a preconference (with a fraction of the main attendees) exploring technical challenges and possible collaborations.  There were also a couple brief presentations, here are my raw notes on Kris Carpenter Negulescu, the Director of the Internet Archive Web Group talking about

"20th Century Find" using Amazon S3 & EC2

Internet Archive stats

3.5 PB
1.5 million downloads/day

this project is about providing full-text search of their web archive for the 20th century 1996-2000, ~22TB

NutchWAX = Nutch + Hadoop
focus moving to Nutch with plugins

Amazon S3
Amazon EC2 (beta)

there are now more EC2 node options: small (default), large, extra large 8 times small performance and better I/O ($0.80/cpu hour)

indexing began in October 2006
1996: indexed via 20 EC2 in ~36 hours
1997: 100+ EC2 nodes
1998: 300+ EC2 nodes

1999 was attempted in September 2007 using cluster of ~270 EC2 nodes but halted due to lack of
consistent CPU/IO across nodes.

deployed (alpha) index is 1.35TB in size, no compression, ~600 mill docs

Enhancements
* multiple instances of a page
* improved ranking of results
* handle dimension of time
* easy UI

Why Amazon Web Services
* pay as you go
* simple to provision
* committed to support
* indeal for indexing Web pages, providing offsite storage, reliable hosting
* great platform for experimentation, iteration
* geographically disperse from Internet Archive ?Data Repository?

Cost Effective, budgeted $20k

* Note: fees can add up fast if not vigilant

Working Well

* APIs
* tech support
* S3
* fee structure
* speed of provisioning
* S3 uniformity of nodes

Challenges - S3

Oct 2006 - June 2007

* (internal?) bandwidth availability
* no specific guarantees for data preservation
* issues related to popularity of the service

Fall 2007

* available bandwidth consistent (~4h to move 7.5TB into EC2)

Challenges - EC2

lots of issues Oct 2006 - June 2007
* location of S3 nodes relative to EC2 was a significant factor for large-scale data processing

July 2007 - present

* working well but hitting IO and CPU constraints on small (basic, default) nodes;
however will continue to use these small nodes

Consider Using AWS When

S3
* need cost-effective backup for data
* multi-provider preservation, geographically diverse

EC2
* if you have spiky computing needs (e.g. spikes in demand)
* you have available R&D resources

Will experiment with AWS for crawling and harvesting, starting Jan 2008, Heritrix/AWS.

Scirus Topic Pages now online

http://topics.scirus.com/

"curated" pages with various resources on a specific topic, e.g. Evidence-Based Laboratory Medicine

Previously:
June 28, 2007  Elsevier pre-announces Scirus Topic Pages

Casey Bisson on Scriblio and OpenLibrary

In the BIGWIG library social software showcase, Casey Bisson writes about his proposal to further enhance Scriblio.  Scriblio is a vision of a modern library web presence that is about finding books, not being a book inventory.

With its partners, the Internet Archive will build an open, structured website that includes tools to provide public and institutional access to library resources around the world. OpenLibrary.org will offer a system of integrated tools that can be used individually or together to meet a library or patron’s needs, that is free to the user and the library. The overall goal of the project is to shorten the distance between initial query and document.

Elsevier pre-announces Scirus Topic Pages

Topic Pages are an innovative way for scientists to communicate in an informal and flexible way. Each Topic Page will provide researchers with summaries of a specific topic written by an authority in the particular subject area, with direct links to relevant scholarly papers, abstracts and citations, supplemented with relevant websites and other online resources from Scirus. In the initial phase, authors for Topic Pages are invited through an editorial process facilitated through Elsevier publishing staff. As more pages are developed, additional authoring options will be considered.

...

At the official Topic Page launch later this year, the functionality of the Topic Pages will allow scientists and researchers to alter the content and provide feedback, allowing each topic to be shaped by the suggestions made by the research community.

from Elsevier PR - Elsevier Announces Partnership with FAST to Deliver Topic Pages for Scientific Communication on the Web

This is presumably the social network that was indicated in Joris van Rossum's Science-specific Search - Scirus and beyond talk at IATUL 2007.

UPDATE 2007-07-20:

http://topics.scirus.com/