Science Library Pad

The OCLC (Dis)Integrated Library System

Andrew Pace writes

OCLC is extending the WorldCat Local platform to include circulation and delivery, print and electronic acquisitions, and license management components. A quick start version of WorldCat Local--available at no additional charge to FirstSearch WorldCat subscribers--is a first step to WorldCat Local and to a truly next-generation cooperative library management service.

...

Five years after I advocated dismantling library management systems, I am confident that using web-scale architectures and a cooperative service model are the right way to put things back together again. The OCLC cooperative is not only uniquely positioned to provide this solution, it is part of our obligation to libraries.

via Lorcan on Twitter

An interesting development. One wonders how it will fit in with OLE, Evergreen, DLF ILS API and other initiatives.  Also I would be interested to know if it will extend to true, article-level integration/linking for licensed content, which has been the barrier to true usefulness for the ILS in a modern academic library setting (right now we have the catalogue for books, separate article-level systems, and then federated search on top to combine the results).

open source ILS (OLE Project) design workshop in Ottawa

http://oleproject.org/2008/12/05/regional-design-workshop-ottawa-canada/

Wednesday, January 14, 2009   9:00 am to 5:00 pm
Thursday,  January 15, 2009   9:00 am to 4:00 pm

Library and Archives Canada
Room 156
395 Wellington St.
Ottawa, ON

Lunch will be provided on both days

Registration is required by Dec. 31, 2008: Register Now

The Open Library Environment (OLE) Project invites you to a 2-day Design Workshop at
Library and Archives Canada in Ottawa. The purpose of this workshop is to provide a
forum for representatives of Canadian research libraries to analyze our current
business processes and discuss ideas on what core functionality a future system should
provide.

Participation is open to any members of the research library community who work with
the Integrated Library System either on a day to day basis or from a higher level.
OLE will be developed as an open source library environment that meets the needs of
research libraries. While care will be taken to design an open and flexible system
that is useful for other types of libraries, such as public libraries, the focus of
the project in this early stage is on research libraries.

Due to space limitations, registration is limited to 50 participants.

CISTI's Stephen Anthony will be giving a 30-minute presentation on SOA as part of the workshop.

are your circulation stats a gold mine?

Chris Keene has an interesting posting about the ALT TILE meeting.

(TILE = Towards Implementation of Library 2.0 & the e-Framework)

(ALT TILE = Association for Learning Technology meeting 'Sitting on a gold mine' - improving provision and services for learners by aggregating and using 'learner behaviour data')

He says

This is one of those things that once you get discussing it you’re never quite sure why it already hasn’t been done before, especially with circulation data. There’s a wide scope, from local library services (book recommendation) to national systems which use data from VLEs, registry systems and library systems. A lot of potential functionality, both in terms of direct user services and informing HE (and others) to help them make decisions and tailor services for users.

Chris Keene - nostuff.org blog - sitting on a gold mine - December 12, 2008

via my FriendFeed and Twitter

I suggested to Chris that TILE should also have a look at all the information Johan Bollen is gathering in the MESUR project.

next generation open library system

I'm so used to bookmarking stuff into my FriendFeed these days I have to remind myself my blog is also a communications vehicle... I'd be remiss in not posting about this major SOA & libraries initiative.

The Open Library Environment (OLE) Project

With support from The Andrew W. Mellon Foundation, the Open Library Environment (OLE) Project will convene the academic library community in the design of an Open Library Management System built on Service Oriented Architecture.

http://oleproject.org/

Duke University is the lead on this project.

UPDATE: Just for clarity I should probably mention that my organisation (CISTI) is neither a core nor an advisory partner.  ENDUPDATE

If you want to get an idea of the framework that this fits into, you can see my posting about the presentation Enterprise Services Architectures - Chris Mackie - DLF Fall Forum 2007.

It is great to see library projects starting to use a service-oriented approach, although given that this is the dominant industry trend these days, I'm not convinced that my four years of online and offline advocacy for library SOA actually had much of an impact on this choice.

(You can see my blog postings in category "Service-Oriented Architecture" going back to November 2004:
http://scilib.typepad.com/science_library_pad/serviceoriented_architecture/ )

Still, one does like to be right.

Building SkyNet for Science - presentation for NISO Discovery Tools Forum

My presentation is available at

http://www.slideshare.net/scilib/building-skynet-for-science-discovering-new-frontiers-using-embedded-knowledge/

A lot of it is conceptual, so you may want to wait until the audio is available from the http://www.niso.org/news/events_workshops/discovery08/ NISO site - http://www.niso.org/news/events/2008/discovery08/ (hopefully next week).

UPDATE 2009-04-16: I don't think audio was made available.  ENDUPDATE

UPDATE 2008-03-28: I forgot to mention that all some of the supporting links for the presentation (will be) are available at http://www.connotea.org/user/scilib/tag/nisodiscovery2008  ENDUPDATE

I thought it went well, although as first speaker up there is a disadvantage of not seeing how other people set up.  I was in a bit of a rush to get started so that I would finish on time, so I didn't do a great job of attaching my mike and I just held the wireless transmitter in my hand.  With the transmitter in one hand and my laser pointer gripped in the other for the entire 50 minutes, it's possible I looked a bit of a prat.

I usually try to remember to keep my hands free for presentations so that I can use more natural body language, anyway lesson learned.

I also forgot to say "The future is not set.  There's no fate but what we make for ourselves." before the last slide.

There are some common themes emerging from the presentations, I'm always amazed when a bunch of people develop presentations in isolation and then they actually all fit together when presented.

I've posted some photos of the presenters in my Flickr under nisodiscovery2008 (my cameraphone can upload directly to Flickr over WiFi), they also show up because of the machine tag linkage on the Upcoming page.  No pics of Chapel Hill yet as I don't have a car and it turns out while we're only about 4km from the town, the most direct route for me to get there I think would be to walk beside a six-lane divided highway, which is not too appealing.

UPDATE 2008-03-28: The carbon offset for my flights from myclimate.org (including the trip to Open Repositories) was about C$118.

Code4Lib Journal 1

The first issue of the Code4Lib Journal is up.
There is an article about the NCSU CatalogWS API - Beyond OPAC 2.0: Library Catalog as Versatile Discovery Platform.

I think it's great to have a library journal that can focus on more technical topics.

Previously:
November 06, 2007  NCSU CatalogWS API - DLF Fall Forum 2007

future directions of library systems

It was great to talk with various enterprise/IT/technology architects at DLF Fall Forum 2007 about our various ideas an approaches to modeling and implementing the new library technology infrastructure.  But it's clear there are still more library technology architects out there (even if they may not have that in their job title), I ran across an interesting blog posting by Jonathan Rochkind:

Notes on future directions of Library Systems from Bibliographic Wilderness blog, September 28, 2007

Here are some notes on near/medium future directions of the library systems environment/architecture, and a sketch of requirements on where we want to go. These notes may or may not end up as part of an internal white paper here, as we analyze where we want to be headed (something good to do anyway, but we got a kick in the pants when the vendor ended development on our current ILS).

...

The library systems environment has grown from being composed of a single “Integrated Library System” at the origin of library systems in the 1980s, to being composed of an ‘ecology’ of systems in most libraries today. These different systems could be from various sources (proprietary and open source), fulfill various–both overlapping and distinct–functions, are used in various ways by different subsets of users (both library staff and our end-users), and interact with each other to varying degrees (some well, some poorly). 

I wonder if I should consider adding a "library technology architecture" category to SLP?

PS Jonathan was at DLF but we didn't formally meet, it's odd how sometimes virtual connection and discovery is easier than in the physical world.

ILS Discovery Interface - DLF Fall Forum - Nov 6, 2007

I'm not sure I really grok this idea of bindings, it sounds like they run the risk of ending up with a catalog of specialised implementations, rather than a unified API.  But it is very much needed work, and we do have to be practical.  There was a follow-up discussion session, but I missed it.

ILS Discovery Interface (ILS-DI)

http://project.library.upenn.edu/confluence/display/ilsapi/

* We need Integrating Library Systems
* We need practical solutions soon
- The ILS may undergo radical redesign... but our users won't wait for that, and we can't

Scope

Out-of-scope
- acquisitions
- cataloging integration
- item management

in-scope
- patron-driven discovery from search to use
  -- finding relevant resources (discovery)
  -- acquiring them (delvery)
  -- managing their usage (patron info and account)

You are here: Rough draft recommendation being presented and discussed

Early 2008: Formal recommendation to be released

Beyond? Recommendation will be updated as new technologies, functions, tools emerge

Survey on Current Use of OPAC and Beyond

Current use
- majority considering replacing ILS in next 2 years
- widespread frustration with OPAC interfaces, metadata schemes, resource scope
- generally ok with ILS inventory management functions

Beyond the OPAC
- 3/4 using supplementary discovery apps
  -- many locally developed
- wide variety of interactions with OPAC
  -- data export most common

a lot of duplicate work being done on extracting info from ILS

1. Improve discovery by supporting interfaces to ILS
2. Articulate a clear set of interaction expectations for ILS and app developers
3. Making recommendations applicable to a wide variety of systems and technologies
4. Make recommendations that are feasible to implement
5. Work with applications beyond "traditional library" domain
6. Be responsive to the user and developer community

Functions
- areas of interest: data aggregation, real-time search/query, delivery, patron info & services,
OPAC embed / escape / entry

Bindings
* Specific technologies that implement desired functions

Data Aggregation (getting the data out of your ILS)
- GetBibliographicRecords
* want to have selective export (by date or record type)

Real-time search/query
- GetAvailability
- Search
- Scan
- SearchAuthorityRecords
- ListCourseReserves
- ID-based record retrieval

Delivery Functions
* Holds
* RecallItem
* Security, policy issues can be complex for these functions

Patron information and Service Functions
- RenewLoan
- CancelHold

...

OPAC embed/escape/entry

* OutputRewritablePage
* OutputIntermediateFormat

Getting to an official recommendation

* Make sure we're not missing any essential functions
* Get more specific on functions
* Also find or point to specific bindings of the functions
* Try to finish in reasonable time
* Now is the time to encorage implementor involvement

How can my solution be incorporated?

- show the group how your solution works as a binding for an abstract function we've specified

- it has to be openly and fully specified

* it ideally has service and client implementations in product

- it also helps to have
  -- no IP encumbrances

Beyond the recommendation

* periodically update
  -- is there a sustainability model?

NCSU CatalogWS API - DLF Fall Forum 2007 - Nov. 6, 2007

Very interesting presentation by NCSU on their CatalogWS Web Services API they have build on top of their Endeca layer.

Here are my raw notes

Library Catalog as Versatile Discovery Platform

UPDATE 2007-11-08: [PRESENTATION] ENDUPDATE

Next Generation Catalogs

Library Technology Reports - Next Generation Library Catalogs

Examples
* Endeca (NCSU Libraries)
* AquaBrowser
* Encore (UQueensland)
* ExLibris Primo
* WorldCat Local
* Solr (UVirginia Project Blacklight)
* vufind (also Solr powered)

Modern search
- relevance
- faceted
- tag clouds
New content
- user contributed content
? enriched content
? content or context awareness?

Focused on optimizing a single discovery context... the OPAC

Question

Why should the discovery of catalogued library collections be limited to user interaction
with a single catalog application?

Catalog as Discovery Platform, beyond the OPAC

Web Services: CatalogWS API

Platform

"a platform is a system that can be reprogrammed and therefore customized by outside developers..."

- quote from Marc Andreesssen

Discovery Happens Elsewhere

"No single website is the sole focus of a user's attention."

- quaote from Lorcan Dempsey

Platform Motivation

* move beyond the one-size0-fits all approach
make it easer to reuse and repurpose catalogue data outside the opac
* build catalog interaces optimised for different use contexts

CatalogWS API

Goals
- can we have RSS feeds for the catalogue
- can we integrate catalog results into library website quick search

Final result
* rich API

[diagram of architecture]

API implemented as layer on top of Endeca

Limitations
- subset of catalog data
- read-only
- not real-time

Technical Design
- RESTful
- Java, Tomcat, XOM, Saxon 8.8, JSON

http://www.lib.ncsu.edu/catalog/ws/

* discovery-oriented
* catalog availability (known item lookup isbn)
* catalog search (both known and exploratory searching)

can pass a style parameter (URL of XSL), for server-side XML transformation, or XML to XHTML

Why new XML schema?
- include as much of data as possible in reponse
- MARC XML and MODS didn't appear capable of capturing the varied data
- XML response includes links

Demo of Current CatalogWS Apps

* integration with external apps
- Quick Search (NCSU library website search)
- iGoogle Widget

* alternative catalog interfaces
- mobiLIB catalog
- facetbrowser

Collection Promotion
- FacetBrowser - ability to easily create blog entries for selected items
- automatically generated "bookwalls"
- RSS feeds for new titles

http://www.lib.ncsu.edu/dli/projects/catalogwsapps/

search hits are the lowest form of humour

Sorry to whomever was searching, but I can't resist sharing this item from my web hits

*.iii.com (Innovative Interfaces Inc)
California, Oakland, United States, 0 returning visits

Date    Time    WebPage
10th September 2007    21:12:40    www.google.com/search?q=opac is dead

software development, staffing and new library technology

Librarian in Black responded to my article in Library Journal netConnect, and Richard Wallis' commentary thereon.  She says

What makes me sad is that both Ackerman [sic] and Wallis have missed a key point: if the future is in web services, how can libraries take advantage of that with their current staff configurations?  How many libraries in the U.S. have a honest-to-goodness computer programmer on staff?  How many have staff with Computer Science degrees?  How many staff do they have devoted to the library's hardware, software, and network?  How many staff do they have devoted to web services?

In the smallest libraries, perhaps all of these are the same one person.

My article is about modern software engineering for libraries, not on how to staff them.  However, in the very same issue of netConnect there's an article by Karen Coombs "Digital Promise and Peril" calling for library staffing to reflect the digital content environment

Many of these digital materials are in jeopardy of being lost because librarians have not yet found adequate ways to collect and manage them. In part, this is because roles and skill sets have been siloed in libraries. Materials preservation issues have typically been the purview of special collections and archives units within the library. In contrast, cataloging expertise has resided in technical services, and technology expertise has typically resided in systems. To collect and manage born digital objects adequately requires these roles and skill sets to come together.

So let me summarize some of the goals and targets of the article, as well as talk about the relationship to promising developments:

• The main focus of the article is to convey to library administrators, managers and planners that the world of networked digital content requires new ways of thinking about developing library systems, and that there are modern software engineering methods and technologies that can support new systems development.
• It may be the case that, as in Sarah's words, the library blogosphere knows that "Of course the future is in web services" but it took years of hard lobbying and education within my organisation to convince my own library leadership - that's why I was happy to have the forum of Library Journal, to reach a wider audience.  I have to believe there are many library managers who have no idea where the technology future lies, and have only a vague notion of web services.
• If I can get one library manager to be able to ask good questions about web services and service-oriented architecture, and even (dare I dream) read one of the books I recommended with more detailed information on the topic, my goal is achieved.
• I recognize that many public libraries don't have the staff for development, and never will.  That's why I have said in the past that librarians should be scripters, not coders.  Most libraries should be using technology developed externally, not trying to do their own custom internal development, or as I said in my article, they should be technology service consumers, not necessarily producers. There are only a few libraries in the world (including mine) which have the size to have a substantial development staff.
• I work at a research library, not a public library nor an academic library.  Research libraries like mine now interact with patrons almost entirely online - our walkin traffic is basically zero.  That means we have to move into the networked information space very aggressively, or basically disappear as a presence for our users, replaced by publisher web sites.
• By moving to standard APIs, using standard modern development methods, and standardizing web services, libraries can take advantage of a much broader development community, and potential staffing pool.  How many software developers do you think know Z39.50, MARC, and SRU/SRW?  That's why we have a tiny community of library hackers trying to make things work.  Now imagine that instead your library software job poster just says "developer needs to work with standard Java tools to develop software using modern methods for standardized APIs".  Getting access to a better, broader staff base is intimately connected to moving library technology into the mainstream of software development.

Where can we look?

Maybe the DLF project on ILS APIs will help.

Maybe the OASIS effort on standardising search services will be useful.

Maybe it happens by using OpenSearch and simple REST interfaces rather than custom library protocols.

UPDATE: There is another important piece, which is about libraries reaching out and speaking the right language.  That's why you need to understand how to express things in terms of SOA, Web Services, and APIs.  There is way more innovation capacity outside your walls than you can ever get inside, even if you have the perfect IT staffing policy and budget.  From your local superpatrons, highschool CS students, and local college and university computer science departments, to, basically, every CS student in the entire world.  You can reach them with contests, with collaboration requests, with invitations to improve your systems... but here's the important part... if you speak their language.  CS people love challenges and programming, but they're not going to learn obscure library jargon and usage like OPAC, Z39.50 and database (which means something completely different in CS).  You can't say "hey, can you help us improve our OPAC because the Z39.50 doesn't federate across our databases".  They're not going to know what the f*** you're asking.  Learn the CS language, and a whole world of programmers will open up to you.

To me one of the single biggest missed opportunities is in the digital library community.  Ever year, lots of computer science groups, flush with energetic grad students, toil away and produce results that are presented at JCDL and ECDL.  And, based on my experience at ECDL 2006, they then present those results entirely to a community of other computer scientists.  Where are the librarians?  Why are you all at library conferences talking to other librarians?  Come to *CDL and ask the computer scientists to build stuff you need.  Yes, it's a difficult transition from research to production, but at least join the conversation.  ENDUPDATE

I don't have the answers and I have certainly asked again and again where people see all these frameworks and groups fitting together, with no response.

The good news is that there are lots of projects out there already - I don't think it's a case that there is no activity.  The fundamental point of my article is that for these projects, we have to use enterprise architecture, service-oriented architecture, web services / standard APIs and the whole toolkit of modern network-based standard-data software development.  Because if we don't, WE WILL BUILD SILO SYSTEMS AGAIN.

That's what I said in my IATUL presentation, and in the short accompanying paper (Scribd), and what I've been saying over and over again in this blog.

[Sidebar on Scribd: be careful browsing around this document hosting site.  Many of the profiles, profile images, and documents are unfortunately very not safe for work.  Scribd really needs to put in a moderation / adult content filtering system.]

Why do I think it's important to talk about these topics?  Because there really are lots of new developments in the library catalogue and OPAC world, including:

• Scriblio - "free, open source CMS and OPAC with faceted searching and browsing features based on WordPress"
• VuFind - "The goal of VuFind is to enable your users to search and browse through all of your library's resources by replacing the traditional OPAC"
• Evergreen Open ILS - including British Columbia Pines project - "The phased implementation of the Evergreen Open ILS for all of BC ("BC PINES") be implemented over the next 5 years. We hope that eventually all public libraries in BC will join"
• eXtensible Catalog (XC) - "an open-source online system that will unify access to traditional and digital library resources"
• CERN systems redevelopment - build a complete high-energy physics (HEP) information system with full-text, data-mining and demonstrate and deploy Web 2.0 applications in the domain of sciences

There are also some great modular browser tools out there, including

• LibX - a Firefox extension that provides direct access to your library's resources [and] an open source framework from which editions for specific libraries can be built.
• Zotero - a free, easy-to-use Firefox extension to help you collect, manage, and cite your research sources

I'm very much hoping that these developments will open libraries up to be better network participants, with a broader community of developers able to build pieces, and with standards enabling libraries with limited development capability to simply plug-and-play.

Some links via The Ten Thousand Year Blog: eXtensible Catalog open source project, VuFind released as open source software.

Previously:
June 29, 2007  Casey Bisson on Scriblio and OpenLibrary

Library Journal netConnect - Web Services and the Social Catalogue

Library Journal netConnect for July is out (July 15, 2007).  The cover theme is the "Social Catalogue".  All of the content is free online as usual.

On this theme are an article I wrote about Library Web Services, advocacy by John Blyberg about the need for open APIs ("Always Pushing Information"), and an explanation of one way to Visualize Your Catalog using Grokker, by Kate Bouman et al.

I want to thank Jay Datema for this opportunity to share these concepts with a wider audience, and in particular for his patience during the extremely long genesis of my article.

I do want to add some supplementary information. 

"web services also refers to one specific technology implementation" - by this I mean the industry Web Services stack, SOAP-based services.  This shouldn't be confused with SOA, Service-Oriented Architecture, which is a conceptual framework.  There is a short version of my sidebar on SOA in the article, but with Jay's permission, the full original version is attached below.

I did a set of supplementary bookmarks for the article, but somewhere along the line I forgot to include them in the article itself, anyway, they're available at

http://www.connotea.org/user/scilib/tag/ljwebservices

UPDATE: As it happens, Jon Udell blogged a plea today for the most basic of catalogue interfaces, a standard ISBN syntax in the URL.  Via panlibus.  ENDUPDATE

I will also try to finish the blog posting of a long-overdue review of the 4 SOA / Web Services books I recommended at the end of the article.

You can of course read more as well in my (sometimes overlapping) blog categories on Service-Oriented Architecture and Web Services.

As well, Peter Murray has been advocating for library SOA, and has set up an aggregator to gather various blog postings on the topic:

http://librarysoa.dltj.org/

Finally, here's the full sidebar that I wrote:

Library SOA

In the library world, we are often bombarded with new technology terms and ideas.

In this article, I will discuss Service-Oriented Architecture (SOA) and Web Services, providing a framework for understanding their differences and complementary nature.

In this way, it should become evident which aspects will be of enduring value, and which will change as technology evolves.

Often SOA and Web Services are used interchangeably but they are actually quite different. SOA is a methodology for software architecture and for guiding software development. The library technology landscape is littered with towering monoliths of software and teetering siloed systems. We shouldn't feel too bad about this: when catalogues and digital libraries were being developed, these were reasonable implementations. It is only as the software engineering field has evolved that we have realized the problems caused by developing closed, inflexible systems.

SOA is a way of thinking about software, of breaking it down into appropriately-sized component parts, and minimizing the dependencies between those parts. It also helps to guide your thinking by emphasizing reusability. Having reusability as a core consideration reminds the developers that their code is not ephemeral - it will almost certainly be around for a long time, which means that it is critical to make it maintainable and extendable.

Web Services are a technology that can be used to implement a Service-Oriented Architecture. They have some characteristics that mesh well with the requirements of an SOA. But Web Services are just the latest technology to come, and are sure to be replaced or added to by future developments. As well, there is a potential danger of complexity creep, as more and more WS-* standards are developed.

So the important thing is to take the concept of Web Services, that of providing a well-defined interface, an API that exposes useful capabilities or data from within your organization, and select a technology as appropriate. This may be full Web Services with SOAP, it may be a simple HTTP REST interface, or both, or neither. It’s the idea that’s important, not the particular technology. In fact, what is a service, within an SOA? It’s simply a bundle of functionality with a well-defined interface.

When developing software, regardless of methodology or technology, it is important is to have the concept of sustainability in mind. There is a balance of course between long-term benefits and short-term gain. Many will argue that too much design or planning interferes with organizational agility, that it is better to rapidly deploy something, than to tilt against the windmill of perfectly engineered software.

But the fact that we now have OPAC 2.0 and “new ILS” initiatives tells us that we need to learn the lesson of sustainable software. To put it bluntly: with the library catalogue, we slowly built a software system that no longer meets all of our needs. With mashups and quickly hacked APIs, we run the risk of rapidly building software systems that will not be able to meet our needs.

Service-Oriented Architecture is the latest, and by no means perfect, method of addressing enduring challenges of software design. It is a new way of describing and communicating the concepts behind the software you build; a way of maintaining the essential conceptual integrity needed to successfully build sustainable computer systems.

SOA is a framework that sits above the development process itself. SOA does not require a waterfall model of development. We know a failed approach is trying to do all the design up front and then, perhaps after substantial time has passed in the design phase, writing the code. In fact, I believe the best results will be obtained by iterating the architecture, that is, continuously refining the framework by going back and forth between implementing defined services, and updating service definitions based on the results. I think it is important to remember that modern software engineering is about the development of software as a conversation between design and implementation, not a one-way street.

Like any methodology, there is a risk that SOA can lead to an excess of complexity, and to trying to over determine requirements that may change quickly. Be wary of elaborate SOA processes that are better matched to giant Fortune 500 companies than to the needs of basic library systems. But with all those caveats in mind, the core of SOA is quite simple: define what functions the business wants to perform, and then break those functions down into the smallest useful pieces.

This is the concept of service granularity: identify services that are large enough to provide substantial useful functionality, without being so large that they are in danger of becoming new silos. As luck would have it, we already have a fairly good idea of basic library functions, as defined by our current catalogues. We first need to break those services out of the ILS silos, and then look at adding new capabilities beyond what the ILS could offer.

There is a possibility for us to all escape the isolation of our OPAC islands, by working together to create common library service definitions, so that we move beyond the catalogue and are interoperable right from the start. I believe this is a tremendous opportunity for the library community, because standard services are the very foundations upon which innovation is built. We look to Amazon and want to capture some of the software development energy that builds upon Amazon’s services, but we have to realize that behind the scenes, Amazon has an internal SOA that helps to guide and sustain its service framework.

SOA is, in a way, like infrastructure. We benefit enormously from the fact that electrical, water, and other services in buildings are standardized, with uniform interfaces and attributes. I don’t think many people would want to work in an environment of “agile electricity”, where every wall socket had a different interface and electrical characteristics. That basic foundation of standards is what SOA can provide to the library world.

We all know that there are successful standards efforts, and there are those that languish. This is the moment, as libraries are starting to explore the world of Web Services, for us to work together as organizations to put in place the infrastructure, the simple set of basic standards, the library Service-Oriented Architecture that will form a platform for sustainable innovation for years to come.

Casey Bisson on Scriblio and OpenLibrary

In the BIGWIG library social software showcase, Casey Bisson writes about his proposal to further enhance Scriblio.  Scriblio is a vision of a modern library web presence that is about finding books, not being a book inventory.

With its partners, the Internet Archive will build an open, structured website that includes tools to provide public and institutional access to library resources around the world. OpenLibrary.org will offer a system of integrated tools that can be used individually or together to meet a library or patron’s needs, that is free to the user and the library. The overall goal of the project is to shorten the distance between initial query and document.

let library machines talk to machines, so that library humans can better serve patrons

Or in other words: "Free the Humans" (available as stylish tshirt).

I think it may be difficult coming from a non-CS background to conceptualize how fundamentally important it is to make library data machine-readable, to enable easy interoperability and extendability.

That's why I loved this posting in ALA Techsource, which explains it all much better than I could:

Out of the Secret Garden: The RDA/DC Initiative

paper on library SOA and the BIBSYS system

Moving towards a service-oriented architecture (abstract) / Erlend Gutteberg (BIBSYS, Trondheim)

from European Library Automation Group - ELAG 2007: Library 2.0 - Papers and Speakers

BIBSYS "supplies library and information systems to over 100 libraries and institutions of higher education in Norway".

the French National Library and more

You may have the impression that I'm a technophile, but I'm actually an appropriate-technology-phile.  People, paper and pens can be appropriate for some activities, for example, voting.

Here's a passage about the Bibliothèque nationale de France that I enjoyed from Adam Gopnik's essay "Lessons from Things, Christmas Journal 3", in his book Paris to the Moon.

Then you search, among consoles set off near the walls, for an empty, operating computer terminal on which to make your book requests.  Most of the terminals are out of order, and when you insert your identity card, they sigh and say that they are initializing.  After fifteen minutes you give up and walk up and down the great hall, looking for a terminal that works.  When you find one, you can penetrate the catalog fairly quickly; then you claim the page and demand the book; the computer registers that you have made the demand and tells you to go sit back down.  The entire library is, in principle, served by, or subject to, the same vast, single computer system, which knows who you are, where you are, what you're doing, and what you want, can track you from visit to visit, and anticipate your interests, etc.  This of course means in practice that any tiny bug in one part of the system destroys the entire operation of the library.  The latest bugs are posted on photocopied sheets Scotch-taped to the terminals: Please, don't ask to "resee" your list, they say, just ask to "revise" it, etc.

...

On the desks there is a single red light that is supposed to illuminate when your books arrive, but these lights have never been known to work.  Or, rather, they have been occasionally known to work.  So you have to get up regularly and check your computer terminal again, to see what's up.  The light may be off because the books haven't arrived yet, and it may be off because it's not working.  ...

It's quite a long essay, covering diverse topics, and I recommend it.  I was interested to read in it about the exhibits in the Jardin des Plantes, since I have recently been there (I photographed some of the buildings, but I didn't go in, I just had time to walk in the garden).  I suppose I have progressed, considering that when I was there in 2004, I didn't even know there was a natural history museum at all.

More about this book

Chapter One, courtesy of the New York Times

Excerpt chapter "A Tale of Two Cafés", courtesy Random House

ISBN-13: 978-0375758232

Photos from the Jardin des Plantes

2007:

2004:

CLA 2007 and preconf in St. John's

This is the best presentation description I've read in a long time

Rick Anderson is convinced that the future of libraries will be a dismal one if we don't let go of certain cherished assumptions and traditional mindsets and realign our thinking in some radical and far-reaching ways. He likens library patrons to a river, and the traditional library to an expensive, unwieldy and, ultimately, fatally flawed system of dams and levies designed to force patrons in unnatural directions, a system that is already cracked and leaking and threatens to give way entirely. Among the topics he addresses are:

    * How the OPAC is destroying the library profession
    * Why most serials management tasks are a waste of time
    * Why it's better to give a man a fish than to teach a man to fish
    * Why we should stop thinking like good librarians and start thinking like bad patrons

CLA 2007 TSIG-SIG preconference - May 23, 2007 - Keynote - Restructuring Our Thinking: New Mindsets for Librarianship in a Radically Changed Information Environment

The main CLA 2007 conference site is

http://www.cla.ca/conference/2007/

PS that's Canadian Libraries, not Californian

OLA 2007 - next generation ILS presentations

Ontario Library Association (OLA) Superconference 2007 presentations are up.

I haven't checked many of them, but I do highly recommend

ILS, The Next Generation: Modularity and Outward Integration (PowerPoint)

and

The Changing Nature of the Catalogue and its Integration with Other Discovery Systems (PowerPoint)

both by Karen Calhoun

UPDATE 2007-02-13: Since I find myself accidentally in the position of "recommender" (thanks LISnews), here are a couple more that I liked

* Perceptions and Expectations of the [Public] Library Brand (PDF)

An interesting examination of the challenge in extending the perception of the public library beyond books.  I liked this slide:

The term "library" cannot [be] extended to include services other than the circulation of books when there is little public trust in the brand.

The good news is that many public libraries have successfully branded themselves as knowledge providers, going far beyond books.

* Lastly, the obligatory blogwikiflickr presentation, although in 2007 I feel this is a bit like being in 1967 and talking about the marvel of the internal combustion engine, aren't blogs already retro?  Aren't we beyond the "blogs are exciting and new, come aboard, we're expecting you" stage?

Web Tools to Inspire Learning (PowerPoint)

OPAC Glory Days

Yeah, just sitting back trying to recapture
a little of the glory of, well time slips away
and leaves you with nothing mister but
boring stories of glory days

Bruce Springsteen, "Glory Days" - from Lyrics Depot

Now, use your library WebOPAC and try to locate a resource with that exact same quote.
It took me about 1 second on Google.  How did you do?

In this month's D-Lib Magazine, Karen Markey has a "think piece" on The Online Library Catalogue.
If it had been the April 1 issue of D-Lib, I might have found it more understandable.

sidebar - my definitions:

catalogue - the electronic version of the card catalogue, with perhaps the added functionality of tracking loans - a book inventory system

ILS financial functions - the part of the Integrated Library System that provides "QuickBooks for libraries" - tracking purchases, serials subscription costs and deadlines, etc.

WebOPAC - an interface for librarians to the book inventory system

end sidebar

Markey has subtitled her opinion piece "Paradise Lost and Paradise Regained?"

Now this is an odd beginning.  Eve had humankind expelled from the Garden because she ate of the tree of KNOWLEDGE of Good and Evil.  You can interpret this either as mankind gaining self-awareness (sentience), or as mankind gaining a sense of comparison, learning that some things are good and some are bad.

Returning to Paradise would therefore imply either a loss of ability to distinguish between useful and useless, or losing the ability to think reflectively entirely.

Let us visit that paradisical past, in Markey's words

online catalog's lofty status

revisit the glorious past of the online library catalog

online library catalog was the jewel in the crown

golden age of the online catalog

This way lies madness.

For a decade and a half beginning in the early 1980s, the online library catalog was the jewel in the crown when people eagerly queued at its terminals to find information written by the world's experts.

Ok, let me rephrase that.  "For a decade and a half beginning in the early 1980s, ending thankfully with the availability of the web in 1995, the only source of information was the mysterious Oracle at the Library Temple.  Students would wait in long queues before the Oracle (who was named Public Access Terminal), not because the system was particularly slow, but because its interface was incomprehensible, requiring some mysterious combination of (TI= AND AU= ) in order to uncover books that were already known.  Almost no one tried to use the terminals to actually discover information, because such would have been impossible.  Books were listed without priority, without difficulty level, without explanation or commentary."

I have explored this territory in depth.  It took me a long time to understand the WebOPAC.  I thought it was Library Google.  It turned out to be an interface to a book warehouse system, designed for locating inventory items on shelves.  I initially called my category for this exploration ILS, but then I figured out that the catalogue itself is fine, because warehouse managers do need to know what their facility holds, and the financial functions are fine, but the OPAC was, well, wrongly provided.

I proclaimed a message of freedom: "Let go of it.  You Are Not the Technology That You Use."

I talked about this ridiculous mythology that Google provides too many hits, of the wrong kinds.

The temple of books has fallen and we are better off for it.

The counter-arguments appear to me to be perniciously anti-civilization.  You know when the library shines out in the darkness?  IN THE DARK AGES.  Apparently the librarian view of mushroom clouds or planetary-scale storm clouds on the horizon is a sigh of relief: "ah, finally they will have a reason to return to the library".

I'd rather not walk that path back to paradise, thanks.

Markey identifies the whole problem with Google searching: users are dumb ("domain novices"), fickle, and lazy.  Insufficiently qualified to search.  Google presents them with easy, untrustworthy answers (that are simultaneously overwhelming, due to too many hits).

Belkin (1980, 137) tells why so few words make up their queries, "Precisely because of the inquirer's lack of knowledge about a problem area, it is impossible to specify what would resolve it."

...

Searching for Something One Does Not Know Is Frenetic, Aimless, and Random...

Debowski (2001, 378) makes similar observations, "It was evident that [people] spent more time inputting, rather than planning a suitable search process. There was little evidence of search quality assessment ... with most entering the next search statement very rapidly ... [People] who search without a solid foundation fail to gain a stronger understanding of the search process. Instead, they appear to develop further erroneous habits as they continue."

I have an alternative hypothesis: People type short queries because they are smart.  People use Google because it uses an advanced algorithm based on what people rate highly, in order to provide them with an immediate choice of probably-good results plus thousands of alternatives to explore.

Her underlying concern: books are being digitized.  Boon for mankind?  Worldwide access to our previously limited information?  Paradise?  No.  Books being digitized, we are informed, will be terrible, because

Each search will result in millions of hits with no guarantee that the top-ranked ones will address your desired topic in depth or at your level of understanding.

Ah, so we come to the heart of it.  Better not to have the chance to know, than to aimlessly wander the entire collected knowledge of mankind.  Thank goodness Alexandria burned, saving us from millions more misleading hits!

Apparently the solution to this problem is... wait for it... the full text of all knowledge should be ingested into the library catalogue, for proper control.  Perhaps we could call it the Landru system.

I have an alternative proposal: don't do that.

Your library website doesn't have to be the starting place for all information seeking.  It only has to be one of many well-respected information destinations.  It has a huge challenge in playing this role, because it has almost entirely only online data about information containers, rather than the online contents of those containers.  Deal with that.  Get Google to digitize everything.

Open up your library data and whatever online content you have so that Google can actually rank you as high as you deserve.  Instead of trying to reinvent Google and Amazon, find areas where you can add value based on the unique expertise and insights of the library field.

presentation on library services 2.0

You know it's strange, I don't really browse videos on YouTube or photos on Flickr, but I do find all sorts of stuff of interest on SlideShare.  I guess because I am an information junkie.

I found a great presentation Services 2.0 dans les bibliothèques vers des bibliothèques 2.0 ? that covers the various aspects of integrating your library into web browsers and into other web sites, as well as improving your own library site.

The author of the presentation blogs at BibliObsession.net.  It's in French.  Be warned there are some possible NSFW images there.

(If for some reason you're interested in the discovery chain, it was via Mediapolis who favorited my Web 2.0 presentation, to the presentation Blogs et wiki : outils et phenomenes - it was third in the "related slideshows".)

in which the OPAC was wrongly provided

Perhaps you have heard this expressed already, but I haven't seen it put this way, and it helps me to frame the issue.

Libraries used to do two things:
- librarians talked to patrons
- librarians used the catalogue

then the Internet came along, and they decided patrons should use the catalogue online

MISTAKE

There are two things libraries need to support in the Internet environment
#1 patron access to library services
#2 advanced patron and external system access to library data and services

#1 is belatedly being attempted with layers on top of the catalogue, like Endeca
Personally I would have just said to Google "make me a website that will return relevant results to my patrons from my catalogue data, and link to the appropriate ordering pages", but since libraries have a hate on for Google, that's basically what they're cooking on their own with Lucene.

Anyway, #1 seems to be understandably the current number one library obsession, and I'm not going to say it's trivial, but it certainly appears technologically straightforward. Step A) rescue your data from its catalogue prison - suck it out somewhere that modern technology can be applied to it. Step B) apply modern off-the-shelf search technology to it.

I'm not even sure you need to do Step B. Step B could equally be "let every search engine in the world index it with appropriate links back to my library, and some way for patrons to identify what geographic location or specific libraries they are interested in".

#2 gets more into the Service-Oriented Architecture areas that interest me. I don't think libraries ever really thought much about machine-to-machine communication of data and provision of services. Even Z39.50 works in the library world something like a) librarian formulates search b) search sent by Z39.50 c) librarian analyzes results of search and e.g. uses for copy cataloging. That is to say: humans. Always humans.

So while I think getting data out of the catalogue is an important first step, there is much more to making a full SOA for libraries, at least one that won't be DOA. Think about

* how can you expose your data for use by other people and machines - microformats? REST interfaces? Web Services?
* how can you expose your library services ("request ILL", "please purchase this book", etc.) for similar use by other people and machines?

I think considering the above issues will lead you to conclude that either your library individually, or all libraries in general, need to do a service-oriented analysis of their business functions and data.

Previously:
December 5, 2006 but how does SOA fix my ILS?

but how does SOA fix my ILS?

It just happens that I am thinking a lot in the SOA / Web Services / ILS / OPAC space and in the computer scientists (please don't call us "techies") talking to librarians space.

I guess one difference that I perceive is that CS people often don't, due to their training, start with a particular technology (Ajax, RSS feeds, whatever) when looking at solving a problem.  Similarly, Enterprise Architecture is about first abstracting the basic desired function ("provide relevance ranking") rather than examining a specific technical solution ("stick Endeca on top of my ILS").

That's why you will find me writing a lot about "what", as in, what are the underlying things you want to do, rather than "how".

So that being said, I see a lot of activity around fixing the ILS, unsucking the OPAC and such.
And I realize I haven't done a very good job of constructing the chain from SOA theory to new improved disintegrated library system.

To some extent, it is the same "you can't get there from here" pig makeover story that has been extensively discussed elsewhere.  You can't incrementally decorate your way out of the underlying problems with the ILS.  Have you seen Extreme Makeover: Home Edition?  They have a simple process for transforming people's houses:

1. Demolish the current house
2. Build a brand-new house (using the latest tools and technology, along with a lot of hard work)

What does that mean in practical terms?  It means you need to be building a brand-new library system, using the latest tools and technology, along with a lot of hard work.  Now if you've seen EM: HE, the other thing you will notice is that in the triangle of people versus time versus scope, they make the time very small, by making the number of people working on the house very large.  So: it takes a village to raise a new library system.  And the construction materials that we have at hand, the latest concrete, wood and utilities, are services (whether you call them Web Services, REST APIs, new standard interfaces, extended OpenURL - it's all services).  Services are the new pieces you need to build the new LS.

But... you're probably ahead of me here.  If you put these pieces together haphazardly, the house will fall down.  If you don't want your new LS falling apart, you need... architecture.  A sort of Service-Oriented Architecture, in fact.  An SOA agreed upon by the community (the library technology village), defining a standard set of services so that we can all build our own unique Extreme Homes.

Without a common set of services, a shared basic architecture, it's as if you're building pyramids with giant chunks of limestone, and I'm building temples with bricks and concrete: we're both using great technologies, but we can't easily build on each other's work, because our fundamental building blocks are completely different.

That's all that SOA is, a way to come to a common agreement, so that we can all use the same pieces, the same technology components.  If you agree that the way to get to a new library system is by using services, then SOA provides the framework to ensure that we all get to use the same pieces, so that no one organization has to build the entire thing themselves.  You use my relevance ranking service, and I use your book cover service.

And all of this does NOT have to be slow, with everyone waiting for some giant SOA committee consensus.  Let's agree on a few services, and combine them in different ways to make a bunch of version 1 library systems.  And then a few more services, and a few more.

Replacing the monolithic ILS with a new silo will fail.  Trying to define everything in advance in some Big Bang Waterfall will fail.  But total anarchy of services will, I believe, also fail.  We can't spend all our time pouring and re-pouring the foundation of the house, putting up a wood frame and then tearing it down and putting up a metal one.  We need to build services on top of services, layer by layer, orchestration by orchestration.

So I apologize if I'm often on about apparently esoteric debates concerning SOA reusability and similar theoretics.  What all the SOA jabber is about is making sure that we have the right methodology to sustain building more and more complex sets of services, or in other words, ever more complex yet maintainable new library systems.

If this explanation is still not doing it for you, please let me know.

the SOA library system?

While I have been wandering around in Second Life, Steve has been traveling a lot in First Life, and has returned with a report on The Future of the ILS.  The consensus tag for the event seems to be ils06.

Wolfram's awesome library hacking and mashups

I had the good fortune to meet Wolfram Schneider at ECDL 2006.

He has done some great work with Z39.50 searching and also library mashups.

He built his app quite a while ago - probably it was one of the first advanced Z39.50 apps on the web, I don't know.

His web app is called ZACK Gateway, it does a Z39.50 federated search (what librarians, always needing to be different, call "broadcast search") across a wide number of libraries.

ZACK - Z39.50 Gateway for international libraries

ZACK Gateway (in German, for Germany)

ZACK Bookmaps

It is particularly specialized for locating books in Germany, but it will also check a small number of libraries worldwide (he has better data for Germany).

Here's a map for The World is Flat (it will take a moment to load).

He has also done a mashup that displays the distribution of MARC and other formats (Z39.50 target types), by catalogue type, worldwide.

ZACK Z39.50 target maps (select a target type to get a map)

I wonder if maybe he could work with LibraryThing - I'm going to suggest this.

the critical library role of providing online access

Librarianship - much ado about inventory?

This is a problem, because in the digital world, every time your patrons fail to access a resource that is available to them through your library, you have failed.

library = community access to reusable stuff
museum = community access to unique stuff
archive = nobody/privileged access to important stuff

It seems to me that properties of the physical world created a mind trap for librarians.

Community in the physical world is about place - if you build it, they will come.  Community is a side-effect of the physical library as people aggregator.

Access in the physical world is so transparent that it's hard to even think about it.  Librarian hands patron a book.  Voila, the access problem, solved.

Reusable is a function of the definition of the library itself - if it wasn't something that could be reused, the library wouldn't hold it.  The library of gasoline and firecrackers might be an exciting place, but I don't think it would stay in business very long.

Stuff is the core of the physical library.  It is so central to the physical library mission that I think for many librarians and for the field of librarianship, it became the sole mission.  Just managing a large inventory of objects was legitimately a challenging task in the physical world, particularly before electronic inventory systems.

Digital turns this all on its head. Let's get back to this "fail to access" statement.
What I'm saying is, don't worry about fancy library 2.0 OPAC whatever.
In the digital world you must have as your core mission providing access.
Anywhere, any time, that a patron encounters a resource that you license, they should automatically get access to it.

Instead, day in and day out, patrons 1) have no idea that resources have been licensed for them and 2) fail to access licensed resources - fail in the sense of "are unable to" or don't know how to, etc.

There's a thing called Government of Canada Central NewsDesk.  Free licensed access to pretty much every newspaper in Canada, as well as Canadian magazines etc.  I'm willing to bet 99% of government employees have no idea this exists.  I'm willing to bet that day after day, they say to their colleagues "I would send you that Globe article, but it's not available free online", when in fact, they have full licensed access.  This is an appalling failure of the library.

The fact that every single link in Google News and Google News Archive doesn't automatically have a link that says "click here for access provided by your library" is a tragedy.

Providing automatic access for patrons to licensed resources wherever and whenever they search should be the single highest priority of libraries until it is so deeply embedded in every discovery system that it is taken for granted.

Think about the competitive space you're in.  "Reusable stuff" is practically the definition of online content (excluding the hopeless DRM efforts that try to prevent this).  Everyone has reusable stuff now.  It's called the web.  Everyone can even inventory their reusable stuff in various ways (iTunes, LibraryThing etc.)  Community is the purview of online aggregators of people, which have replaced in some (many?) ways the offline aggregation that used to take place in physical spaces.  The library website/OPAC is most definitely not a leading online community destination.

So what's left?
What's left is access.

Do that.