Science Library Pad

E-Science and libraries

As I said over a year and a half ago

A lot of science today is very computation and data intense.  I think there is a big role for academic libraries as custodians of data and research output.

Science Library Pad - the role of the academic library and librarian - January 13, 2006

Fortunately, there are lots of people thinking about the role of the library in relation to cyberinfrastructure, as well as e-research and publishing.

This month's D-Lib has a number of relevant articles and opinion pieces, including

Library Support for "Networked Science"
by Bonita Wilson
doi:10.1045/september2007-editorial

Cyberinfrastructure, Data, and Libraries, Part 1: A Cyberinfrastructure Primer for Librarians
Anna Gold, Massachusetts Institute of Technology
doi:10.1045/september2007-gold-pt1

Cyberinfrastructure, Data, and Libraries, Part 2: Libraries and the Data Challenge:  Roles and Actions for Libraries
Anna Gold, Massachusetts Institute of Technology
doi:10.1045/september2007-gold-pt2

The editorial by Bonita Wilson points to "The Dawn of Networked Science" in The Chronicle for Higher Education (which is not open to read, so no link for them).

I also recommend the August 2007 CTWatch Quarterly, which was on the topic "The Coming Revolution in Scholarly Communications & Cyberinfrastructure".

A reminder that there will be an E-Science track in the upcoming OECD meeting on October 3, 2007.

For more information on this topic, you can see my E-Science category.

CIHR Policy on Access to Research Outputs approved

The CIHR Policy on Access to Research Outputs was recently approved by the CIHR Governing Council.  As a publicly-funded agency, CIHR has a fundamental interest in ensuring that research results are available to the widest possible audience and at the earliest possible opportunity in order to maximize the utility and impact of the research it funds.  This policy aims to increase the diffusion of research publications supported by CIHR, while respecting the freedom of researchers to publish where they deem their research will have its greatest impact. 

...

This policy applies to all grants awarded January 1, 2008 and onward, which have received funding in whole or in part from CIHR. While not required, researchers holding grants that were awarded prior to January 1, 2008 are encouraged to adhere to the requirements of this policy.

...

For now, CIHR has decided to limit this policy to peer-reviewed journal publications and publication-related biomedical research data, which is typically deposited into public databases as a condition of publication. CIHR is committed to improving access to research outputs and will explore broadening the policy to include research materials and other research data in the future.

...

• Grant recipients are now required to make every effort to ensure that their peer-reviewed publications are freely accessible through the Publisher's website (Option #1) or an online repository as soon as possible and in any event within six months of publication (Option #2).

• Under the second option, grant recipients must archive the final peer-reviewed full-text manuscripts immediately upon publication in a digital archive, such as PubMed Central or the grantees institutional repository. Publications must be freely accessible within six months of publication, where allowable and in accordance with publisher policies. Grant recipients may use the SHERPA/RoMEO database to locate summaries of publisher copyright policies. The SHERPA/RoMEO database will help grant recipients determine which journals allow authors to retain copyright and/or allow authors to archive journal publications in accordance with funding agency policies. However, CIHR recommends confirming with editorial staff whether archiving the postprint immediately, and making it freely accessible within six months, is permissible.

• Grant recipients may also wish to submit their manuscripts to a journal that provides immediate open access to published articles (if a suitable journal exists). CIHR considers the cost of publishing in open access journals to be an eligible expense under the Use of Grant Funds.

Access to Research Outputs - Modified: 2007-09-04 - Accessed: 2007-09-04
Policy on Access to Research Outputs - September 2007

Disclosure: CISTI is far from a disinterested observer in this development, but as always, if you want any kind of statement about anything CISTI is doing, please contact CISTI Communications, not me.

Previously:
July 21, 2007  Globe and Mail on Open Access
November 29, 2006  CARL and SPARC respond positively to CIHR draft research access policy
October 13, 2006  Institutional Repositories - David Moorman SSHRC

Also see other postings in my Open Access category.

open science and the impact factory

Jean-Claude Bradley instigated a session in Second Life - SciFoo Lives On: Open Science.

Next week will be something like "Medicine 2.0".

You can see in the transcript that one part of SciFoo that definitely lived on was a discussion around Open Science and webliometrics, both definitions and how to handle impact.  It seems to me that we get tangled in endless debates about definitions.  I have proposed that the nodalpoint Open Science wiki page be used to come to a consensus definition, but in the meantime:

open science

opening your scientific activities up to public examination, making work available without it having gone through formal peer review

peer review

The process of a group of scientific peers assessing the quality of a submitted piece of scientific work, currently most commonly associated with gatekeeping into a scientific publication, wherein it may also involve aspects of improving both the scientific thinking used in the paper and the expression thereof.  There is no relationship between peer review and closed or open access.

open access

making a publication available without subscription fee, but possibly with usage limitations

free access

unfortunate term due to existing definition of open access, adding element of unrestricted usage and reuse (e.g. text mining)

impact factor

An imperfect measure of the scientific "importance" of an entire journal.  Misused to measure the quality of individual scientific output

(Marked up using HTML definition lists, which you have probably never heard of, which incidentally is why the Semantic Web will fail.)

Yes, there are many types of peer review in different disciplines, and yes, things are often considered published and citable without having gone through peer review, such as conference papers and presentations which often go through a sort of editorial board selection instead.

I know these definitions are far from perfect, but good lord, can we get to good enough and go beyond this debate?

What I keep hearing is, how can we impact factorize open science.  Well, the answer is, you can't.  Let's stop trying to find some magic algorithm whereby a machine tells us what quality science is.  What's completely mad to me about this is that we already have processes to assess science quality.  Every time you review a new student, every time you look at a grant proposal, heck, even on the infamous tenure committees and research assessments, a group of humans looks at a portfolio of existing or proposed work, and decides whether it is good enough.

So if I may modestly propose, let's continue to do that, and no one other than journal publishers should ever look at impact factor numbers again.  Arise, qualitative assessment, begone quantitive nonsense.

There is still a place for technology, but it's not in providing some bogus seemly-quantitative quality measure.  It's in enabling us all to present our scientific portfolios online, or to use Euan's words, our "professional lifestreams".  And there is a real problem to be solved.  It starts with students and their scholarly output stuck in closed university systems.  Students move around.  Scientists move around.  Their work history should move with them, not be lost in some scholarly dark web, or frozen as some web page at their previous institution that they no longer can access.

The European e-Portfolio is one effort to address this for students.
Electronic Theses and Dissertations is another piece.
The next step is to have those integrate into some, shall we say, flow or... flux (sekrit inside Nature joke) of the rest of their scholarly activity when they graduate.  Bookmarks created, databases curated, papers reviewed, etc. etc.

That's the technology piece.

The other piece, however, cannot be solved with technology.

Find better ways for humans to review scholarly portfolios and make decisions based on them.  That's going to address this problem of evaluation far better than anything else.

SIDEBAR

And of course you can do some side bits with technology of course once you have all this info circulating around, like ranking relevance to help people find the best, most relevant work in the flood of science that is sloshing around.  Usage factor, other metrics, these may all help in recommending things to read.

END SIDEBAR

References

Richard Monastersky, "The Number That's Devouring Science", Chronicle of Higher Education, Volume 52, Issue 8, Page A12 (2005)

The PLoS Medicine Editors, "The Impact Factor Game", PLoS Med 3(6): e291 doi:10.1371/journal.pmed.0030291 (2006)

Peter A. Lawrence, "The Mismeasurement of Science". Current Biology, 17 (15), r583. doi:10.1016/j.cub.2007.06.014 (2007)

Bruno Granier, "Impact of research assessment on scientific publication in Earth Sciences" (PDF), a presentation at ICSTI June 2007 Public Conference on Assessing the quality and impact of research: practices and initiatives in scholarly information

Richard Akerman, "Web tools for peer reviewers...and everyone" (PDF), a presentation at ICSTI June 2007 Public Conference on Assessing the quality and impact of research: practices and initiatives in scholarly information

Corie Lok, "Scifoo: day 1; open science" (2007)

Alex Palazzo, "Scifoo - Day 2 - Science Communication" (2007)

Alex Palazzo, "Scifoo - Day 3 (well that was yesterday, but I just didn't have the time ...)" (2007)

Previously:
June 2007  Science Library Pad: ICSTI 2007 category

impact factor considered harmful

It is not so funny that, in the real world of science, dodgy evaluation criteria such as impact factors and citations are dominating minds, distorting behaviour and determining careers.

Modern science, particularly biomedicine, is being damaged by attempts to measure the quantity and quality of research. Scientists are ranked according to these measures, a ranking that impacts on funding of grants, competition for posts and promotion. The measures seemed, at first rather harmless, but, like cuckoos in a nest, they have grown into monsters that threaten science itself.

The Mismeasurement of Science. Current Biology, August 7, 2007: 17 (15), r583.
doi:10.1016/j.cub.2007.06.014 ( http://dx.doi.org/10.1016/j.cub.2007.06.014 )

via DC's goodscience

via nsaunders Google Starred links

UPDATE 2007-08-20: Also see new posting open science and the impact factory.

the peer review logo

In the session "Reinventing scientific publication (Web 2.0, 3.0, and their impact on science)" led by James Hendler at SciFoo, one of the items was an idea from Geoffrey Bilder, for publishers to provide a "peer review logo" that could be attached to (at this point I am interpreting based on my own understanding) e.g. blog postings, some sort of idea of a digital signature to indicate peer reviewed content.  (I know the list well since I'm afraid my major contribution to the evening, despite having thought about this topic a lot, was transcribing the list).

2) ID, logoing, review status tag, trust mechanisms
- other peer review status

I wonder if we should make a wiki where we list all of the grand (and not so grand) challenges of web science communication and discovery, and then people can pick off projects.  The SciFoo prototypes list is one angle on this.  Of course, in the perpetual-beta web world, it's probably faster to just create a wiki, than to try to start a discussion about whether one should be created.  It's in that "just do it" spirit that I'm pleased to find there is already a peer review logo initiative in the works, although the angle is to indicate that you're writing about a reviewed work, not that your work itself has been reviewed.  From Planet SciFoo:

Cognitive Daily - A better way for bloggers to identify peer-reviewed research, by Dave Munger

[we] have decided to work together to develop such an icon, along with a web site where we can link to bloggers who've pledged to use it following the guidelines we develop

via Bora Zivkovic, via Peter Murray-Rust

(it's strange and also good to be blogging now about people that I've finally met)

UPDATE: I do have a vague idea in a similar space, which would be a "repeatability counter".

As I have learned more about peer review, I have understood that it has many aspects, but preventing fraud is not one of them.  Peer review can help to create a paper that is well-written and has "reasonable" science, but it can't stop a determined fraudster.  (This isn't my insight, but comes from a presentation I saw by Andrew Adrian Mulligan of Elsevier - "Perceptions and Misperceptions - Attitudes to Peer Review".)  What does address fraud, and keep science progressing, is falsifiability: someone else does the experiment and sees if they get the same results.  Now I realise there are many different classes of results, but it's interesting that many of these are not publishable, and are maybe not captured in the current system:

• We tried to repeat the experiment, but it failed because we didn't have enough information on the protocol
• We tried to repeat the experiment, but it failed and we think the paper is in error
• We successfully repeated the experiment
• (probably more scenarios I haven't considered)

So I think it would be interesting to have a sort of "results linking service" where you would click and you would get links to all the people who had tried to reproduce the results, and indications of whether or not they succeeded.  We use citation count as a sort of proxy for this, but it's imperfect, not least of which because there is no semantic tagging of the citation so you don't know if it was cited for being correct or incorrect.  I think this kind of experiment linking might add a lot of value to Open Notebook Science and to protocols reporting (whether in the literature like Nature Protocols, or in a web system like myExperiment).  Otherwise I worry that the amount of raw information from a lab notebook makes it hard to extract a lot of value from it.

UPDATE: Christina in the comments rightly chides me on my loose use of "falsifiability".  Basically I'm trying to get at two aspects of testability: 1) is there enough information in the paper to test the author's claims?  2) What are the results of such a test?

Scholarly impact: from ranking to assessment - Johan Bollen - June 21 - ICSTI 2007 Nancy

This is some very interesting work and a huge project that should greatly enrich our understanding of the usage of scientific information.

Johan Bollen - Los Alamos National Laboratory
"Scholarly impact: from ranking to assessment"

Scholarly evaluation matters
- qualitative and quantitative indicators

many features in scholarly status space
- prestige
- novelty
- visibility
etc.

[MESUR project]

various opportunity to extract metrics in the scholarly life-scucle
- usage data
- review data
- citation data

usage data is available before citation data

public.lanl.gov/jbollen/Publications.html

From ranking to assessment

we're in mode ranking 0.6
- single data source
- single criterion

... to assessment 3.0 [yecch]

- situate item in value landscape
- multiple sources of scholarly information

question: which dimensions to choose?

1) MESUR project
- survey wide range of possible indicators
2) Peer review
- study peer review process

Marko Rodriguez and others

Can we improve on citation data and the impact factor?

- perhaps usage data applies to a larger subset of the scholarly community,
  capturing more scholarly objects and activities beyond journal articles

usage: COUNTER, IRS [?], SUSHI, CiteBase

MESUR: Metrics from Scholarly Usage of Resources

1 ontology to model the scholarly process
2 beg for usage data
3 dedupe
4 create semantic network

2/5th through the project

[ontology]

data 700million usage events and 1 bilion citations
10-15 billion triples

COUNTER logs, item-level data, SFX, etc.

link resolver data very good

[paper in JCDL 2006 about link resolver data gathering architecture]

they are using Franz's AllegroGraph triplestore

Network usage: usage graphs

"we should stop counting: we should look at relationships"

journal network - 50,000 journals

Example: Flow of information

[pretty network]

Metrics survey

many large organizations and sites are participating

U Texas case study...

[my comment: but isn't there an undergrad effect based on the articles they are assigned?]

principle component plot

[paper at JCDL 2007]

many issues and challenges

quality evaluation of scientific publications - Denis Jérome - June 21 - ICSTI 2007 Nancy

Denis Jérome - CNRS, Académie des Sciences
"Evaluation based on scientific publications: experiences in physics"

* public funding is needed for basic research
* evaluation is needed
* can one use publications to evaluate research

[chart showing 64% of (european) physics letters published in US]

Paradox

* Europe is the first (largest) contributor to physics publications
* yet EU is a minor actor for scientific publications

A Mandatory Plurality

* an overwhelming concentration is dangerous
* need a variety of editorial policies

The need for evaluation

* peer review [of grants, and scientists]
* but also bibliometrics

* e.g. Impact Factor

IF is an indicator for publishers
*** misuse of IF for individual evaluation ***

Nature: 25% of articles receive 90% of citations

Nature & Science only have small number of physics papers therefore:
ban IF for evaluation

IF is about journal popularity, not about the actual citations

Need indications about quality

Physicists publish mostly in small number of journals listed in Web of Science

Databases

Google Scholar
Scopus
NASA ADS adsdoc.harvard.edu
ISI Thomson

Hirsh Index: H

Leo Egghe, 2006 "G" index

analysis: G seems to be more reliable than H

Grain of Salt

* clean scientist names [need unique scientist numbers]
* self-citations
* team work
* negative citations
* cronyism
* quality of citations

must be handled by scientists

open access publishing and collaborative peer review - Ulrich Pöschl - June 21 - ICSTI 2007 Nancy

Ulrich presented a very interesting open review model for publications, unfortunately his talk was a bit rushed due to factors outside his control.  Definitely an approach worth investigating further.

Ulrich Pöschl - Max Planck Society
"Interactive open access publishing and collaborative peer review for improved scientific communication and quality assurance"

www.mpch-mainz.mpg.de/~poeschl

* many motivations to do open access
- improve scientific quality assurance

with OA you can do collaborative peer review

problems with scientific publications
- fraud
- carelessness

speed vs quality
- but then neglect thorough review

Two-stage OA publication with collaborative peer review

www.egu.eu
* they [the journals] are financially viable
* they have good impact factor

Bernard F. Schultz - Albert Einstein Institute - Future styles? of assessment

Vision

- OA to high quality scientific publications
- documentation of scientific discussion (e.g. publish referee comments)
- demonstration of transparency and rationalism

Proposition

- prescribe OA to publically funded research
- transfer funds for subscription to OA
- foster OA publishing and collaborative peer review
- mere access is not enough (need to get all layers, data etc.)
- evaluate individual papers
- refine statistical parameters for citation, downloads, usage, interactive commenting and rating

scientific visibility and IF - Bruno Granier - June 21 - ICSTI 2007 Nancy

Bruno Granier - University of Western Brittany
"Impact of research assessment on scientific publication in earth sciences"

- Misuse of IF [impact factor]
citation: "The number that's devouring science"

- The Goal

* the only common goal is how visible you are...
because visibility is the qualitative factor used to assess your work

he started an OA journal - Notebooks in Geology

As an author
- (particularly in industry) you may want paper published ASAP
  - you may reiterate your message in other publications
- in academe you want impact factor

ways to increase visibiity
- slicing
- bogus signatures / invitation
- author names appear in alternate positions in similar papers
- selective or inexact quotations
- self-citation
- cutting and pasting
- lift information

Evaluators should use weighted averages for multi-author papers, 1st authorship worth much more

Question: How to detect frauds?

Answer: You need a good reviewer

As a reviewer

- the reviewer remains the sine qua non of the evaluation process

As an editor

- blacklist repeat offender authors
- use computer programs to detect plagiarism

- often citations are incorrect or not relevant

As a publisher

- OA gives happy google effect

- monitoring
  shall i use new tools/facilities (couunters) to discriminate the kind of papers that
  get the larger readership

Impact Factor

- a huge part of the scientific inofmration is not given any consideration,
since IF covers only well-established journals

The use and misuse of metrics is responsible for the death of many lab, museum etc. publications in
France and elsewhere.

Conclusion

- bibliometrics or not, the only goal remains to increase your visibility

- the Google effect is at our doors

technology and scholarly publishing: audio of SXSW panel

Ok, they called it

Web 2.0 and Semantic Web: The Impact on Scientific Publishing (MP3)

but I refuse to use such buzzterms.  Or to call it a podcast when it's just an audio file. Anyway, it's interesting listening.  The panel members are

Moderator: John Wilbanks Exec Dir of Science Commons, Creative Commons

Matthew Cockerill   Publisher,   BioMed Central Ltd
Melissa Hagemann   Program Mgr,   Open Society Institute
Timo Hannay   Dir of Web Pub,   Nature Publishing Group
Amit Kapoor   Managing Dir,   Topaz

from south by southwest festivals 2007
via Peer-to-Peer

ChemRank

Mitch just launched ChemRank, a website where we can comment on and vote thumbs up or down for scientific articles.

via chem-bla-ics

So far what it appears to tell us is that the Blue Obelisk chemoinformatics people like articles about Blue Obelisk and chemoinformatics...

publicationslist.org

Seems to me that this is a niche within a niche, but it is nevertheless a real need.

PublicationsList.org exists to let researchers and research organizations maintain a reliable web-based record of their academic output without any fuss.

They will also provide (for a fee) space for hosting self-archived papers (although I would imagine you have to check your publisher rights, usually it says something like "on your work web site or in your institution's repository").

They do have a number of options for embedding the generated list in your own website / blog etc., which is very nice to see.

via HubLog

cyberinfrastructure for the humanities

In Data Mining, Collaboration, and Institutional Infrastructure for Transforming Research and Teaching in the Human Sciences and Beyond, Cathy Davidson discusses many aspects of how cyberinfrastructure can and is transforming teaching and research

That brings me to another point, which may appear tangential but which is at the heart of the matter. New ways of thinking need support. If, at present, academic rewards go to the author of a monograph, especially one that posits a different analytical or interpretive hypothesis, for Human Sciences 2.0 we need to think of ways to reward teams of scholars working cross-culturally on collaborative projects. Collaborative work should count, and here humanists can use models that scientists have developed for determining credit in co-authored projects with multiple investigators.

...

We also need to rethink paper as the gold standard of the humanities. If scholarship is better presented in an interactive 3-D data base, why does the scholar need to translate that work to a printed page in order for it to “count” towards tenure and promotion? It makes no sense at all if our academic infrastructures are so rigid that they require a “dumbing down” of our research in order for it to be visible enough for tenure and promotion committees.

CTWatch Quarterly, Volume 3, Number 2, May 2007.

As a side note, the Cyberinfrastructure Technology Watch (CTWatch) blog relaunched on May 25, 2007.

Previously:
March 25, 2005  Cyberinfrastructure Tech Watch Blog operational

link resolver issues studied by UKSG

The UK Serials Group report Link Resolvers and the Serials Supply Chain (May 21, 2007) covers the components of the OpenURL resolving equation - OpenURLs, resolvers, Knowledge Bases, and a critical component: the data provided by publishers.

In a perfect world, automated and entirely correct data would flow from publishers directly into our resolver knowledge bases.  It is not a perfect world.

The report discusses some of the major issues and proposes some solutions as well as areas for further research.

One of the main things I get from it is that everyone involved needs to understand how resolvers work, and how correct data benefits everyone in the "serials supply chain," from publisher to end user.

via LiveSerials blog
with more info in LiveSerials blog posting Framework for Improving Link Resolver Systems

2007 IATUL and ELPUB programmes - open access and beyond

The ELPUB 2007 and IATUL 2007 programmes are up.  Both will feature topics in the areas of open access and scholarly communication.  It's interesting to me how much the technology and interests of the scholarly publishing community and the academic library community seem to be converging.

At ELPUB 2007, CISTI's Judy Best will be presenting

Challenges in the Selection, Design and Implementation of an Online Submission and Peer Review System for STM Journals

my presentation on Internet community for Allen Press Emerging Trends

Here's my presentation "The Internet - A Scholarly Community?" from the 2007 Allen Press Emerging Trends seminar:

Download AllenPress.ppt (converted to PowerPoint)

Original Keynote format available upon request.

I have also posted it to Slideshare.net

There are supplementary bookmarks available at

http://www.connotea.org/user/scilib/tag/ap2007akerman

I thought the presentation went well.  A lot of the content is in me talking, so I don't know how much you'll get from just the slides.

Sidebar 1: Using K790 as Bluetooth Remote

I tried using my K790 phone as a remote control but that was not a big success for a couple reasons:
1) It kept going to sleep, so instead of just clicking to the next slide, I first had to hit a safe button to wake it up, then click to the next slide
2) Although Bluetooth should have good range (10 metres), I wasn't able to control from either end of the stage and at at least one point, I lost the connection altogether and had to re-establish it

So: not recommended unless you're going through your slides quickly (or know how to disable sleep mode) and you're fairly close to the podium.

Sidebar 2: Carbon Offset for Travel

I'm doing double carbon offset, on the theory that at least one of them might do some good:

Green My Flight Summary
Itinerary:
OTTAWA - YOW to WASHINGTON - IAD: 725.0 km

Total Distance: 1449.9 km
Total Emissions: 197.0 kg
Program Cost: $7.00

MyClimate

flight from: Ottawa, ON [Macdonald-Cartier International Airport], Canada, YOW
flight to: Washington, DC [Washington Dulles International Airport], USA, IAD
return, economy
flight distance: 1'448 km
flight passengers: 1
CO2 Emissions: 0,34 t
Total costs for compensation of your flight: SFr. 13.71
+ SFr. 5 "handling charge"
+ credit card foreign exchange fees

(Google says 18.71 Swiss francs = about 17.50 Canadian dollars)

thoughts and notes on Allen Press Emerging Trends Seminar

I enjoyed the Allen Press seminar, I really value the opportunity to meet with people working on ways to support scholarly activity using technology, and to see presentations of some of their work.

The strongest theme I heard from the technology side was a plea for increased semantic markup of journals, to build the "web of data".

I find it interesting that the more I progress in my career, the more I find most challenges are about people/"humanities" issues and finances - I often wonder whether some courses in human behaviour and economics should be mandatory for all technologists.

In this vein, some other themes I heard were more about psychology and sociology:
- can we find ways to recognize and reward scientific contributions outside of the published paper?
- if you're not providing an explicit reward, can you find ways to tap into activities that researchers already are doing (e.g. helping them to capture and annote citations, as Zotero does)
- is it possible to construct systems that discourage negative behavior (e.g. to prevent "flamewars" of vitriolic discussion)

As I suggested in my talk, I think we need to leverage knowledge from existing systems, like the "ignore list", moderation, FAQs, and other conventions of USENET discussion groups.

My presentation will be online within the next few days, it's been ready for months, but I still need to make sure all of the supporting Connotea bookmarks are available.  The presentations are also supposed to be up on the Allen Press site within a few days.

UPDATE 2007-04-15: My presentation is now online.  ENDUPDATE

By some sort of technogeek harmonic convergence, I ended up at the speakers' dinner the night before sitting with Konrad Förstner (homepage, blog) and Josh Greenberg.  Not only did we have a lively discussion, it worked out well because the three of us were all speaking in the same section of the programme, at the start of the day.

Konrad's presentation is already available: Revolutionizing scientific communication and collaboration (PDF).

I took some raw notes on a few of the presentations.
I don't know how useful they are, but since this work blog started as a place to hold my conference notes and make them searchable, it seems appropriate to continue this tradition.

Donald King

sources used by researchers for their work
- journal article 35%
- discussion with colleagues 20%

the ultimate factor for an information channel is its accessibility

Competition for scientists' time
- additional time is due to communication activity

National Survey

* key attribute: trustworthiness
* followed by timeliness
* followed by quality

trustworthiness of websites (as judged by surveyed researcher) was low

trends in scientists' reading patterns

* reading more
* rely on libraries more

everyone has to continue to learn

trends in how articles are identified: no big change 1977 to 2006

main shift is from A&I to online searching

most readers obtain articles from library collections, not preprint archives

252 artices read per scientist per year (increase from 150 in 1977)
source is about 50/50 library collection and other (131/121)

reasons
* decrease in personal subscriptions
* more reading of articles identified by online bib searches
** electronic collections have broadened access to articles

a lot more articles now identified by automated searches, 63 per scientist now

library collections: about 80% electronic use - this saves readers about 20 hours/year
breadth of reading has increased, read double the journals from 1977

time per article has declined from 48 minutes to 34

Article Age - older articles are judged more valuable and are more likely to come from libraries

(decline in personal paper journal libraries)

Reading of Older Materials may be increasing (with digitization)

* Libraries will continue to be an essential participant in the journal system for the foreseeable future

* Newly proposed system models need to be advance cautiously and with demonstrable successes

Q: Are researchers still browsing in the same way?
A: Mostly from personal paper subscriptions.
Also browsing means keeping up with literature in general.

------------------------

Jaron Lanier
"Avoiding Mob Science"

about Ted Nelson

about history of Internet/networking

people as "pack animals"
social roles

when you allow anonymous comments on blog entries... it can evolve into fighting packs

Internet allows "increased efficiency" of harassment

How to use governance to bring out the best in people?

morality and economics

open source: a lot of talent spent duplicating things that were already available, and diverted into
politics

we can start to observe now what does and doesn't work on the web

- ArXiv (quiet "kook filter")
- edge.org is similar

having a lot of openness is good, but total opennes is a disaster

filtering "can be cheap but it can't be free"

economic challenges - Web 2.0 seems to transmute volunteer labour into money

hard to make "partially open systems"

the iPod is a dongle

"Can we physicalize scholarly publishing in some new way?"

be realistic in adopting Web 2.0, prepare for negative behavior, design to manage it
preserve the value of scholarly publications and the scholarly process

Q: Could a tshirt be a way to physicalize a journal?

(Richard's comment: "I wrote this paper and all I got was this lousy tshirt" :)

-----------------------------------------------

Panel: The Great Promise of Research Data Commons

slide showing decline of availability of (collected/generated) research data over a creator's lifespan

John Wilbanks
"The Research Data Commons"

a commons approach makes research go faster, and makes it easier to use technologies that don't yet exist

seems to be easier for fundamental data vs. individial experiments

example of Bermuda agreement

to make the sharing possible, you need markup and storage

when there are gaps in the data, you block the ability to map the network of knowledge

"we need to treat the literature itself as data, because we need computers to process it for us"

"this has to be about answering questions, not technology"

the commons will help scientists to ask better questions

three things for publishers to do:

- markup
- provenance - e.g. link to other stuff from same author
- trust

sw.neurocommons.org

-----

Steve Bryant
PubChem
"An open information resource linking chemistry and biology"

Entrez
1,500,000 users/day
50,000,000 hits/day

having the data available lets you do many computations with it,
as well as comparisons with other data sets

NIH Roadmap

"GenBank model"
highly automated

18,000,000 chemical structure records

links to contributor web sites and other NCBI databases

about 35,000 people/day

optimize discoverability

(complex example)

exploratory structure-discovery tools

(Richard says: free the humans)

-------

Robert Peet
"Data sharing and exchange: Experiences within the Ecological Society of America"

supplemental data

way to cite and find data

need to change the culture to encourage deposit of supplementary info

vegbank.org

To be published somehow as online proceedings.  "U.S. National Vegetation Classification"
- dynamic standard

www.ecoinformatics.org

metadata, datasets, workflow

Taxonomic database challenge: Standardizing organism and community names

* movement within ESA to require access to data

Set standards
- formats
- archiving and access

* assure quality control (peer review)

Digital repositories and libraries
* archive and provided access
* meet instituational responsibility to granting agencies
* (one I missed)

Publishers
* Embed citation links for standard data elements
* (others)

Q: Open Text Mining ?
A: Creative Commons is trying to jumpstart

------------------------

Peer Review, Dynamic Documents, and the Wisdom of Crowds

Irv Rockwood

Recommends as a good overview "Peer Review in Health Sciences" (PDF), from the BMJ

"it's easy to criticize peer review, but it's harder to come up with a better system"

New possibilities:
- open peer review
- author selects reviews
- post-publication review

ETAI all articles posted for 90 day review
- revise, pass-fail

Biology Direct
- author selects reviewers
- reviews published alongside author's response

PLOS One
- prepub review on technical issues
- open peer review

Is editorial peer review broken, and if so, what should we do about it?

Jaron Lanier

- there's a tendency to (try to) use Web 2.0 to replace hard work with superficial easy work

Chris Surridge
PLoS ONE

- peer review slow, inefficient, more papers than ever

asked to do three things
1. separate good from bad
2. select papers for journals
3. give status to a paper
(4. improve the paper)

academic editors decide whether to publish (perhaps using some degree of peer review)

David Baldwin
Eco Society of America (Ithaca, NY)

- humanities has system of "working papers"

Q: different models of peer review for different disciplines?
A: yes

peer review and research assessment - 2007 ICSTI Public Conference

The 2007 ICSTI Public Conference "Assessing the quality and impact of research: practices and initiatives in scholarly information" will be

June 21-22, 2007
Nancy, France

I am pleased to announce that I will be speaking about peer review and certification.
(Following Elsevier, the largest scientific publisher in the world, so, you know, no pressure.)

You can check out the preliminary programme; more info is available on the ICSTI site.

I will blog this event under category/tag ICSTI2007.

Following the event, there is the opportunity to tour INIST on the afternoon of the 22nd.

(Note that the ICSTI General Assembly, which is closed to the public, continues for two days after the public conference.)

Getting There

There is a new TGV line running direct Paris-Nancy that is supposed to open June 10, 2007; I can see the train schedules at voyages-sncf.com, but it won't let me book them.  The new line is TGV Est Européen, and the TGV Est site says tickets will be bookable starting April 10, 2007.  Also, if I'm reading correctly, there are discounted prices from June 10, 2007 to August 26, 2007.  If the TGV is running, it will only be 1.5 hours for the Paris-Nancy run, e.g.

Otherwise if you don't take a direct, it is showing times of about 3 hours in June.  Currently, without the new line, it is showing direct times of about three hours, and non-direct times of about 4 hours.  I think you can reserve and pick up your tickets in France.  Otherwise for Canada and USA, the site is Rail Europe, but it doesn't show the new TGV schedules for June.  For Canada see specifically

http://www.raileurope.ca/canada/

For booking the new TGV East from Canada there is a new page, but just to make things a hassle, it doesn't list Nancy as an option in the pull-down menus

http://www.raileurope.ca/canada/rail/tgvest/

UPDATE 2007-05-10: The TGV Est leaves from Gare de l'Est (PARIS EST).  To get there from Charles de Gaulle (CDG) Roissy airport, you can take the "Roissyrail" RER B train from the airport to Gare du Nord, and then walk to Gare de l'Est (it is just a few hundred metres away).  You can either go outside at Gare du Nord, or there is supposed to be a connecting hallway ready by the time the TGV Est starts running.

The RER B is easy to get from within the airport, you can buy tickets at automated kiosks or line up to buy them from the SNCF ticket counters, you can of course also order them in advance online.  It should only take about half an hour from CDG to Gare du Nord (although you might want to figure an hour, just to be safe).  There are two trains, a local and an express, they leave from opposite sides of the same platform.  Look at the signs beside each track - dots will indicate the stops the train makes.

ENDUPDATE

UPDATE 2007-06-17: You can check the status of the train lines, as well as individual trips, via

http://www.infolignes.com/

specifically you can check SNCF - Infolignes - Trafic Est

ENDUPDATE

UPDATE 2007-04-17: I have made a Google Earth KMZ of the suggested hotels and the INIST venue (which is about 5km from Nancy centre).

Download ICSTI2007.kmz

You can also view these placemarks in Google Maps.

By copying the placemarks using "Save to My Maps", I have also created a Google My Maps version.

The INIST site has information both on how to get to Nancy and how to get to INIST itself.

The tram1 runs directly from downtown to INIST.  The INIST site says to take it in direction Vandoeuvre CHU Brabois and get off at the Faisanderie stop.

See reseau-stan.com for more information on Nancy area transit.

Disclaimer: I'm approximating the location of INIST based on what I could find on their website; it's possible the location is off slightly as I've never been there.

ENDUPDATE

Previously:
January 22, 2007  ICSTI event on academic user behaviour
April 28, 2006  conference proceedings: ICSTI eScience, Bielefeld + Taiga + LtF on academic library future

Note:
ICSTI = International Council for Scientific and Technical Information
CISTI = Canada Institute for Scientific and Technical Information
INIST = L’Institut de l’Information Scientifique et Technique

e-journal preservation workshop presentations

Lots of interesting presentations available from the March 27, 2007 E-Journal Archiving and Preservation Workshop organized by the UK Digital Preservation Coalition.

Blackwell Publishing carbon neutral

Every organization contributes to carbon dioxide emissions through its day to day operations. The scientific consensus is that increasingly levels of CO2 emissions are causing climate change, and that this is potentially the most serious threat facing the world today.

Blackwell Publishing has decided to take action and become a CarbonNeutral organization. Blackwell is the first major publisher to make the investment to become CarbonNeutral and we hope that we will lead the way for others in the industry.

To begin this project, Blackwell worked with The CarbonNeutral Company to identify our ‘carbon footprint,’ or the approximate volume of CO2 generated by the organization. It is calculated by assessing energy usage, business travel, and all components of the company’s operations that consume power or generate waste and by-products.

http://www.blackwellpublishing.com/carbonneutral/

eigenfactor journal ranking

Borrowing methods from network theory, eigenfactor.org ranks the influence of journals much as Google’s PageRank algorithm ranks the influence of web pages [2]. By this approach, journals are considered to be influential if they are cited often by other influential journals. Iterative ranking schemes of this type, known as eigenvector centrality methods [3], are notoriously sensitive to “dangling nodes” and “dangling clusters”: nodes or groups of nodes which link seldom if at all to other parts of the network. Eigenfactor modifies the basic eigenvector centrality algorithm to overcome these problems and to better handle certain peculiarities of journal citation data.

Website provides two measures

Article Influence (AI): a measure of a journal's prestige based on per article citations and comparable to Impact Factor.
Eigenfactor (EF): A measure of the overall value provided by all of the articles published in a given journal in a year.

So if I understand it, it's basically a free journal-level impact factor calculated using network analysis.

http://www.eigenfactor.org/

via email from Carl T. Bergstrom

article metrics: Faculty of 1000 biology and medicine

Key articles selected and evaluated by a global faculty of top researchers and clinicians.

Faculty of 1000 Biology and Medicine are authoritative online services in which almost 5,000 leading researchers and clinicians share their expert opinions by highlighting and evaluating the most important articles in biology and medicine.

http://www.facultyof1000.com/

It is a subscription service, but you can get a flavour of it from the page of "Hidden Jewels"

http://www.f1000biology.com/top10/jewels

It includes links to full text or to purchase the article.
Articles receive an "F1000 factor", by analogy to classical Impact Factor, as well as additional ratings, e.g. "Must Read", "Exceptional".

Another example of a "soft" peer review system that is yet quite rigourous (one assumes) and, surprisingly, not free.

via a talk by Francis Ouellette, Open Access at the CIHR? (PowerPoint, or available as a PDF)

Euro report: scientific information in the digital age

A remarkably concise (10 pages) report for a commision

scientific information in the digital age: access, dissemination and preservation (PDF)

http://ec.europa.eu/information_society/activities/digital_libraries/doc/scientific_information/communication_en.pdf

In order to become an increasingly competitive knowledge-based economy, Europe must improve the production of knowledge through research, its dissemination through education, and its application through innovation. All research builds on former work, and depends on scientists’ possibilities to access and share scientific publications and research data. The rapid and widespread dissemination of research results can help accelerate innovation and avoid duplication of research efforts, although some delay for the first use by researchers or for commercial purposes can be justified.

...

Initiatives leading to wider access to and dissemination of scientific information are necessary, especially with regard to journal articles and research data produced on the basis of public funding. With respect to journal articles, the Commission is observing and considering experiments with open access publishing.

Fully publicly funded research data should in principle be accessible to all, in line with the 2004 OECD Ministerial Declaration on Access to Research Data from Public Funding.

Moreover, the Commission draws particular attention to the need for clear strategies for the digital preservation of scientific information.

via CISTI's Mary Low

UPDATE 2007-03-09: N. Miradon has posted a comment (to a mailing list as we had problems with TypePad comments).

BioMed Central OA event presentations, BMC blog

Slides and audio (MP3) are available for various presentations from the BioMed Central Open Access Colloquium held on February 8, 2007.

I was particularly interested by

UK PubMed Central within the open access movement by Richard Boulderstone, Director of e-Strategy and Programmes, British Library - PowerPoint, MP3 audio

via Francis Ouellette, indirectly

I also notice that BMC has various blogs, which appear to have all launched in January-February of this year.  Lots of interesting info about various tools to explore articles and data.

To keep you up to speed with the latest developments at BioMed Central, and to give you our take on relevant news from elsewhere, we're pleased to launch the BioMed Central blog, along with a Chemistry Central blog and a PhysMath Central blog.

Previously:
January 08, 2007  UK PubMedCentral (UKPMC) launches
September 08, 2006  the concepts of open science and information commons

soft peer review

Via Library 2.0 - Social Software and New Opportunities for Peer Review, I find a fantastic posting about the many different ways in which formal "hard" peer review can be enhanced by open web technologies:

Academic Productivity blog - Soft peer review? Social software and distributed scientific evaluation

Although I don’t think social software will ever replace hard evaluation processes such as traditional peer review, I suspect that soft evaluation systems(as those made possible by social software) will soon take over in terms of efficiency and scalability.

This is very much the position I took in my submission to the Nature peer review web focus, and which I will be re-emphasizing with a community-interaction angle in my upcoming Allen Press Seminar presentation.  I really like the comprehensive view that Academic Productivity has taken in looking at the many types of peer evaluation that can be provided on the open web, and the call for more APIs exposing more data for all of us to use.

I do think the debate about "open" peer review vs. traditional peer review is a bit of a red herring, and it very much concerns me when people suggest that open review can replace traditional (or in the language of this posting "hard") peer review.  We have already had open peer review for years, it's called preprint feedback, mailing lists, ArXiV, letters to the editor... it's a tremendous addition to, but not replacement for, the rigourous anonymous peer review system needed to provide a publication filter.