thoughts and notes on Allen Press Emerging Trends Seminar

I enjoyed the Allen Press seminar, I really value the opportunity to meet with people working on ways to support scholarly activity using technology, and to see presentations of some of their work.

The strongest theme I heard from the technology side was a plea for increased semantic markup of journals, to build the "web of data".

I find it interesting that the more I progress in my career, the more I find most challenges are about people/"humanities" issues and finances - I often wonder whether some courses in human behaviour and economics should be mandatory for all technologists.

In this vein, some other themes I heard were more about psychology and sociology:
- can we find ways to recognize and reward scientific contributions outside of the published paper?
- if you're not providing an explicit reward, can you find ways to tap into activities that researchers already are doing (e.g. helping them to capture and annote citations, as Zotero does)
- is it possible to construct systems that discourage negative behavior (e.g. to prevent "flamewars" of vitriolic discussion)

As I suggested in my talk, I think we need to leverage knowledge from existing systems, like the "ignore list", moderation, FAQs, and other conventions of USENET discussion groups.

My presentation will be online within the next few days, it's been ready for months, but I still need to make sure all of the supporting Connotea bookmarks are available.  The presentations are also supposed to be up on the Allen Press site within a few days.

UPDATE 2007-04-15: My presentation is now online.  ENDUPDATE

By some sort of technogeek harmonic convergence, I ended up at the speakers' dinner the night before sitting with Konrad Förstner (homepage, blog) and Josh Greenberg.  Not only did we have a lively discussion, it worked out well because the three of us were all speaking in the same section of the programme, at the start of the day.

Konrad's presentation is already available: Revolutionizing scientific communication and collaboration (PDF).

I took some raw notes on a few of the presentations.
I don't know how useful they are, but since this work blog started as a place to hold my conference notes and make them searchable, it seems appropriate to continue this tradition.

Donald King

sources used by researchers for their work
- journal article 35%
- discussion with colleagues 20%

the ultimate factor for an information channel is its accessibility

Competition for scientists' time
- additional time is due to communication activity

National Survey

* key attribute: trustworthiness
* followed by timeliness
* followed by quality

trustworthiness of websites (as judged by surveyed researcher) was low

trends in scientists' reading patterns

* reading more
* rely on libraries more

everyone has to continue to learn

trends in how articles are identified: no big change 1977 to 2006

main shift is from A&I to online searching

most readers obtain articles from library collections, not preprint archives

252 artices read per scientist per year (increase from 150 in 1977)
source is about 50/50 library collection and other (131/121)

reasons
* decrease in personal subscriptions
* more reading of articles identified by online bib searches
** electronic collections have broadened access to articles

a lot more articles now identified by automated searches, 63 per scientist now

library collections: about 80% electronic use - this saves readers about 20 hours/year
breadth of reading has increased, read double the journals from 1977

time per article has declined from 48 minutes to 34

Article Age - older articles are judged more valuable and are more likely to come from libraries

(decline in personal paper journal libraries)

Reading of Older Materials may be increasing (with digitization)

* Libraries will continue to be an essential participant in the journal system for the foreseeable future

* Newly proposed system models need to be advance cautiously and with demonstrable successes

Q: Are researchers still browsing in the same way?
A: Mostly from personal paper subscriptions.
Also browsing means keeping up with literature in general.

------------------------

Jaron Lanier
"Avoiding Mob Science"

about Ted Nelson

about history of Internet/networking

people as "pack animals"
social roles

when you allow anonymous comments on blog entries... it can evolve into fighting packs

Internet allows "increased efficiency" of harassment

How to use governance to bring out the best in people?

morality and economics

open source: a lot of talent spent duplicating things that were already available, and diverted into
politics

we can start to observe now what does and doesn't work on the web

- ArXiv (quiet "kook filter")
- edge.org is similar

having a lot of openness is good, but total opennes is a disaster

filtering "can be cheap but it can't be free"

economic challenges - Web 2.0 seems to transmute volunteer labour into money

hard to make "partially open systems"

the iPod is a dongle

"Can we physicalize scholarly publishing in some new way?"

be realistic in adopting Web 2.0, prepare for negative behavior, design to manage it
preserve the value of scholarly publications and the scholarly process

Q: Could a tshirt be a way to physicalize a journal?

(Richard's comment: "I wrote this paper and all I got was this lousy tshirt" :)

-----------------------------------------------

Panel: The Great Promise of Research Data Commons

slide showing decline of availability of (collected/generated) research data over a creator's lifespan

John Wilbanks
"The Research Data Commons"

a commons approach makes research go faster, and makes it easier to use technologies that don't yet exist

seems to be easier for fundamental data vs. individial experiments

example of Bermuda agreement

to make the sharing possible, you need markup and storage

when there are gaps in the data, you block the ability to map the network of knowledge

"we need to treat the literature itself as data, because we need computers to process it for us"

"this has to be about answering questions, not technology"

the commons will help scientists to ask better questions

three things for publishers to do:

- markup
- provenance - e.g. link to other stuff from same author
- trust

sw.neurocommons.org

-----

Steve Bryant
PubChem
"An open information resource linking chemistry and biology"

Entrez
1,500,000 users/day
50,000,000 hits/day

having the data available lets you do many computations with it,
as well as comparisons with other data sets

NIH Roadmap

"GenBank model"
highly automated

18,000,000 chemical structure records

links to contributor web sites and other NCBI databases

about 35,000 people/day

optimize discoverability

(complex example)

exploratory structure-discovery tools

(Richard says: free the humans)

-------

Robert Peet
"Data sharing and exchange: Experiences within the Ecological Society of America"

supplemental data

way to cite and find data

need to change the culture to encourage deposit of supplementary info

vegbank.org

To be published somehow as online proceedings.  "U.S. National Vegetation Classification"
- dynamic standard

www.ecoinformatics.org

metadata, datasets, workflow

Taxonomic database challenge: Standardizing organism and community names

* movement within ESA to require access to data

Set standards
- formats
- archiving and access

* assure quality control (peer review)

Digital repositories and libraries
* archive and provided access
* meet instituational responsibility to granting agencies
* (one I missed)

Publishers
* Embed citation links for standard data elements
* (others)

Q: Open Text Mining ?
A: Creative Commons is trying to jumpstart

------------------------

Peer Review, Dynamic Documents, and the Wisdom of Crowds

Irv Rockwood

Recommends as a good overview "Peer Review in Health Sciences" (PDF), from the BMJ

"it's easy to criticize peer review, but it's harder to come up with a better system"

New possibilities:
- open peer review
- author selects reviews
- post-publication review

ETAI all articles posted for 90 day review
- revise, pass-fail

Biology Direct
- author selects reviewers
- reviews published alongside author's response

PLOS One
- prepub review on technical issues
- open peer review

Is editorial peer review broken, and if so, what should we do about it?

Jaron Lanier

- there's a tendency to (try to) use Web 2.0 to replace hard work with superficial easy work

Chris Surridge
PLoS ONE

- peer review slow, inefficient, more papers than ever

asked to do three things
1. separate good from bad
2. select papers for journals
3. give status to a paper
(4. improve the paper)

academic editors decide whether to publish (perhaps using some degree of peer review)

David Baldwin
Eco Society of America (Ithaca, NY)

- humanities has system of "working papers"

Q: different models of peer review for different disciplines?
A: yes