Science Library Pad

Recognizing an open data success

Note: This is a technical examination of available COVID-19 open data.  For general information please visit your relevant local website, including:

• Canada.ca/coronavirus
• Ontario.ca/coronavirus
• OttawaPublicHealth.ca/coronavirus

Summary

Sometimes when advocating for a particular change, it can be hard to step back and recognize the progress that has been made.

COVID-19 open data in Ontario is available and is provided in a way that meets many of the goals that open data advocates had set out.  The City of Ottawa and the Province of Ontario both prominently link directly to COVID-19 open data on their dashboard pages.  The Government of Canada also links to open data, but not as prominently or consistently.  There is also an academic site providing data and code for Ottawa.  All sites provided clear licensing.

These data visualisations and open data can be considered a kind of infrastructure.  In the context of academic use of open data, particularly with the addition of open source computer code, these sites can even be considered part of a digital research infrastructure.

Goals for Open Data

There were several elements that were envisioned for open data in general:

• available
• open license
• usable - clear descriptions
• usable - machine readable
• visible, in the sense that clear links to underlying data would be provided e.g. on websites
• discoverable through search

Basically, that anyone could find it and use it.  That it would provide a usable machine-readable alternative to scraping HTML web tables or hand-extracting data from PDF or Word documents.  And also that it would be provided from every organisation at every level of government.

For sustainability, it was also hoped that open data would be part of each organisation's own internal processes:

• used by the service providers themselves

There is an additional key element I have to say wasn't strongly considered by many of the initial advocates, which is that there should be tools provided to view and work with the data without having to go in and do your own spreadsheets or write your own code:

• tools are provided for anyone to work with the data

And I would add lastly, as the community thought through the provision of tools (e.g. website visualisations):

• are the tools themselves made available for open reuse, e.g. can you access the underlying code and pipelines that are used to make the website views

These goals sometimes seemed very far off.  But with COVID-19 data in Ontario, I would consider it a success.

COVID-19 open data is available at the municipal level (e.g. City of Ottawa), at the provincial level, and federally.  (And beyond, through international aggregation sites.)

 

City of Ottawa

Ottawa COVID-19 Open Data

Ottawa's open data site is https://open.ottawa.ca/

COVID-19 open data can be found just using "COVID-19" as a search.  In this case I've also filtered for just CSV format files.

https://open.ottawa.ca/search?q=covid-19&type=csv

There are 8 datasets, including:

• COVID-19 Cases and Deaths in Ottawa
• COVID-19 Weekly Cases and Rates by Age in Ottawa (Last 6 Weeks)
• COVID-19 Reproduction Number (R(t))
• Data tables for Public COVID-19 Maps

The only one that doesn't seem to be updated frequently is "Data tables for ONS Neighbourhood COVID-19 Maps".

So the good news is the data exist, the data are being kept up-to-date, and the data are in a usable, machine-readable format.

So let's look at three more aspects:

• Is the data surfaced with tools for people to work with the data?
• Is the data clearly linked when it is used?
• Do the city's own websites/tools use the open data (and is it clear that they do so)?

Ottawa COVID-19 website

The main general use site is Ottawa Public Health - Statistics on Coronavirus 19 (COVID-19) in Ottawa, it has links to:

• Daily COVID-19 Dashboard
• COVID-19 Vaccination Dashboard
• Mapping Products
• Wastewater COVID-19 Surveillance
• Projections

The dashboard is in Microsoft Power BI, which I have seen used on a number of COVID-19 information sites.

The data pipeline is clear, but I don't think it is one that is available to the public: "Data are extracted from [the provincial Case and Contact Management (CCM) reporting system] at 3 pm daily and loaded to the dashboard the following day."  In other words, it doesn't look like the Power BI visuals are generated directly from the Ottawa open data, but instead from the provincial closed system.

Open Data Clearly and Prominently Available

However, the really good news is right below the explanation of the data source used (CCM), and before showing the dashboard, the page links directly to the open data: "The following data tables are available for download on Open Ottawa". 

So this meets a key goal, it makes the data a kind of equal, so that you easily click the links and do your own analysis whether in a spreadsheet, by writing code, or using other data analysis tools.

It's hard to overstate what a big change this is in how data is provided to the general public.  Linking to the open data at all is a big deal, but linking to it right at the top before the visualisation is a really strong and clear statement of openness.

There is a gap in that as far as I know there's no way to access the underlying Power BI setup (which incidentally generates five different pages, use the < > way down at the bottom of the Power BI frame on the page to see other Power BI pages), which means that you can see the really complicated visuals but you don't get much insight into how they're made or how you could use Power BI to make your own.

There is also a gap in that, I guess due to different cut-off time for updates, the municipal dashboard numbers are different from the provincial ones.

In terms of discoverability there are basically two pathways to the open data, one directly from Open Ottawa and one from Ottawa Public Health, but they are not quite equal.

If you land directly on Open Ottawa, you would find out it has COVID-19 open data even without a search as they are surfacing several datasets (I would guess this is automated based on traffic to the datasets, but it could be manually curated).

But you wouldn't find wastewater data, because it's on a different site https://613covid.ca/wastewater/

The wastewater site uses more of an academic format, and presents all of the data visualisations up front, before you find out below that you can "See the Methods page for more information on how the samples were collected, access to the data, and how the plots were created." 

There are some issues at this point that are more web design and user-centric design than data.  For example the Methods page is actually called model, and you can't get directly to the top of the page by clicking the Methods menu item, you have to click "Projection models" within that menu.  If you're looking for data you're more likely to pull down the Methods menu and go to "Plot data", but that won't take you directly to the open data - basically you will have to navigate around the page quite a bit before you find out that the data are actually linked under "Waterwater surveillance".  The data are made available on GitHub.

https://github.com/Big-Life-Lab/covid-19-wastewater

So not only could the wastewater site benefit from more directly and clearly linking to the GitHub data near to where it is shown in visualisations and tables, but you can't find the wastewater data at all if you start on the Open Ottawa site.  So there is a gap in integrating these different open data sources.  However, the use of GitHub does provide another path for discoverability, and adds the ability to easily share not just data but also code.

The situation for data is similar for COVID-19 projections, which can be found at

https://613covid.ca/

The source data for the projection models is the Ottawa open data, which is a good clear reuse of the data directly.  The projection models themselves are drawn from a variety of clearly linked sources.  You can see that the wastewater data is kind of added on to the models, which may be one reason for some content visibility issues in the website design.  By navigating on GitHub I can also find that the 613covid.ca projections website is generated from

https://github.com/Big-Life-Lab/Ottawa-COVID-Projection

but I don't think you can find this on the projection website itself.  It's good that the website content is on GitHub as it means pull requests can be submitted with suggested changes; however creating and submitting pull requests for a site built from GitHub is a high barrier for most users.

 

Province of Ontario

Ontario COVID-19 Open Data

The Province of Ontario's open data site is https://data.ontario.ca/

COVID-19 open data can be found just using "COVID-19" as a search.  In this case I've also filtered for just CSV format files.

https://data.ontario.ca/dataset?keywords_en=COVID-19&res_format=CSV

There are 19 dataset pages, which may link to more than one dataset per page.  For example, to find daily new cases you first go to the page

Status of COVID-19 cases in Ontario

and then select the dataset

Daily change in cases by PHU

(PHU means Public Health Unit.)

Having a full page means that a lot of explanatory and related information can be provided.  The data website itself has good functionality including the ability to preview a CSV dataset on the website itself as a grid (spreadsheet-like view), as a graph, or as a map.

https://data.ontario.ca/dataset/status-of-covid-19-cases-in-ontario/resource/8a88fe6d-d8fb-41a3-9d04-f0550a44999f

This is a great way to allow people to explore data without needing their own tools.

There is also an API available for the datasets.

https://data.ontario.ca/api/1/util/snippet/api_info.html?resource_id=8a88fe6d-d8fb-41a3-9d04-f0550a44999f

Ontario COVID-19 website

The main general use site is COVID-19 (coronavirus) in Ontario.

Right below a website display of COVID-19 cases, it has a prominent button:

Get COVID-19 data and details

and next to it,

Download the raw data from the Ontario Data Catalogue

So you have two highly visible pathways to the data.  The data and details page provides tables, maps and charts, so you can work directly with the data on the web without needing specialised tools.  At the bottom of the data and details page, it again links to the underlying data, in a "Where the numbers come from" section.  You are left a bit to navigate the data catalogue to figure out which datasets generate which charts.  The page would benefit from adding a CSV download button with each visualisation.

For example, I had to try several different data catalogue pages and examine a number of datasets before finding the new cases dataset.

Nevertheless, this is overall a really strong open data success, with multiple really clear linkages to the open data.

If you land directly on the Ontario Data Catalogue, you would find out it has COVID-19 open data even without a search as they are surfacing several datasets both under What's New (which I'm guessing is manually curated) and in the lists of Most popular and Recently updated datasets.

I will note however that their custom 2019 Novel Coronavirus group includes many more datasets than you would find just by searching COVID-19.  There is a pathway from a dataset to the group, but you would have to notice that there is a Groups option at the top of every dataset.

 

Government of Canada

Government of Canada COVID-19 Open Data

The Government of Canada open data site is https://open.canada.ca/en/open-data

COVID-19 open data can be found just using "COVID-19" as a search.  In this case, I've filtered to include CSV, JSON, GeoJSON, XLS, XLSX, XML only:

https://search.open.canada.ca/en/od/?sort=score%20desc&page=1&search_text=covid-19%20OR%20coronavirus&od-search-col=Non-Spatial|Open%20Maps&od-search-format=CSV|JSON|GEOJSON|XLS|XLSX|XML

With these filters, there are 118 datasets.

If you look at e.g. Cumulative number of COVID-19 vaccine doses administered in Canada by jurisdiction

https://open.canada.ca/data/en/dataset/3b75a8d6-c5a9-48f9-834b-626eec16363f

you can see a clear description of the dataset and a link to download it in CSV format, but no tools to work with the dataset on the page itself.

Government of Canada COVID-19 website

The main general website is Government of Canada - Coronavirus disease (COVID-19).

A page about data can be found linked right in the top section

Current situation - COVID-19 data trends

Once we get there though, the situation is somewhat mixed.

The first link, Daily epidemiological report, takes you to many visualisations.  But it is really PDF bound, including a link to a PDF version of the page at the top, as well as (somewhat indirectly) linking to a weekly PDF summary (week of April 9, 2021 in this example).

There are some links to underlying open data, e.g. to CSV open data underlying "Current situation", but you have to be pretty sharp-eyed to spot them.

That being said, on the data trends page, there are two links in trend data that will take you to data:

• Downloadable data on COVID-19 across Canada takes you to the Government of Canada open data website, but it takes you to a single dataset, Public Health Infobase - Data on COVID-19 in Canada rather than a group or search or summary page listing all of the 100+ COVID-19 datasets.
• You can also find CSVs linked from some of the visualisations in Visual data gallery: COVID-19 graphs, charts, maps and more.

Overall I would say that while kudos is due to the Government of Canada for linking to open data, it could take lessons from the City of Ottawa and the Province of Ontario in more prominently and consistently surfacing the open data as part of its websites.

Similarly, if you land directly on the Government of Canada open data website

https://open.canada.ca/en/open-data

you get no indication that it has COVID-19 open data, unlike the prominent highlighting of COVID-19 open data on both the Ottawa and Ontario websites.

International

I will mention very briefly Our World in Data, which aggregates and then republishes international open data.  For COVID-19 specifically see

https://ourworldindata.org/coronavirus

There is a clear open license at the top of the web page, and there are visualisation tools (charts) provided.

Notably both the data and the code are made available on GitHub.

https://github.com/owid/covid-19-data/tree/master/public/data

Sidebar: Coding and Open Data

It is now much easier to work directly with open data in modern programming languages, particularly with CSV format data.  As an example, you can see my previous blog post about Juypyter Notebooks (Python coding) using Google Colaboratory, and a Jupyter Notebook that I created to chart new cases of COVID-19 in Ottawa is available on GitHub: https://github.com/scilib/ontario_cv19_opendata_plot

License for this blog post

This blog post is Copyright © 2021 Richard Akerman, and the text and associated images are licensed in the Creative Commons BY-NC 4.0 International.

This blog post is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.

This license applies only to this blog post and not to the entire blog as a whole.

Chan Zuckerberg Open Science

Information about Chan Zuckerberg Initiative (CZI) Science can be found at

https://chanzuckerberg.com/science

In particular, under "Our Approach to Supporting Projects", there is context about the CZI perspective on Open Science

https://chanzuckerberg.com/science#Approach

• Intellectual Property Should we build an IP portfolio, or should IP be generated by a CZI-supported investigator or research project, our policy is that the IP should be made freely available for academic and noncommercial use. Where IP is developed together with other organizations, we will promote non-exclusive licensing of technologies so that they have the broadest reach and impact.
• Data sharing and software We are deeply committed to developing and using platforms that disseminate data openly and freely. We are also committed to releasing software under maximally permissive open-source licenses, and developing software collaboratively in the open through sites like GitHub. We are committed to the long-term sustainability of our projects – when needed, we will explore alternative ways to ensure data access through permanent archives. Also when needed, we will work with regulatory groups to ensure that data requiring protection or controlled access are handled in a controlled and proper way in agreement with national and international standards.
• Publication We support full publication without conditions or restrictions on academic and publication freedom. We strongly encourage, and in some cases, may require, researchers to deposit manuscripts as preprints before peer review to increase access to research findings and to communicate results more quickly.

There is some additional text in the RFA for the Neurodegeneration Challenge Network, in the Collaboration and Open Science section

CZI supports open science values and principles. To accelerate scientific discovery and collaboration and rapid dissemination, CZI supports a consent, sharing, and publication policy for open and rapid dissemination of research results and a policy for software development that maximizes accessibility, reuse, and shared development.

You can also watch Dr. Cori  Bargmann, the President of Chan Zuckerberg Science, present a 2018 AAAS plenary entitled Accelerating Biomedical Research

http://players.brightcove.net/53038991001/Byx2STOH0_default/index.html?videoId=5736045412001

Previously:
April 25, 2017  Chan Zuckerberg - Meta and preprints

an open science diagram

As part of working in the open, here is a draft diagram intended to set the Government of Canada federal open by default pilot project in the context of potential open science activities.

diagram Copyright © 2017 Government of Canada, available for reuse under the Open Government License

This diagram is part of a larger briefing on how the federal open by default pilot project, which provides public access to draft government documents, could be used in an open science context.

Your thoughts about this representation and about draft documents in an open science context are appreciated.  You can leave a comment or email [email protected]

I take no credit for the diagram (I am terrible at graphics).  It is the work of the rest of the open science team at Environment and Climate Change Canada, with particular credit to Amanda McLeod.

The government does not currently have activities in all of the areas in the diagram above.  For more about the government's open science activities, please see the Open Science commitment in the 2016-2018 Plan to the Open Government Partnership.

Note: Crossposted to Medium https://medium.com/@scilib/an-open-science-diagram-3c3671478b7c

working in the open - public servants in Canada

While there have been public servants using social media including blogs and Twitter for a long time, there has been a recent upsurge in government employees deliberately choosing to use various online sites to do what I would call narrating their work.  This kind of open sharing of work in progress can be a great way to demystify what happens in government as well as to make new connections and get broader feedback.

Some recent examples from within government:

• Doing user research for the [Ontario] Digital Service Standard by Kate Kalcevich (Ontario Digital Service)
• Progress on the [Ontario] Digital Service Standard by Honey Dacanay (Ontario Digital Service)
• Hello Ontario! by Hillary Hartley (Ontario Digital Service)
• Responsible AI in the Government of Canada: a Sneak Peek by Michael Karlin (Government of Canada, guest post for Code for Canada)
• GC trip to UK — reading list day 1 by Olivia Neal (Government of Canada)

The above examples are all on Medium, a platform developed by Twitter co-founder Evan Williams.

You can also find public servants sharing their activities in other channels, for example CIO of Canada Alex Benay on Twitter and LinkedIn.

This increased public visibility of individual public servants and their work builds on years of conversations and experiments both "inside the walls" of government and on social media.  For example, there was a Canadian government event in 2010 called Collaborative Culture Camp that touched on many of the challenges of working collaboratively and openly.

I found that Twitter took a lot of my energy away from longer-form writing about my work.  Here's what I wrote about it in 2011

I think one loses a lot by not blogging. Twitter can to some extent maintain a presence online, but it can't expand it or make substantial impact. .... If you want to share your ideas in a way that will generate substantial discussion and spark interest in a major way, you have to write in the long form. ... to have an impact you must be writing your ideas, narrating your work. Not just for others, but as importantly, to better understand yourself, to have an online archive of your thoughts and work over time.

I am trying to return to doing more blogging and working in the open.

The above has a particular focus on the descriptive type of working in the open, there are other kinds of open work as well, for example open code on Github.

Kudos to Mary Beth Baker (Twitter @bethmaru) for her leadership role in getting the GC on Github.

As one example of open code, the website for the Canadian Digital Service digital.canada.ca is generated from Github, and is available for people to file issues or pull requests at https://github.com/gcdigital-gcnumerique/digital-canada-ca

For more on these topics, see my blog categories social networking and open source.

Note: Crossposted to Medium https://medium.com/@scilib/working-in-the-open-public-servants-in-canada-e103b4145dfd

Chan Zuckerberg - Meta and preprints

The Chan Zuckerberg initiative will be funding science in various areas, with a particular focus on diseases.

To accelerate this progress, we will bring scientists and engineers together in new ways, create computational and experimental tools to empower the scientific community and build a movement to support basic science research.

from https://chanzuckerberg.com/initiatives/

Chan Zuckerberg has launched with an initial US$3 billion commitment to basic research, as reported in Inside Philanthropy.

There is a particular emphasis on funding high-risk ideas:

Forty-seven investigators will receive up to US$1.5 million each in the next five years from the Chan Zuckerberg Biohub, a partnership between the couple’s Chan Zuckerberg Initiative and three universities: Stanford, the University of California, Berkeley, and the University of California, San Francisco. Together, the biohub grants announced on 8 February total more than $50 million.

“We told researchers, give us your riskiest ideas,” says biohub co-leader Stephen Quake, a bioengineer at Stanford.

from Nature - 'Riskiest ideas' win $50 million from Chan Zuckerberg Biohub - February 8, 2017

Several projects and technologies have been announced, including:

Chan Zuckerberg Biohub - this appears to be the central focus for the science activities

Meta - a science search engine that Chan Zuckerberg purchased and has promised to make available to the public for free.  It is planned it will become a sort of AI analysis tool for the science literature.

There is no open science policy statement available that I can find on any of the Chan Zuckerberg sites.  There is apparently a requirement to post preprints at time of submission.

The biohub investigators must post their manuscripts on open-access preprint servers, such as the arXiv, as soon as they submit the paper to a peer-reviewed journal. But researchers are permitted to file for patents, which would be owned jointly by the biohub and the scientists’ home institution.

from Nature - 'Riskiest ideas' win $50 million from Chan Zuckerberg Biohub - February 8, 2017

The Chan Zuckerberg Initiative Manager of Computational Biology, Jeremy Freeman, has made some statements about making data and code available.

Will all your research be peer-reviewed, published in journals and published for all to see?
We are absolutely committed to open dissemination of data and code and knowledge; that’s something I have been committed to all the time I’ve been a scientist.

The Guardian - Mark Zuckerberg’s man in the lab: ‘I want science to happen faster’ - March 5, 2017

Scientific research is still shared using a system developed more than a hundred years ago: the academic paper. “That’s how we disseminate knowledge, it’s a PDF document,” Freeman said. The data, the code and the interactive elements of the research should all be interactive, he argued. “There’s an obvious need to modernise and scale-up the technology we’re using,” Freeman said.

Wired - Meet the man tasked with speeding-up science at the Chan Zuckerberg Initiative - March 12, 2017

So for the moment it looks like preprints, open code (open source?) and open data.  But still not clear about open access.

A unified position statement from Chan Zuckerberg on their website about their open science approach would provide some clarity.  This could be a page along the lines of the Gates Foundation Open Access Policy.

UPDATE 2017-04-26: An announcement today from Cold Spring Harbor Laboratory (CSHL) that Chan Zuckerberg Initiative will be providing funding for the development and expansion of the bioRxiv preprints server and that "All software developed through this collaboration will be made available as open source."

Nature has a bit more information in BioRxiv preprint server gets cash boost from Chan Zuckerberg Initiative, including the possibility of better enabling bioRxiv for text mining and connecting it to Meta.

Previously:
February 17, 2017  Many preprint services

csvconf 2014 - The Content Mine - notes

csv,conf 2014 - July 15, 2014

14:00 The Content Mine by Peter Murray-Rust @petermurrayrust

Presentation: http://www.slideshare.net/petermurrayrust/csvconf

• "liberating scientific facts for humanity"
• crawl, then data mine the literature, publish resulting facts CC0
• uses technology to extract information from PDFs, "can now turn PDFs into science"
• information will go to Wikipedia and as linked open data to DBpedia
• site is contentmine.org

Peter is using the UK Exceptions to Copyright for Research (PDF), specifically the text and data mining exception.

Government Open Source: Web Experience Toolkit

The Web Experience Toolkit has been in the news lately, with mentions in Mashable and Fast Company due to appearing on GitHub and Government.

It also got positive mentions from David Eaves and Ben Balter at the Code for America Summit.

Below is a Storify of recent mentions:

Web Experience Toolkit - CodeFest 2013

A modern technology for web components, the Web Experience Toolkit, had its second CodeFest.

The open source code and community are hosted on GitHub.

https://github.com/wet-boew/

Government of Canada employees have played a substantial leadership role in the creation of the toolkit, in its release on GitHub, and in growing the community.  It's a great example of government working in a modern way.

I've embedded links to the keynotes below, as well as a Storify of the event (I didn't attend, but I watched the keynotes on YouTube).

Event info at http://wet-boew.github.io/codefest2.0/

Keynote presentations

Ben Balter @BenBalter - http://ben.balter.com/open-sourcing-government/
https://github.com/benbalter/open-sourcing-government (repository for slides) - video from 14m43s http://youtu.be/E-AxzfaZQVc?t=14m43s to 1h13m

Raymond Camden @cfjedimaster -
https://github.com/cfjedimaster/CuttingEdgeHTML (repository for the slides) - video from 1h35m32s http://youtu.be/E-AxzfaZQVc?t=1h35m32s

Hillary Little @hilittle - http://www.hilarylittle.com/blog/2013/08/09/codefest-2013-keynote-2/ - video http://youtu.be/51hN1trnpXA

Previously:
January 11, 2013  open source in government - The Web Experience Toolkit

Related articles

The Software That Builds Software

G8 Open Data Charter, updated Canadian national open data portal

Prime Minister Stephen Harper today formally adopted an Open Data Charter with other G-8 Leaders at the Lough Erne Summit in Northern Ireland.  In keeping with the Charter, the Prime Minister announced the upcoming launch of a new Government Open Data Portal, data.gc.ca, which will provide Canadians with unprecedented access to government data and information.  The next-generation Open Data Portal will be officially launched on June 18, 2013, by Treasury Board President Tony Clement.

press release - PM announces new measures to make Government data more accessible to Canadians

Open Data Charter

Each principle contains high-level "we will" commitments.

Principle 1: Open Data by Default

Principle 2: Quality and Quantity

Principle 3: Usable by All

Principle 4: Releasing Data for Improved Governance

Principle 5: Releasing Data for Innovation

https://www.gov.uk/government/publications/open-data-charter

UPDATE: Down in the technical annex there are some very specific commitments, including metadata mapping on GitHub.

Action 1: G8 National Action Plans

• We will publish individual action plans detailing how we will implement the Open Data Charter according to our national frameworks (October 2013)
• We will report progress on an annual basis (via the G8 Accountability Working Group) (2014 and 2015)

Action 2: Release of high value data

• As a first step, we will collectively make key datasets on National Statistics, National Maps, National Elections and National Budgets available and discoverable (from June 2013), and we will work towards improving their granularity and accessibility (by December 2013)
• We recognise that collective action by all G8 members has the potential to unlock barriers and foster innovative solutions to some of the challenges we are facing. We therefore agree on a mutual effort to increase the supply of open government data available on key functions of our States, such as democracy and environment. We will work on identifying datasets in these areas by December 2013, with an aim to release them by December 2014.
• We will set out in our national action plans how and when we will release data under the remaining categories according to our national frameworks (October 2013).

Action 3: Metadata mapping

• We have contributed to and commit to maintaining the G8 metadata mapping exercise (June 2013)
• This mapping can be viewed on Github and comprises a collective mapping ‘index’ across G8 member’s metadata, and a detailed page on each G8 member use of metadata within their national portal.

ENDUPDATE

2013 Lough Erne G8 Leaders' Communiqué

UPDATE 2013-06-19: The communiqué contains additional commitments in the Open Data section, including

48. This Open Data Charter will increase the supply of open government data across a number of key categories including health, environment and transport; support democratic processes; and ensure that all data supplied are easy to use. We encourage others to adopt this Charter. G8 members will, by the end of this year, develop action plans, with a view to implementation of the Charter and technical annex by the end of 2015 at the latest. We will review progress at our next meeting in 2014.

49. In keeping with the Open Data Charter principles, transparent data on G8 development assistance are also essential for accountability. We have all agreed to implement the Busan Common Standard on Aid Transparency, including both the Creditor Reporting System of the OECD Development Assistance Committee and the International Aid Transparency Initiative (IATI), by 2015. To show greater G8 leadership we will ensure data on G8 development assistance is open, timely, comprehensive and comparable.

50. G8 members should over time apply the Busan common transparency standards to their respective Development Finance Institutions and international public climate finance flows consistent with the reporting of climate finance under the UN Framework Convention on Climate Change (UNFCCC).

ENDUPDATE

Updated National Open Data Portal

Updated http://data.gc.ca/ site, now running CKAN data engine with Drupal web front-end.

Includes:

• Updated Open Government License
• subsite for developers
• blog

There's an article in the Globe and Mail: Ottawa to grant app developers access to federal information.

Related articles

Why we should welcome a G8 Open Data Charter

open service development and the Expenditure Database

On April 22, 2013 TBS released the Expenditure Database.  It allows one to browse and search expenditures organised in ways that are easier to understand than the "votes" system under which they are actually allocated.  It is great progress within the overall open government initiative.

However, from a technology, design and process perspective, I think there are some opportunities that are being missed.

Victorian Service Delivery

Minister Clement is fond of saying that the government's data is like grandmother's silver, hidden away.  In fact, our current model of delivering applications is more like hiding the entire kitchen from view.  In Ottawa's Victorian homes, the kitchen is usually at the back of the house, closed off from the dining room.  The servants were supposed to work there in obscurity, with only the final result appearing with a dramatic flourish in the formal dining room. 

In most of those homes that wall has now been smashed down, because we found that in the post-servant age, the kitchen is the hub of activity in the house, where we socialise and cook together openly, a very human and social activity.

But in government software development, that wall still stands.  Applications are developed behind closed doors by public servants, and then suddenly appear, fully formed, on release day. This model of closed software development has some very real consequences:

• There is no opportunity for issues to be caught and corrected during the development process; instead we have to wait post-launch to find design issues that are only visible after public use
• It makes application development seem mysterious and closed off from the world.  In a country where we want young people--in particular young women--to see software development as an interesting career, we've hidden all the real human aspects of development away, all the interactions, discussions, decisions, and compromises that have to be made as we assemble the software from the available ingredients in our coding kitchen.
• It closes the government developer community off from the much larger national and worldwide community of developers, both reducing the network that they can tap into as well as hiding our own government excellence and expertise.

We need not just open data, we need the entire philosophy of open source web development: we need to develop in the open.

Modern Government Service Design

The UK Government has released a fantastic Government Service Design Manual.  It has some explicit statements and some built-in assumptions.  The underlying concepts are to develop government services using modern software engineering processes.  This means being open about the code as it is in development (e.g. through blogs explaining the work and github repositories making the code available) and iterating through the design, from Discovery, through Alpha, Beta, Live and (all important in the government) Retirement.

The UK Government Digital Service does its work in the open, through blogs, twitter, and github.

Software development is a process: the application you release at a moment in time is not an end, what's important is that the application should tell a story about itself, so that it can be improved and so that it can be an inspiration for further work.

The Expenditure Database

The Expenditure Database is a great step in providing easy citizen access to the underlying data.  But it doesn't tell any of the story of how it works, how the data is processed, and who did the work.  The meal has been delivered and it looks very nice, but the kitchen and the process of making it are still closed off from view, hidden in mystery.

Code on the Web

Of course this is not quite as simple a story because it is a web application, so in theory you could look through all of the JavaScript.  But that means reverse-engineering the code, trying to figure out how it works without having access to documentation (the recipe) and the developers (the cooks).  When someone is trying to understand an application the first thing they will do is read its story about how it works and how it was put together, and then ask the developers when there are aspects not clear or not covered in the documentation.  Without open code development, this avenue of understanding is closed.

Instead one has to dig through the code and make guesses.  For example, it looks upon examination that it is drawing the data from a local file: http://www.tbs-sct.gc.ca/ems-sgd/edb-bdd/data.js

If instead the code were in a shared repository and documented, it would be much easier to understand how it works and the rights to reuse and modify would be a lot clearer.  As well one would expect there would be both examples of using the data file as well as an explanation of how it was generated.

Expenditure Database Development

What Expenditure Database development would look like if it followed this model:

• In advance of coding, in the Discovery phase there would be an internal call for requirements and expertise, e.g. on GCpedia (the Government of Canada internal wiki)
• An alpha using open source code would be quickly released
• The alpha would use public budget open data from data.gc.ca
• If the data wasn't available, it would have to be released (the rule is, internal developers shouldn't have any special or privileged access to information)
• A beta would be released to the public incorporating feedback and code changes
• At the beta phase at the latest, code would be shared on github (a public code repository and developer community) and there would be public explanations and documentation of the work going on.
• The developers would be using their own accounts; it would be clear whom to contact for questions about the software.
• Once the system goes live, there would be an ongoing process to incorporate improvements to the code, improve the documentation, and respond to internal and external questions from other developers and users.

What We're Missing

Just a few examples of what we're missing as a consequence of the current implementation:

• There are no permanent URLs to departments or searches.  That means if I want to point you at the expenditures for the National Research Council, I can't just give you a URL.  I have to say "go to http://www.tbs-sct.gc.ca/ems-sgd/edb-bdd/index-eng.html and search or scroll to National Research Council and then click on it and then scroll down and click on the details for various parts of it".  A good standards-based design and testing process should have revealed this issue.
• The Expenditure Database "stands alone" on a difficult-to-remember GC standard URL.  You have to know to navigate to it, or get the link from somewhere.  Why not tbs-sct.gc.ca/expenditures as a short link?  There is also no link from the database to data.gc.ca nor is there a link from data.gc.ca to the database.
• Without documentation about how the data were created, it's not clear if it's using budget data from data.gc.ca or whether it is using a custom database, and how the numbers are being generated; without access to the code there is a lost opportunity for other developers to learn how to parse, manage and display the budget data.
• Without the code, it's not possible for developers to either make their own websites using a similar approach, or to improve the Expenditure Database code itself.
• Without pulling the data directly from data.gc.ca there is no opportunity to explore all of the challenges that external developers need to solve, such as how to deal with versioning (what if a number has to be changed in the released data? how to tell if it has been changed?  how to see the previous version?) and performance (should there be an API instead of a download?  should the download be cached locally?  for how long?) and updates (how to tell when a new budget year has been released?)
• Without the code shared and the work publically described, people are left to speculate as to how the results are generated.
• Without the code shared and the work publically described, there is no entry point for external developers; there is no community-building for external developers.
• There's no contact point. The site gives no idea who developed the Expenditure Database, how it works, or who made the design decisions. The code and its development is made mysterious, the work of the kitchen hidden, rather than being demystified.
• There's no way to download the data displayed; you can only take a screenshot.
• There is no standard web navigation (e.g. the back button doesn't work)
• There's no way to compare departments

Additional information

TBS PR - Searchable Information on Government Spending is now Just a Click Away

Call for Feedback

I welcome your feedback on this post, with any clarifications, corrections, suggestions or pointers.

Thanks to Stephen Anthony, Adam Frankowski, Patrice Collin and Nick Charney for providing ideas and feedback that improved this post.

open source in government - The Web Experience Toolkit

The Government of Canada has its Web Experience Toolkit on GitHub

https://github.com/wet-boew

By making the WxT open source, it has been possible for organisations outside of the government to use it for their websites and to improve it.  Both the University of Ottawa and the City of Ottawa are now using WxT.

There was a writeup in Wired about the WxT: Canadian [Coders] Solve Mystery of Open Source Government.

the Treasury Board of Canada hosted a CodeFest to invite hackers — mostly government staffers — to hack its Web Experience Toolkit, or WET — a set of open-source tools that the Treasury Board uses for building websites. One hundred and fifty people came. Many of them were young developers, excitedly swapping code and sharing ideas across tables.

There are some minor issues: It's Treasury Board Secretariat, not Treasury Board, and TBS is the government's central policy agency not an "obscure Canadian tax-collecting agency".  (Admittedly it is difficult to decode that from the TBS website, but if you read far down enough you get to "policies, directives, regulations, and program expenditure proposals with respect to the management of the government's resources".)

You can follow Web Experience Toolkit info on Twitter using the hashtag #wxt and the account @WebExpToolkit

Author of the article is Robert McMillan @bobmcmillan

Featured or indirectly mentioned are:

• Paul Jackson @pjackson28
• Mary Beth Baker @bethmaru
• Lucia Harper @Lucialand
• Laura Booker (of OpenPlus) @bookerlaura
• @OpenPlusSoft
• Mike Gifford (of OpenConcept) @mgifford
• Luke Charde @lukecharde

UPDATE: There is a Drupal working group, the next meeting is January 25, 2013 at Ottawa City Hall.

Rebooting the public square with open data and open source

there are myriad categories of raw data that are locked up inside the boundaries of government.  If governments were simply to publish this data, then private companies, civil society organisations, citizens and other government agencies will self-organise to use this data to create public value.  This holds the promise of better, cheaper government.

Don Tapscott - 03m43s to 4m06s

The fifth and final episode of CBC's ReCivilization was "Rebooting the Public Square".  Ignoring the apocalyptic "this is the end of everything we know" tone at the beginning, it looks at the infrastructure of open sharing online (through discussions with Google and Mozilla), open data (through a discussion with Sir Tim Berners-Lee), and the kinds of activities that are enabled in the government sphere (through a discussion with the White House Office of Science and Technology Policy).

I think it's an ok starting point for understanding some of the pieces, although there is a lot more to open government than is covered in the segment.  I was disappointed that as a Canadian show there was no mention of Canadian open government initiatives, which include

• national - open.gc.ca and participation in the Open Government Partnership
• provincial - data.gov.bc.ca
• municipal - ottawa.ca/opendata

(More sites are listed on Wikipedia - Open Data in Canada.)

The audio is available as broadcast on the web, there are also extended interviews available.  For those interested specifically in open government, the extended interview with Thomas Kalil is the most informative.  The series is also available (for free) in iTunes.

Twitter:

• @recivilization
• @dtapscott
• @adw_tweets
• @WhiteHouseOSTP
• @ericschmidt
• @MitchellBaker
• @timberners_lee

Previously:
December 30, 2006  Tapscott: turn your organization inside-out
December 28, 2006  Tapscott on collaboration and Wikinomics in the Globe
June 6, 2005  SLA2005 - live, semi-awake blogging of opening session - Monday 9:30 AM (featuring Tapscott keynote)

UK Open Government and Transparency

Francis Maude is UK Minister for the Cabinet Office.  He spoke about the UK experience in January 2012 for the World Bank.

Governments are finding transparency risky, difficult and uncomfortable. But transparency sticks – it’s irreversible once you start. And I believe transparency will become the defining characteristic of future public policy.

...

putting out raw data in the public domain for entrepreneurs and businesses to work with. Creating an information marketplace. And this is where I believe the UK is leading the way today.

Data sharing is underpinning everything we do to improve public services and to drive new waves of growth.

...

Companies large and small are also using the data to create innovative, products and applications.

Already we’re aware of 47 independent app developers working in the UK giving information to rail passengers through their smartphones – in a market that has for the most part open up in only a few major cities.

London commuters can now use their phone apps to decide whether to rush for the train or get a cup of coffee thanks to greater transport data.

Francis Maude's speech to the World Bank – Transparency and Open Data

You can also watch the webcast recording (Windows Media format).

In addition to the UK open data site data.gov.uk there are two other initiatives of note: gov.uk from the Government Digital Service and G-Cloud.

gov.uk is a project to have a single homepage to access UK government services.  They have been using an agile methodology, and the gov.uk site is now in beta, following an alpha.   From a homepage that is a blog to an official Twitter feed at @govuk that retweets team members, it is a great example of innovation at the heart of government.

G-Cloud includes government use of the cloud as well as data centre consolidation.  They just launched the CloudStore, a list of approved services.  It includes the notion of trying to open up government procurement to a wider variety of players.  They announced CloudStore Open for Business on February 19, 2012 in their blog.  The CloudStore, which runs on MS Azure, will be available as open source.  You can follow them at @G_Cloud_UK

RT @cantwaitogo The full UK Gov #cloudstore will be released as #opensource. < great, thx #oss #govit #gov20

— Andrew Stott (@DirDigEng) February 19, 2012

the Open City - the next driver for innovation?

Mayor Gregor Robertson and Coun. Andrea Reimer want the City of Vancouver to support open-source software and open standards.

They also want the city to make as much data as possible freely available to the public. Reimer will introduce a motion [PDF] next Tuesday (May 19) that would see the city endorse the principles of open source, open standards, and open data, as well as start work on publishing data on the Web using open standards.

In a press release issued today (May 14), Robertson said that an “open city” philosophy would help create new opportunities in the information-technology sector.

City of Vancouver set to back open source, open standards, open data - straight.com - May 14, 2009

via Twitter - Rob Giggey (Rob works at the City of Ottawa) - May 15, 2009

(I tried to find the Gregor Robertson press release referenced above, but I haven't been successful - can anyone point me to it?)

Toronto Mayor Miller has also announced toronto.ca/open (which still shows "under construction").

In Ottawa, supported by some City of Ottawa staff but not (yet?) endorsed as any kind of official policy, we're starting the open data discussion as part of ChangeCamp Ottawa.

What can you do as citizens and what can we do as libraries to enable the sharing of our civic data?  Is sharing civic data a next logical step for public libraries as enablers of the public space?

Previously:
May 5, 2009  Web APIs explained on CBC Spark - and Open Data Under Construction

next generation open library system

I'm so used to bookmarking stuff into my FriendFeed these days I have to remind myself my blog is also a communications vehicle... I'd be remiss in not posting about this major SOA & libraries initiative.

The Open Library Environment (OLE) Project

With support from The Andrew W. Mellon Foundation, the Open Library Environment (OLE) Project will convene the academic library community in the design of an Open Library Management System built on Service Oriented Architecture.

http://oleproject.org/

Duke University is the lead on this project.

UPDATE: Just for clarity I should probably mention that my organisation (CISTI) is neither a core nor an advisory partner.  ENDUPDATE

If you want to get an idea of the framework that this fits into, you can see my posting about the presentation Enterprise Services Architectures - Chris Mackie - DLF Fall Forum 2007.

It is great to see library projects starting to use a service-oriented approach, although given that this is the dominant industry trend these days, I'm not convinced that my four years of online and offline advocacy for library SOA actually had much of an impact on this choice.

(You can see my blog postings in category "Service-Oriented Architecture" going back to November 2004:
http://scilib.typepad.com/science_library_pad/serviceoriented_architecture/ )

Still, one does like to be right.

NLA announces Library Labs

The [National Library of Australia] has recently opened this "Library Labs" wiki space:

https://wiki.nla.gov.au/display/LABS/Home

The aim of this space is to let our colleagues know what we are doing, to invite comments, questions and feedback and to provide a space for discussion and collaboration.

We have started to redevelop our digital library services using a service-oriented architecture and open source software solutions where these are functional and robust.  We are also aiming to take a common ("single business") approach to collection management, discovery and delivery.

We are interested in forming a community of Australian business analysts and developers who are working on similar problems and who are interested in  interoperable, standards-based solutions. We are also interested in working with colleagues at an international level to provide prototypes and testbeds for new and emerging standards.

via Warwick Cathro
Assistant Director-General, Innovation
National Library of Australia

software development, staffing and new library technology

Librarian in Black responded to my article in Library Journal netConnect, and Richard Wallis' commentary thereon.  She says

What makes me sad is that both Ackerman [sic] and Wallis have missed a key point: if the future is in web services, how can libraries take advantage of that with their current staff configurations?  How many libraries in the U.S. have a honest-to-goodness computer programmer on staff?  How many have staff with Computer Science degrees?  How many staff do they have devoted to the library's hardware, software, and network?  How many staff do they have devoted to web services?

In the smallest libraries, perhaps all of these are the same one person.

My article is about modern software engineering for libraries, not on how to staff them.  However, in the very same issue of netConnect there's an article by Karen Coombs "Digital Promise and Peril" calling for library staffing to reflect the digital content environment

Many of these digital materials are in jeopardy of being lost because librarians have not yet found adequate ways to collect and manage them. In part, this is because roles and skill sets have been siloed in libraries. Materials preservation issues have typically been the purview of special collections and archives units within the library. In contrast, cataloging expertise has resided in technical services, and technology expertise has typically resided in systems. To collect and manage born digital objects adequately requires these roles and skill sets to come together.

So let me summarize some of the goals and targets of the article, as well as talk about the relationship to promising developments:

• The main focus of the article is to convey to library administrators, managers and planners that the world of networked digital content requires new ways of thinking about developing library systems, and that there are modern software engineering methods and technologies that can support new systems development.
• It may be the case that, as in Sarah's words, the library blogosphere knows that "Of course the future is in web services" but it took years of hard lobbying and education within my organisation to convince my own library leadership - that's why I was happy to have the forum of Library Journal, to reach a wider audience.  I have to believe there are many library managers who have no idea where the technology future lies, and have only a vague notion of web services.
• If I can get one library manager to be able to ask good questions about web services and service-oriented architecture, and even (dare I dream) read one of the books I recommended with more detailed information on the topic, my goal is achieved.
• I recognize that many public libraries don't have the staff for development, and never will.  That's why I have said in the past that librarians should be scripters, not coders.  Most libraries should be using technology developed externally, not trying to do their own custom internal development, or as I said in my article, they should be technology service consumers, not necessarily producers. There are only a few libraries in the world (including mine) which have the size to have a substantial development staff.
• I work at a research library, not a public library nor an academic library.  Research libraries like mine now interact with patrons almost entirely online - our walkin traffic is basically zero.  That means we have to move into the networked information space very aggressively, or basically disappear as a presence for our users, replaced by publisher web sites.
• By moving to standard APIs, using standard modern development methods, and standardizing web services, libraries can take advantage of a much broader development community, and potential staffing pool.  How many software developers do you think know Z39.50, MARC, and SRU/SRW?  That's why we have a tiny community of library hackers trying to make things work.  Now imagine that instead your library software job poster just says "developer needs to work with standard Java tools to develop software using modern methods for standardized APIs".  Getting access to a better, broader staff base is intimately connected to moving library technology into the mainstream of software development.

Where can we look?

Maybe the DLF project on ILS APIs will help.

Maybe the OASIS effort on standardising search services will be useful.

Maybe it happens by using OpenSearch and simple REST interfaces rather than custom library protocols.

UPDATE: There is another important piece, which is about libraries reaching out and speaking the right language.  That's why you need to understand how to express things in terms of SOA, Web Services, and APIs.  There is way more innovation capacity outside your walls than you can ever get inside, even if you have the perfect IT staffing policy and budget.  From your local superpatrons, highschool CS students, and local college and university computer science departments, to, basically, every CS student in the entire world.  You can reach them with contests, with collaboration requests, with invitations to improve your systems... but here's the important part... if you speak their language.  CS people love challenges and programming, but they're not going to learn obscure library jargon and usage like OPAC, Z39.50 and database (which means something completely different in CS).  You can't say "hey, can you help us improve our OPAC because the Z39.50 doesn't federate across our databases".  They're not going to know what the f*** you're asking.  Learn the CS language, and a whole world of programmers will open up to you.

To me one of the single biggest missed opportunities is in the digital library community.  Ever year, lots of computer science groups, flush with energetic grad students, toil away and produce results that are presented at JCDL and ECDL.  And, based on my experience at ECDL 2006, they then present those results entirely to a community of other computer scientists.  Where are the librarians?  Why are you all at library conferences talking to other librarians?  Come to *CDL and ask the computer scientists to build stuff you need.  Yes, it's a difficult transition from research to production, but at least join the conversation.  ENDUPDATE

I don't have the answers and I have certainly asked again and again where people see all these frameworks and groups fitting together, with no response.

The good news is that there are lots of projects out there already - I don't think it's a case that there is no activity.  The fundamental point of my article is that for these projects, we have to use enterprise architecture, service-oriented architecture, web services / standard APIs and the whole toolkit of modern network-based standard-data software development.  Because if we don't, WE WILL BUILD SILO SYSTEMS AGAIN.

That's what I said in my IATUL presentation, and in the short accompanying paper (Scribd), and what I've been saying over and over again in this blog.

[Sidebar on Scribd: be careful browsing around this document hosting site.  Many of the profiles, profile images, and documents are unfortunately very not safe for work.  Scribd really needs to put in a moderation / adult content filtering system.]

Why do I think it's important to talk about these topics?  Because there really are lots of new developments in the library catalogue and OPAC world, including:

• Scriblio - "free, open source CMS and OPAC with faceted searching and browsing features based on WordPress"
• VuFind - "The goal of VuFind is to enable your users to search and browse through all of your library's resources by replacing the traditional OPAC"
• Evergreen Open ILS - including British Columbia Pines project - "The phased implementation of the Evergreen Open ILS for all of BC ("BC PINES") be implemented over the next 5 years. We hope that eventually all public libraries in BC will join"
• eXtensible Catalog (XC) - "an open-source online system that will unify access to traditional and digital library resources"
• CERN systems redevelopment - build a complete high-energy physics (HEP) information system with full-text, data-mining and demonstrate and deploy Web 2.0 applications in the domain of sciences

There are also some great modular browser tools out there, including

• LibX - a Firefox extension that provides direct access to your library's resources [and] an open source framework from which editions for specific libraries can be built.
• Zotero - a free, easy-to-use Firefox extension to help you collect, manage, and cite your research sources

I'm very much hoping that these developments will open libraries up to be better network participants, with a broader community of developers able to build pieces, and with standards enabling libraries with limited development capability to simply plug-and-play.

Some links via The Ten Thousand Year Blog: eXtensible Catalog open source project, VuFind released as open source software.

Previously:
June 29, 2007  Casey Bisson on Scriblio and OpenLibrary

the future of the scientific paper and more on open web science

Timo Hannay has an interesting presentation The Scientific Paper of the Future (9.2 MB PDF), from the Microsoft eScience Workshop at JHU.

The idea of the journal article as a window or an interface to data is a very interesting one, I'm reminded of Leigh Dodds' presentation on The Scientific Paper as Palimpsest.

Via Nascent, the Berkman Luncheon Series, and MediaBerkman blog I also find pointers to PowerPoint (23 MB - unfortunately many images don't show up in Windows PowerPoint 2000), audio (MP3) and video of Timo giving a presentation on What the Web Means for Science.

This in turn leads me to a presentation by Dan Burk on Open Source Strategies for Science.

Peter Murray-Rust on open science

It was his posting Is Openness “ethically flawed”? - September 2nd, 2006 that got me on my recent track of thinking about open science, he has followed that up with several more meaty chunks of open pondering:

• Open Data, Open Science. Closed Data… - September 7, 2006
• Open Source, Open Data and the science commons - September 7, 2006

Thanks to Open Access News for reminding me to revisit Rust's blog.

the concepts of open science and information commons

First, a challenge for you librarians out there: it would be nice if someone tracked down the origins and history of the "open science" concept.  Who coined it, with its current meaning?  What papers and conferences have discussed open science?  Let the referencing begin!

Is it meaningful to have distinctions between e-Science, Open Science, and Science Commons?

I, being a non-librarian, have gathered some references using that library arch-nemesis, Google.

John Willinsky, First Monday, Volume 10, Number 8 — 1 August 2005, The unacknowledged convergence of open source, open access, and open science

CODATA - Creating the Information Commons for e-Science: Toward Institutional Policies and Guidelines for Action, 1-2 September 2005

International Seminar - Open Access for Developing Countries, September 21-22, 2005
- presentation by Paul Uhlir Creating a global information commons for public science (PDF)

ICSTI - Information and Data in e-Science: Making Seamless Access a Reality, February 3, 2006

mitchell's blog, August 30, 2006, Open Source, Open Science

At Sci Foo Brian Behlendorf and I hosted a session about how the lessons learned from the open source software experience might be applicable to scientific endeavors. The hope is that we can support the “open” movement in science as well. By ‘open” I mean a system where effort and resources are pooled and the result shared. This is in contrast to an increasing focus on what’s “my intellectual property, how can I best protect that intellectual property, use it to create a closed system and then extract the most value for me from that closed system.”

CODATA - Creating the Global Information Commons for Science

Global Information Commons for Science is a multi-stakeholder initiative arising from the second phase of the World Summit on the Information Society in Tunis in November 2005.

Commons of Science Conference - Creating a Vision for Making Scientific Data Available Across Disciplines, October 3-4, 2006.  Washington, DC.  By invitation only.

UNESCO - Consultation meeting on WSIS Action Line C7 ''E-science'', 22 October 2006, Huarun Hotel, Beijing, China

If this is all a bit too heavy, you can enjoy the Flickred photos from iSummit 2006 (the one in Rio, not the one in Toronto) by John Wilbanks.

Some other starting points for further reading are the (very slow) Science Commons Blog (feed at http://sciencecommons.org/weblog/rss ), and the OnTheCommons blog entry A Renaissance of the Science Commons.

So are there four elements to open science, viz. open source + access + data + discourse?

Any other important articles or events?

Previously:
September 04, 2006  open discourse + access + data equals open science?
April 28, 2006  conference proceedings: ICSTI eScience, Bielefeld + Taiga + LtF on academic library future

Word 2007 and the bibliographic processing future

There is an opportunity for the library and scholarly community to work with the various word processor builders to get to some common standards.

The main players that I'm aware of being Word 2007 and OpenOffice.

Jennifer Michelstein of Microsoft has the (mis)fortune of starting her MS blogging career with the posting [Word 2007] Academic features: citation & bibliography tools.

There's some nice stuff, but I have two concerns:

1. The focus is on building your Master List inside of Word.  Yes, you can share lists, but I don't want Word to be the citation master at all

Once a source is created, it lives in two places: your Master List and your Current List. The Master List is the database of all sources ever created. The Current List includes all of the sources that will be used in the current document.

The purpose of the Master List is to save you from re-typing and re-entering information about sources that you commonly use. For example, if you are a Shakespeare scholar and always cite the same five Shakespearean references, you can just select these sources in your Master List and click Copy to add them to your Current List. Now you can cite them throughout your document.

Maybe Connotea is my "master list", or EndNote, or Bookends.  You know, separation of concerns?  Let my bibliographic software be the master of citation management.  Make Word the master of communicating with all major citation repositories, whether local on my machine, or out on the web.

2. They will talk to libraries... somehow.  With a software development kit (SDK) that doesn't exist yet.  Um, how about we get that kit, immediately?

we are building a platform on the Research and Reference pane, to enable connecting to a library database and importing metadata about sources. We’ll publish an SDK so that Microsoft or any 3rd party data provider can build a service that fits nicely into our Bibliography tools. This project is underway, but the functionality isn’t available externally in Beta2. This sounds like a topic better handled in its own post.

darcusblog (Bruce D'Arcus, of the OpenOffice Bibliographic Project) has written extensively (and critically) about the current state of the art in citation management in Word 2007

via Dan on eScience

a couple upcoming events - repositories and code

*** 1 Open Repositories 2007 - 2nd International Conference on Open Repositories (ICOR2007)
January 23-26, 2007
San Antonio, Texas

CFP deadline:

October 2, 2006     Extended abstract, less than 500 words, double spaced

via Disruptive Library Technology Jester

*** 2 code4lib 2007
February 28 - March 2, 2007
Athens, Georgia

Bioclipse - open-source visual platform for chem and bio

The Bioclipse project is aimed at creating a Java-based, open source, visual platform for chemo- and bioinformatics based on the Eclipse Rich Client Platform (RCP).

There is a wiki and code on SourceForge.
They plan to release version 1.0 on May 19.

Info via the interesting chem-bla-ics blog.

LogicBlaze: open source SOA

Note: Open source doesn't mean free.

InformationWeek has an article SOA Goes Open Source

LogicBlaze has launched an integrated suite of open source code that's designed to get businesses started with service-oriented architecture. Called Fuse, it includes several pieces of open source code from the Apache Software Foundation, plus three from the Apache incubator. The claim that all the pieces have matured must take into consideration that incubator projects at Apache aren't yet full-fledged, open source projects; rather, they're getting an organization and community established around a core code donation.

elemental links has a good overview of the components in Fuse.

SOA for Canadian Higher Education

here is the link to the outputs of the SOA workshop in
Vancouver last month.  Go to

http://educationcommons.org/projects/display/SOA/Home

Click on the Workshop March 2006 link to see all the workshop material.    Look for the
16 (or so) "service candidates" we identified as emerging out of the
high-level business process of "apply for admission".  We demonstrated the
power of service oriented analysis and the existence of abundant reusable
services at or close to the infrastructure level (though Thomas Erl calls
these "application services").  The MIT/OKI [Open Knowledge Initiative] crowd and UK representative [University of Hull - e-Services Integration] were particularly helpful in working this through.

We are starting to piece together some of the next steps and have identified
the following "tracks" going forward:

Entity definition
Business process discovery
Application service candidate discovery
Experimental implementation exercises (Proof of Concept)
Data representation architecture and schema definition
Higher education "basic profile" definition
Security and privacy requirements definition
Technology alternatives selection -- should be deferred to Service-Oriented
Analysis stage

By the time we reach CANHEIT, we should have a penultimate version of the
report (feasibility study) funded by the Mellon Foundation.  Be sure to
attend the presentation by Richard Spencer and Leo Fernig [A Community Source Student Services System].

Ted Dodds
Associate Vice President, Information Technology & CIO
The University of British Columbia

http://www.e-strategy.ubc.ca/

via Bill St. Arnaud - CAnet News - SOA in Higher Education

Previously:
January 05, 2005  Hull.ac.uk portal built using SOA