Prime Minister Stephen Harper today formally adopted an Open Data Charter with other G-8 Leaders at the Lough Erne Summit in Northern Ireland. In keeping with the Charter, the Prime Minister announced the upcoming launch of a new Government Open Data Portal, data.gc.ca, which will provide Canadians with unprecedented access to government data and information. The next-generation Open Data Portal will be officially launched on June 18, 2013, by Treasury Board President Tony Clement.
Each principle contains high-level "we will" commitments.
Principle 1: Open Data by Default
Principle 2: Quality and Quantity
Principle 3: Usable by All
Principle 4: Releasing Data for Improved Governance
Principle 5: Releasing Data for Innovation
UPDATE: Down in the technical annex there are some very specific commitments, including metadata mapping on GitHub.
UPDATE 2013-06-19: The communiqué contains additional commitments in the Open Data section, including
48. This Open Data Charter will increase the supply of open government data across a number of key categories including health, environment and transport; support democratic processes; and ensure that all data supplied are easy to use. We encourage others to adopt this Charter. G8 members will, by the end of this year, develop action plans, with a view to implementation of the Charter and technical annex by the end of 2015 at the latest. We will review progress at our next meeting in 2014.
49. In keeping with the Open Data Charter principles, transparent data on G8 development assistance are also essential for accountability. We have all agreed to implement the Busan Common Standard on Aid Transparency, including both the Creditor Reporting System of the OECD Development Assistance Committee and the International Aid Transparency Initiative (IATI), by 2015. To show greater G8 leadership we will ensure data on G8 development assistance is open, timely, comprehensive and comparable.
50. G8 members should over time apply the Busan common transparency standards to their respective Development Finance Institutions and international public climate finance flows consistent with the reporting of climate finance under the UN Framework Convention on Climate Change (UNFCCC).ENDUPDATE
There's an article in the Globe and Mail: Ottawa to grant app developers access to federal information.
On April 22, 2013 TBS released the Expenditure Database. It allows one to browse and search expenditures organised in ways that are easier to understand than the "votes" system under which they are actually allocated. It is great progress within the overall open government initiative.
However, from a technology, design and process perspective, I think there are some opportunities that are being missed.
Minister Clement is fond of saying that the government's data is like grandmother's silver, hidden away. In fact, our current model of delivering applications is more like hiding the entire kitchen from view. In Ottawa's Victorian homes, the kitchen is usually at the back of the house, closed off from the dining room. The servants were supposed to work there in obscurity, with only the final result appearing with a dramatic flourish in the formal dining room.
In most of those homes that wall has now been smashed down, because we found that in the post-servant age, the kitchen is the hub of activity in the house, where we socialise and cook together openly, a very human and social activity.
But in government software development, that wall still stands. Applications are developed behind closed doors by public servants, and then suddenly appear, fully formed, on release day. This model of closed software development has some very real consequences:
We need not just open data, we need the entire philosophy of open source web development: we need to develop in the open.
The UK Government has released a fantastic Government Service Design Manual. It has some explicit statements and some built-in assumptions. The underlying concepts are to develop government services using modern software engineering processes. This means being open about the code as it is in development (e.g. through blogs explaining the work and github repositories making the code available) and iterating through the design, from Discovery, through Alpha, Beta, Live and (all important in the government) Retirement.
The UK Government Digital Service does its work in the open, through blogs, twitter, and github.
Software development is a process: the application you release at a moment in time is not an end, what's important is that the application should tell a story about itself, so that it can be improved and so that it can be an inspiration for further work.
The Expenditure Database is a great step in providing easy citizen access to the underlying data. But it doesn't tell any of the story of how it works, how the data is processed, and who did the work. The meal has been delivered and it looks very nice, but the kitchen and the process of making it are still closed off from view, hidden in mystery.
Instead one has to dig through the code and make guesses. For example, it looks upon examination that it is drawing the data from a local file: http://www.tbs-sct.gc.ca/ems-sgd/edb-bdd/data.js
If instead the code were in a shared repository and documented, it would be much easier to understand how it works and the rights to reuse and modify would be a lot clearer. As well one would expect there would be both examples of using the data file as well as an explanation of how it was generated.
What Expenditure Database development would look like if it followed this model:
Just a few examples of what we're missing as a consequence of the current implementation:
I welcome your feedback on this post, with any clarifications, corrections, suggestions or pointers.
The Government of Canada has its Web Experience Toolkit on GitHub
By making the WxT open source, it has been possible for organisations outside of the government to use it for their websites and to improve it. Both the University of Ottawa and the City of Ottawa are now using WxT.
There was a writeup in Wired about the WxT: Canadian [Coders] Solve Mystery of Open Source Government.
the Treasury Board of Canada hosted a CodeFest to invite hackers — mostly government staffers — to hack its Web Experience Toolkit, or WET — a set of open-source tools that the Treasury Board uses for building websites. One hundred and fifty people came. Many of them were young developers, excitedly swapping code and sharing ideas across tables.
There are some minor issues: It's Treasury Board Secretariat, not Treasury Board, and TBS is the government's central policy agency not an "obscure Canadian tax-collecting agency". (Admittedly it is difficult to decode that from the TBS website, but if you read far down enough you get to "policies, directives, regulations, and program expenditure proposals with respect to the management of the government's resources".)
Author of the article is Robert McMillan @bobmcmillan
Featured or indirectly mentioned are:
UPDATE: There is a Drupal working group, the next meeting is January 25, 2013 at Ottawa City Hall.
Mayor Gregor Robertson and Coun. Andrea Reimer want the City of Vancouver to support open-source software and open standards.
They also want the city to make as much data as possible freely available to the public. Reimer will introduce a motion [PDF] next Tuesday (May 19) that would see the city endorse the principles of open source, open standards, and open data, as well as start work on publishing data on the Web using open standards.
In a press release issued today (May 14), Robertson said that an “open city” philosophy would help create new opportunities in the information-technology sector.
City of Vancouver set to back open source, open standards, open data - straight.com - May 14, 2009
via Twitter - Rob Giggey (Rob works at the City of Ottawa) - May 15, 2009
(I tried to find the Gregor Robertson press release referenced above, but I haven't been successful - can anyone point me to it?)
Toronto Mayor Miller has also announced toronto.ca/open (which still shows "under construction").
In Ottawa, supported by some City of Ottawa staff but not (yet?) endorsed as any kind of official policy, we're starting the open data discussion as part of ChangeCamp Ottawa.
What can you do as citizens and what can we do as libraries to enable the sharing of our civic data? Is sharing civic data a next logical step for public libraries as enablers of the public space?
May 5, 2009 Web APIs explained on CBC Spark - and Open Data Under Construction
I'm so used to bookmarking stuff into my FriendFeed these days I have to remind myself my blog is also a communications vehicle... I'd be remiss in not posting about this major SOA & libraries initiative.
The Open Library Environment (OLE) Project
With support from The Andrew W. Mellon Foundation, the Open Library Environment (OLE) Project will convene the academic library community in the design of an Open Library Management System built on Service Oriented Architecture.
Duke University is the lead on this project.
UPDATE: Just for clarity I should probably mention that my organisation (CISTI) is neither a core nor an advisory partner. ENDUPDATE
If you want to get an idea of the framework that this fits into, you can see my posting about the presentation Enterprise Services Architectures - Chris Mackie - DLF Fall Forum 2007.
It is great to see library projects starting to use a service-oriented approach, although given that this is the dominant industry trend these days, I'm not convinced that my four years of online and offline advocacy for library SOA actually had much of an impact on this choice.
(You can see my blog postings in category "Service-Oriented Architecture" going back to November 2004:
Still, one does like to be right.
The [National Library of Australia] has recently opened this "Library Labs" wiki space:
The aim of this space is to let our colleagues know what we are doing, to invite comments, questions and feedback and to provide a space for discussion and collaboration.
We have started to redevelop our digital library services using a service-oriented architecture and open source software solutions where these are functional and robust. We are also aiming to take a common ("single business") approach to collection management, discovery and delivery.
We are interested in forming a community of Australian business analysts and developers who are working on similar problems and who are interested in interoperable, standards-based solutions. We are also interested in working with colleagues at an international level to provide prototypes and testbeds for new and emerging standards.
via Warwick Cathro
Assistant Director-General, Innovation
National Library of Australia
What makes me sad is that both Ackerman [sic] and Wallis have missed a key point: if the future is in web services, how can libraries take advantage of that with their current staff configurations? How many libraries in the U.S. have a honest-to-goodness computer programmer on staff? How many have staff with Computer Science degrees? How many staff do they have devoted to the library's hardware, software, and network? How many staff do they have devoted to web services?
In the smallest libraries, perhaps all of these are the same one person.
My article is about modern software engineering for libraries, not on how to staff them. However, in the very same issue of netConnect there's an article by Karen Coombs "Digital Promise and Peril" calling for library staffing to reflect the digital content environment
Many of these digital materials are in jeopardy of being lost because librarians have not yet found adequate ways to collect and manage them. In part, this is because roles and skill sets have been siloed in libraries. Materials preservation issues have typically been the purview of special collections and archives units within the library. In contrast, cataloging expertise has resided in technical services, and technology expertise has typically resided in systems. To collect and manage born digital objects adequately requires these roles and skill sets to come together.
So let me summarize some of the goals and targets of the article, as well as talk about the relationship to promising developments:
Where can we look?
Maybe the DLF project on ILS APIs will help.
Maybe the OASIS effort on standardising search services will be useful.
Maybe it happens by using OpenSearch and simple REST interfaces rather than custom library protocols.
UPDATE: There is another important piece, which is about libraries reaching out and speaking the right language. That's why you need to understand how to express things in terms of SOA, Web Services, and APIs. There is way more innovation capacity outside your walls than you can ever get inside, even if you have the perfect IT staffing policy and budget. From your local superpatrons, highschool CS students, and local college and university computer science departments, to, basically, every CS student in the entire world. You can reach them with contests, with collaboration requests, with invitations to improve your systems... but here's the important part... if you speak their language. CS people love challenges and programming, but they're not going to learn obscure library jargon and usage like OPAC, Z39.50 and database (which means something completely different in CS). You can't say "hey, can you help us improve our OPAC because the Z39.50 doesn't federate across our databases". They're not going to know what the f*** you're asking. Learn the CS language, and a whole world of programmers will open up to you.
To me one of the single biggest missed opportunities is in the digital library community. Ever year, lots of computer science groups, flush with energetic grad students, toil away and produce results that are presented at JCDL and ECDL. And, based on my experience at ECDL 2006, they then present those results entirely to a community of other computer scientists. Where are the librarians? Why are you all at library conferences talking to other librarians? Come to *CDL and ask the computer scientists to build stuff you need. Yes, it's a difficult transition from research to production, but at least join the conversation. ENDUPDATE
The good news is that there are lots of projects out there already - I don't think it's a case that there is no activity. The fundamental point of my article is that for these projects, we have to use enterprise architecture, service-oriented architecture, web services / standard APIs and the whole toolkit of modern network-based standard-data software development. Because if we don't, WE WILL BUILD SILO SYSTEMS AGAIN.
[Sidebar on Scribd: be careful browsing around this document hosting site. Many of the profiles, profile images, and documents are unfortunately very not safe for work. Scribd really needs to put in a moderation / adult content filtering system.]
Why do I think it's important to talk about these topics? Because there really are lots of new developments in the library catalogue and OPAC world, including:
There are also some great modular browser tools out there, including
I'm very much hoping that these developments will open libraries up to be better network participants, with a broader community of developers able to build pieces, and with standards enabling libraries with limited development capability to simply plug-and-play.
June 29, 2007 Casey Bisson on Scriblio and OpenLibrary
It was his posting Is Openness “ethically flawed”? - September 2nd, 2006 that got me on my recent track of thinking about open science, he has followed that up with several more meaty chunks of open pondering:
Thanks to Open Access News for reminding me to revisit Rust's blog.
There is an opportunity for the library and scholarly community to work with the various word processor builders to get to some common standards.
The main players that I'm aware of being Word 2007 and OpenOffice.
Jennifer Michelstein of Microsoft has the (mis)fortune of starting her MS blogging career with the posting [Word 2007] Academic features: citation & bibliography tools.
There's some nice stuff, but I have two concerns:
1. The focus is on building your Master List inside of Word. Yes, you can share lists, but I don't want Word to be the citation master at all
Once a source is created, it lives in two places: your Master List and your Current List. The Master List is the database of all sources ever created. The Current List includes all of the sources that will be used in the current document.
The purpose of the Master List is to save you from re-typing and re-entering information about sources that you commonly use. For example, if you are a Shakespeare scholar and always cite the same five Shakespearean references, you can just select these sources in your Master List and click Copy to add them to your Current List. Now you can cite them throughout your document.
Maybe Connotea is my "master list", or EndNote, or Bookends. You know, separation of concerns? Let my bibliographic software be the master of citation management. Make Word the master of communicating with all major citation repositories, whether local on my machine, or out on the web.
2. They will talk to libraries... somehow. With a software development kit (SDK) that doesn't exist yet. Um, how about we get that kit, immediately?
we are building a platform on the Research and Reference pane, to enable connecting to a library database and importing metadata about sources. We’ll publish an SDK so that Microsoft or any 3rd party data provider can build a service that fits nicely into our Bibliography tools. This project is underway, but the functionality isn’t available externally in Beta2. This sounds like a topic better handled in its own post.
via Dan on eScience
*** 1 Open Repositories 2007 - 2nd International Conference on Open Repositories (ICOR2007)
January 23-26, 2007
San Antonio, Texas
October 2, 2006 Extended abstract, less than 500 words, double spaced
*** 2 code4lib 2007
February 28 - March 2, 2007
Note: Open source doesn't mean free.
InformationWeek has an article SOA Goes Open Source
LogicBlaze has launched an integrated suite of open source code that's designed to get businesses started with service-oriented architecture. Called Fuse, it includes several pieces of open source code from the Apache Software Foundation, plus three from the Apache incubator. The claim that all the pieces have matured must take into consideration that incubator projects at Apache aren't yet full-fledged, open source projects; rather, they're getting an organization and community established around a core code donation.
elemental links has a good overview of the components in Fuse.
here is the link to the outputs of the SOA workshop in
Vancouver last month. Go to
Click on the Workshop March 2006 link to see all the workshop material. Look for the
16 (or so) "service candidates" we identified as emerging out of the
high-level business process of "apply for admission". We demonstrated the
power of service oriented analysis and the existence of abundant reusable
services at or close to the infrastructure level (though Thomas Erl calls
these "application services"). The MIT/OKI [Open Knowledge Initiative] crowd and UK representative [University of Hull - e-Services Integration] were particularly helpful in working this through.
We are starting to piece together some of the next steps and have identified
the following "tracks" going forward:
Business process discovery
Application service candidate discovery
Experimental implementation exercises (Proof of Concept)
Data representation architecture and schema definition
Higher education "basic profile" definition
Security and privacy requirements definition
Technology alternatives selection -- should be deferred to Service-Oriented
By the time we reach CANHEIT, we should have a penultimate version of the
report (feasibility study) funded by the Mellon Foundation. Be sure to
attend the presentation by Richard Spencer and Leo Fernig [A Community Source Student Services System].
Associate Vice President, Information Technology & CIO
The University of British Columbia
via Bill St. Arnaud - CAnet News - SOA in Higher Education
January 05, 2005 Hull.ac.uk portal built using SOA
If your organization has been trying to find a way to get some neat code developed, the Google Summer of Code 2006 may help.
5. What are the eligibility requirements for mentor organizations?
Mentor organizations must be organizations or individuals running an active and viable open source or free software project whose applications are approved by Google's Open Source Program Office. ...
15. How are payments structured?
Google will provide a stipend of 5000 USD per student developer, of which 4500 USD goes to the accepted student applicant and 500 USD to the mentoring organization upon successful completion of a project. Students will be paid 500 USD upon acceptance of their application to the program, 2000 USD mid-program provided sufficient progress has been made on their project, and 2000 USD at close of program provided their project has been completed.
as seen on Slashdot Summer of Code 2006 is On /.
I don't know how good this is, but for what it's worth
The 10th International SOA Web Services Edge Conference, colocated with the First Annual Enterprise Open Source Conference, will take place on June 5-6, 2006 at the historical Roosevelt Hotel in New York City. ...
This year, more than 100 distinguished conference faculty members will present 96 cutting-edge SOA, Web Services and Enterprise Open Source topics in educational classes, panels, and keynotes in six simultaneous tracks for two information-packed days, plus a bonus track, a "Real-World AJAX" two-day seminar
The bibliographic project will design and build an easy to use and comprehensive bibliographic facility within OpenOffice. It will be easy to use for the casual user, but will meet all the requirements of the professional and academic writer. The new bibliographic facility will utilise the latest open standards and will make the fullest use of emerging XML, XSLT, RDF and SRU/W technology.
I think this might be a promising central point for collaboration in the area of bibliographic integration with software. Since you need the citation information in your document composition environment, why not just build bibliographic capabilities right in?
A couple interesting postings from his blog:
while open source software — due to its strong impact on business and on bridging the digital divide — has drawn much attention, it may provide false hopes for the sustainability of openness in other areas of content that need careful examination. The conference FM10 Openness: Code, science and content — Making collaborative creativity sustainable provides a platform for such analysis and discussion, resulting in concrete proposals for sustainable models for open collaboration in creative domains.
The conference will draw on the experience of First Monday as the foremost online, peer–reviewed academic journal covering these issues since May 1996. Not only has First Monday published numerous papers by leading scholars on the topics of open collaboration, open access, and open content in its various forms, it is itself an example of open collaboration in practice: for nearly a decade, the journal has been published on a purely voluntary basis, with no subscription fees, advertising, sponsorship or other revenues. The success of First Monday is demonstrated by thousands of readers around the world, downloading hundreds of thousands of papers each month. This conference celebrates First Monday’s tenth anniversary.
We invite papers for the conference and for a very special issue of First Monday. These papers will be reviewed by a special conference editorial committee. Authors of selected papers will be invited to the conference, scheduled to take place at the University of Illinois at Chicago, 15–17 May 2006. The conference is supported in part by the John D. and Catherine T. MacArthur Foundation (http://www.macfound.org/) and the Open Society Institute (http://www.soros.org/). Other selected papers will be published in a special issue of First Monday, to appear in June 2006.
Papers should address the issues involved in building sustainable models for openness in science, software and content. They can examine technical, sociological, economic/business and legal issues, and can be conceptual or practical in nature.
CFP deadline 6 February 2006
completed papers by 1 May 2006
I couldn't remember the name of one of the Apache projects at SOA Symposium, which maybe tells you what kind of presence it has. I kept thinking "synergy" but it's actually Synapse
It did get tons of press
unfortunately I'm not sure that
There is also of course Apache Axis, which lives with a bunch of other related Web Services projects at