I would claim that, by the time I left, the technical team within which I worked had a better overview of the business processes of the university than any other part of the organisation. We were able to successfully re-apply that knowledge to subsequent development, to great effect.
Paul Walk - SOA and reusable knowledge - February 26th, 2008
Without having ever attended, it nevertheless appears that this group is probably the closest in alignment to the sorts of activities and challenges faced by my architecture group at CISTI.
ITANA’s Constituent Group meeting [was October 25, 2007 as part of EDUCAUSE]. We had approximately 40 people attend. Many of the attendees were from newly formed architecture groups.
Jim Phelps started out the meeting with a short presentation on ITANA, how it is supported and how you can participate. He reviewed some of the findings from the CIO Survey on EA (Enterprise Architecture).
There was active discussion around the role of Architecture in Business Process Improvement. Most CIOs stated that Business Process Improvement was where EA had the least impact on the enterprise. The reasons that were put forth included:
- EA is often seen as a “technology group” with no real input or understanding of the business side
- EA is often on the technology side and the changes actually need to come from the business side and be made by the business side.
- The business owners often want to re-implement the processes that they are currently doing not adopt new processes. We often make the technology match the current business process rather than change the business process.
There was discussion about the role of EA in business process change. This mostly focused on the fact that EA cannot implement the change but they can highlight the issue, document the current process and the improvements and work to lead the discussion around the change.
We also discussed the way that EA engages in the enterprise. Some institutions use EA as a gate function. Projects don’t get the go-ahead without EA’s approval or they are allowed to continue with contingencies. Other institutions use EA to guide projects and illuminate issues. Where does EA plug-in to the organization and the project / funding processes?
itana.org - EDUCAUSE CG Meeting Notes - October 26, 2007
There is also an ITANA wiki.
I was privileged to have been paired with Chris Mackie of the Andrew W. Mellon Foundation on the DLF programme, and fortunately I think our talks were quite complementary. Chris talked about Enterprise Services Architectures (ESA), presenting a very clear-eyed view of the sustainability challenges of IT infrastructure for higher education, and how ESA and some specific Middleware ESA (MESA) projects are addressing those challenges. His presentation (PDF) is now available from the DLF site.
It was great to talk with various enterprise/IT/technology architects at DLF Fall Forum 2007 about our various ideas an approaches to modeling and implementing the new library technology infrastructure. But it's clear there are still more library technology architects out there (even if they may not have that in their job title), I ran across an interesting blog posting by Jonathan Rochkind:
Notes on future directions of Library Systems from Bibliographic Wilderness blog, September 28, 2007
Here are some notes on near/medium future directions of the library systems environment/architecture, and a sketch of requirements on where we want to go. These notes may or may not end up as part of an internal white paper here, as we analyze where we want to be headed (something good to do anyway, but we got a kick in the pants when the vendor ended development on our current ILS).
...
The library systems environment has grown from being composed of a single “Integrated Library System” at the origin of library systems in the 1980s, to being composed of an ‘ecology’ of systems in most libraries today. These different systems could be from various sources (proprietary and open source), fulfill various–both overlapping and distinct–functions, are used in various ways by different subsets of users (both library staff and our end-users), and interact with each other to varying degrees (some well, some poorly).
I wonder if I should consider adding a "library technology architecture" category to SLP?
PS Jonathan was at DLF but we didn't formally meet, it's odd how sometimes virtual connection and discovery is easier than in the physical world.
Overall I think it went very well, the room was packed (to my considerable surprise) and I got good questions from the audience - for once actually too many questions for my allotted time, so I unfortunately stole 10 minutes from Chris Mackie, whose presentation followed mine.
The slides are up on Slideshare, they were done in Apple Keynote, in case you're wondering.
http://www.slideshare.net/scilib/serviceoriented-architecture-for-libraries/
All of the links and background info for the presentation are available at
http://www.connotea.org/user/scilib/tag/dlf2007akerman
The following is not the actual narrative, but it gives you enough to follow the (mostly conceptual) slides:
I started with a great quote via Katherine Kott of DLF Aquifer, from the previous night's session on collaboration, stating that we need "vision translated into operational success".
Service-Oriented Architecture addresses problems in technology planning,
design and implementation.
First, we must understand what some of these problems are.
Fundamentally, technology implements the desires of the organisation.
Too often we focus on our immediate desires ("What do you want") rather
than long-term planning.
This has meant we get what we asked for (e.g. incremental catalogue
interface changes) rather than what is best for the organisation
(transformed catalogue)
In order to better align our implementations with our goals, CISTI uses
an SOA derived from our Enterprise Architecture. In our EA methodology,
we start with business needs (Who are you?), and only then proceed to
identifying the gaps to be filled. But you must look beyond the current
state (Why are you here? Where are you going?)
So what are the transformational challenges we face?
In a way, we are in a fantastic world for the traditional goal of a
library - ubiquitous, zero-cost, perfect copies of information. No more
scribes devoting their lives to copying a few books. But this has
tremendous implications in terms of preservation, versioning, citation
etc.
Also, there are major changes in moving from a physical world where we
engineer bridges, to a virtual world where we "engineer" software.
Software doesn't have most of the constraints that apply in the physical
world, and we have to be careful with analogies to physical objects. It
would make very little sense (except perhaps to a US Senator) to build
half a bridge, or a bridge that goes nowhere. But in the software
world, building components, even if they are only part of an overall
goal, can be extremely useful. And software is not location constrained
- it can be accessed from any location - there should be no "software to
nowhere". The network is everywhere, and we need to understand how that
transforms our systems design.
So how do we apply the notions of architecture that we understand from
the physical world, and that have for centuries and continue today to
enable us to build very complex structures with many components (heat,
lighting, electrictiy, plumbing, etc.)
Well, we can use the idea of modeling. Just as we make models of
physical objects, we can make models of business processes. But we must
ensure that we don't get stuck in our models or frameworks, and do move
to implementation at some point.
CISTI has found that its methodology, which is focused on getting to the
implementation framework, and beyond that to design and development, has
been invaluable for turning business ideas into implemented software.
We are fortunate in that we are a library with substantial in-house
technology resources, including a dedicated architecture team (the only
library EA team in the world??)
SIDEBAR: I met the Head of eArchitecture for the British Library at my presentation, so now I think I should say "one of the very few library EA teams in the world". END SIDEBAR
This structure may be different for
academic organisations, with faculty groups that have departmental IT,
central IT, and academic consortia.
Within this larger mehtodological framework, SOA is an approach to
identifying "servicifiable" functions in the organisation, and building
larger service offerings from various combinations of services.
In the architecture analysis phase, CISTI models all of our
technology-related activities, at a high-level this model describes all
the technology-supported activities, with functions such as "provide
library service" that then decompose down into many many lower-level
models.
Recently we have been specifically modeling a Trusted Digital
Repository, using the OAIS model and converting and adapting it.
This EA modeling work then leads to the derivation of SOA services
[depending on audience profile I may talk a lot on this slide about the
details of SOA, or not].
CISTI has used and is using SOA successfully as the foundation of
various projects - this is not just theory.
SOA is also not an agility-killer bureaucracy - services actually
provide pieces that can be used for small experiments, while
simultaneously moving the organisation in its stated general direction.
Ideally, we should all be collaborating on SOA, rather than building SOA
silos. I have blogged a lot about this, but with little response. Are
the individual framework groups only talking amongst themselves? How
can we use their efforts most effectively?
SOA is certainly being explored in many areas of the library space,
including various digital library initiatives and modeling efforts (some
of which will be presented in other forum sessions).
SOA also offers the possibility of improving the catalogue by breaking
it into component services, and enabling the sort of simple layers
(widgets, mashups, javascripty stuff) by providing sustainable,
well-architected underlying service capabilities.
For academic libraries in particular, SOA offers a compelling
opportunity to connect with existing and future scientific activities,
as many Cyberinfrastructure projects have SOA at their cores. Can we
use modelling at even the highest level (the entire scientific
communication cycle) to guide our service implementations?
Ultimately to build sustainable, non-siloed, modern information systems
to support scholars and citizens worldwide, we need to build many
bridges (this one happens to be built of lego components, in case you're
worried that giant ducks have invaded the US). Between libraries and
scientists, between technology and business, between funders and
software developers, between computer science and library science...
We need to move beyond frameworks, using solid governance to ensure that
SOA plans become running SOA code, while avoiding getting stuck in giant
projects that don't deliver. As well, even if you don't have the scale
or the capacity to build your own SOA, you can still participate as a
service consumer.
I'm not crazy in this (at least I hope not), as evidence for which I
present some starting points about SOA in the academic and library
sectors.
Questions...
The End
SIDEBAR 2: In case you're wondering, there's a slight science fiction theme running through the presentation, inspired by the "federation" part of the organisation name. The Four Questions are from Babylon 5, the goodish aliens, the Vorlons, ask "Who Are You?" and represent the forces of order, the evilish aliens, the Shadows, ask "What Do You Want?" and represent the forces of chaos. "There Are Many Copies" is from the opening sequence in the new Battlestar Galactica, using the great Creative Commons images I found worked out as a nice way of avoiding using a screencapture (plus using a screencap might have been a bit distracting as it shows a scene from BG where the many copies of Sharon Valerii/Number Eight are not so much wearing any clothes). I had a better closing quote but I couldn't remember it, so I used a tag line from Buckaroo Banzai Across the 8th Dimension, to close out the "where are you going" questions theme. END SIDEBAR 2
SIDEBAR 3: I was actually amazed to be able to locate the "bridge to nowhere" Google Earth image. I had seen the unfinished highway bridge out of the airplane window as the plane was coming in to land in Philadelphia and by the time I thought to GPS-mark it or take a photo it was already past. Incredibly, doing an image search on
"bridge to nowhere" Pennsylvania
brought it up as the first hit - Google Search, Google Earth and the Internet continue to amaze me. END SIDEBAR 3
SIDEBAR 4: I bought my carbon offsets from MyClimate.org
flight from: Ottawa, ON [Macdonald-Cartier International Airport], Canada, YOW
flight to: Philadelphia, PA [Philadelphia International Airport], USA, PHL
flight via: Toronto, ON [Lester B. Pearson International Airport], Canada, YYZ
return, economy
flight distance: 1'845 km
flight passengers: 1CO2 Emissions: 0,511 t
Total costs for compensation of your flight: SFr. 20.00
According to Google
20 Swiss francs = 16.3543582 Canadian dollars
Closing Date
06/26/2007 (CLOSED)
Applications will be accepted until 23:59 Eastern Standard Time
Ottawa - Ontario
CS-3
This is a 3 year term term position from the date of reporting.
Education
A Bachelor degree in computer science, computer or software engineering, or other related field.
Formal training in Enterprise Architecture is an asset.
Language Requirements
English
I am pleased to announce that I will be presenting my paper "Library Service-Oriented Architecture to Enhance Access to Science" at IATUL 2007, my session is tentatively scheduled for
Tuesday June 12th 2007 at 10 am
The presentation will be about 20 minutes plus 5 minutes for questions.
UPDATE 2007-06-12: The presentation has now been completed and is available online. ENDUPDATE
My paper is a brief summary of CISTI's EA and SOA success, based on the efforts of the rest of the architecture and application development teams. I intend it both as a demonstration of what is possible, as well as a discussion starter, to help other libraries move beyond basic theoretical frameworks to start building a richer network of library machine-to-machine data services that we can all use for applications, websites, mashups, and whatever is coming next.
John pointed out to me the Stats Canada Information Technology (IT) Conference 2007, April 4-5, 2007, Ottawa.
The conference is entirely about Enterprise Architecture and Service-Oriented Architecture,
including speakers from inside and outside of the Canadian government.
http://www.statcan.ca/english/conferences/it2007/
http://www.statcan.ca/english/conferences/it2007/all_abstracts_E.pdf
Among the new approaches, enterprise architecture has the objective of building a broad consensus within the IT community and among users. Having emerged in recent years as a central concept in the engineering of information systems, this architecture approach appears to result in a better organization of applications into discrete services.
Service Oriented Architecture (SOA) promotes an approach where automation logic is partitioned into individual units (services) that are highly interoperable, but generic enough to fulfill both immediate and future automation requirements. The major benefits associated with a successful adoption of SOA typically begin with improving an organization’s ability to share data across disparate systems and establishing an architectural model that is increasingly independent of the enabling technologies. Services are deliberately positioned as highly reusable IT tools that can be exploited over a long period of time. The end result is an increase in an organization’s overall business responsiveness (agility).
Service-Oriented Architecture (SOA) is not for the faint of heart, nor for those weak in technology expertise.
SOA is fundamentally a methodology for systems design, informed by an overall Enterprise Architecture (EA).
If you don't have the scale of organization, or the organizational technical expertise, to sustain an EA and advanced development group, then SOA may not be for you. You may want to focus more on consuming services and connecting services together.
Or in other words, we just have the latest in a line of IT best practices:
Keeping that in mind, the library environment as a whole is clearly of the scale to benefit from an SOA, it's just a matter of how we put it all together. Who defines the services? Who will the service providers be? What will the business model be for service consumers?
All this by way of introducing some recent reports and presentations on SOA, as well as signalling that I will be providing more of my library SOA thoughts in a few upcoming venues.
Firstly the National Library of Australia IT Architecture Project Report (PDF) has gotten some attention for its prominent mention of SOA. (Thanks to Lorcan for pointing it out to me, DLTJ also mentioned it today.)
Second, via Brian Kelly I see that there were recent JISC conference presentations on EA and SOA.
e-Framework, SOA and EA: Using the e-Framework, Service Orientated and Enterprise architectures to develop more flexible and better aligned ICT infrastructures Abstract
- Session commentary
- Presentation "The e-Framework, SOA & Enterprise Architecture" - JISC (PowerPoint)
- Presentation "Enterprise Architecture" - Open Group (PowerPoint)
Overall I think both SOA and EA are challenging topics to present simply. That being said, I think both are typically presented far over to the extreme of complexity. What was it that the rather inaccurately titled SOA for Dummies said you had to have in your SOA? ESB, registry, broker, supervisor, BPM, yadda yadda... It's simply not true. That's what you'll get if you let the kindly vendor hold you by the hand and take you into the twisty SOA maze. All you actually need for SOA is to make sure you're using some basic software design principles:
autonomous, loosely-coupled and coarse-grained services with well-defined interfaces that support
IT architect is one of the hottest jobs in tech right now, thanks in part to more companies trying to build service-oriented architectures. The problem: Not everyone agrees about what IT architects are supposed to do or know.
...
IT standards consortium Open Group is launching the industry's first professional association for IT architects. Called the Association of Open Group Enterprise Architects [AOGEA], it's looking to "elevate" the profession by promoting certification and establishing ethics standards and architecture principles.
...
A shortage of IT pros who can guide SOA development is one of the biggest hurdles for SOA adoption. Analyst firm ZapThink and staffing company Excel, seeing a market in that shortage, have launched the Architect Resource Center, a consulting service
from InformationWeek - IT Architects In High Demand - January 29, 2007
I'm all for architecture principles and standards. I think certification is bullsh-t though (not just for IT architecture, for any kind of IT). And if you can't figure out ethics on your own, no Association is going to help you. AOGEA seems to have a needlessly complex structure based around certification levels. Count me out for the time being. Err, although the conference in Paris looks nice...
For the record, my title is Technology Architect - IT, but since that is rarely meaningful to people, I usually tell them I'm a technology planner (or if I'm feeling grand, an enterprise strategic technology planner).
Dave Pollard has some interesting ideas in The Challenge of Reintermediation.
My comment was that the reintermediated IT professional role he describes sounds a bit like enterprise architect to me.
I do think that "information gathering" is going to be a big challenge to convince people to reintermediate. Senior managers, who could benefit most from the time-saving, are too aloof to either surf the web or to request this kind of information service, so they operate in a perpetual information vacuum.
The rest of us are quite content to take a bit of time to surf the web on our own.
Remember the challenge of "switching cost". People will only switch to a new service if you can demonstrate that it is DRAMATICALLY better than the previous one. That's why we have settled on CDs/MP3s DVDs/Divx Windows 2000/XP Google. It's good enough. There are billions of dollars trying to move us to AAC and super audio CD and HD-DVD and Windows Vista, and you can see how successful that has been. Now you're going to try, without billions of dollars, to move me from Google to a human intermediated search? I don't think so.
Summary
I am reviewing this book primarily in the context of Service-Oriented Architecture (SOA) for Libraries, which is a rather dramatically different audience than it is intended for.
I recommend the first three chapters of this book, along with the "loosely coupled" section of chapter 5 as a good introduction to this topic. As well, Chapter 11, on SOA Governance, is excellent. However, its main audience is large organizations (who are the main current adopters of SOA), so much of the content may not apply to libraries, particular those with limited technical resources.
Although the Dummies books proclaim "A Reference for the Rest of Us", this book would really more accurately be titled "SOA for Dummies... Who Work at Giant Fortune 500 Companies". Much of the technology and processes they present are beyond the reach of all but the largest organizations.
To my surprise and dismay, there isn't a single mention of Enterprise Architecture (EA) in this book, whereas I feel that for the scale of organization this book targets, EA is absolutely essential. Without EA, the scale of activities and technology described in this book is impossible.
There is also a very important discussion that needs to be had about agility versus structure. I am not convinced you can maintain organizational agility as a goal, with the complexity of the management and technology structures this book describes. The book does not address this issue at all.
Overall the flow of the book is a bit uneven, both in terms of writing style and in content, and it doesn't present a coherent "SOA story". To some extent this is due to the nature of the Dummies books, where they want each chapter to stand alone. I found it made for a bit of a "here's an important concept" ... "and here's another one, with some vague undefined relationship to the previous one".
This book will mostly give you a lot of "why" and "what" about SOA, not a lot of "how". That is the nature of the beast, however - I don't think there's any other way to present it, with out it being a 3000 page book.
The companion website www.dummies.com/go/soafordummies is highly promoted on the front and back covers of the book but it does not seem to exist, which seems like a bit of rather bad planning.
Sidebar
UPDATE 2007-08-09:
If you're coming here from SOA Center, you may be expecting Beth Gold-Bernstein's review.
The SOA Center link is currently wrong. She blogs at ebizq:
July 24, 2007 Beth Gold-Bernstein SOA for Dummies
July 31, 2007 Beth Gold-Bernstein Reactions to SOA for Dummies Book Review (in which she refers to my review as having some similar points to hers)
I did leave a comment on SOA Center correcting the link, but it has not been posted.
ENDUPDATE
Details
I don't come to this book with a blank slate, quite the opposite, I've been trying to find ways of clearly explaining SOA to both business people and technical people for the past few years.
Just to stake my prior claim, I think I did a pretty good job in my December 9, 2005 presentation "Service-Oriented Architecture Methods to develop Networked Library Services" (video, PowerPoint) which predates this book by about a year.
It is a difficult story to tell to be sure. To my mind, SOA sits at an intersection of Enterprise Architecture, software engineering, Web Services, business and technology. So in a way, the challenge is "let's take a bunch of things that people don't understand, and try to explain a layer on top of them, that depends on all of them".
Also, in the interests of transparency, the informal style used in this book is not one that I enjoy for learning. My ideal informational book is just facts piled upon facts - I don't need a story, or to be put at ease.
I tried in my analysis to ignore that stylistic aspect, as well as trying to come with as blank a slate as possible, keeping in mind I've been deep into SOA for the past couple years.
Here are my thoughts, chapter by chapter.
Chapter 1 - SOA What? (this makes more sense if you pronounce it so-a, as the authors do, rather than S-O-A)
All of Chapter 1 (PDF) is available free online from Wiley.
They talk about their audience, but I do think they make a mistake in not scoping the book. The full SOA they describe in the book's 300+ pages is one that could only be used by a large organization. Think, General Motors revising their information technology, not Bob's Corner Store. They should have made it clear that you can adopt some SOA concepts, without going to the full, complex, vendor-rich (and vendor-enriching) extent that they document.
For example, they start out with some clear explanations about how SOA is an enabler for business to work better with the IT department. You may at that point be thinking, "my problem is not working with the IT department, my problem is I don't have an IT department".
So in reading the book, be mindful of their audience. And also, I suggest you think of how SOA could help the entire community of libraries, collectively, if the scale of your individual organization is not appropriate for SOA.
They do extract good key points, like the fact that reuse is at the core of the SOA goals and methodology. At this level, they face a challenge that I don't think there is a good solution for, which is that you have to talk a lot at the beginning about SOA "why" and "what", without being able to easily walk through a lot of SOA "how".
I also think that their definition of SOA is a bit over-broad. Some of what they describe as SOA is what I would consider Enterprise Architecture (EA). Now, I certainly think SOA works best within an EA context, but there doesn't seem to be any mention of EA at all. So I disagree when they say
1. The IT organization will partner with the line of business managers to create a high-level map of what the business will look like.
That surely needs to be done, but that's EA, not SOA. To me, Service-Oriented Architecture is just the architecture of the services, not all the processes of business process analysis as well.
Chapter 2 - Noah's Architecture (oh I get it -- literally, just now - it's a play on words, see. sigh. I spent the entire chapter wondering, what does Noah have to do with it?)
So at this point, the conceptual assault really gets underway. To get to the heights of SOA, you have to build concept upon concept. Unfortunately, each one of those concepts could be a book in itself, and here they get a couple pages at best. I'm not being critical - I don't see any other way you could do this, and it's the same approach I have used, but I can imagine it being overwhelming reading for someone coming from outside a software engineering background.
They give an ok explanation of software architecture, and hit the high points of the history of software engineering that got us here. I liked their mentions of business logic and the key concept of separation of concerns. Again, they are talking about big organizations, for example, on page 16, they state
In the Good Old Days, a business unit would ask the IT department to create an application to solve a specific problem. To accomplish this goal, the IT department would write a set of customized programs.
This is absolutely, 100% true. For a certain scale of organization. If you're thinking instead, as I mentioned before, "I wish I had business units. Or an IT department." then SOA may not be a good match for you.
They then take a bit of a side trip into components. I'm not sure I would have used this terminology. It's certainly true that we tend to run out of useful worlds to describe certain concepts, but to me components are about a specific subfield of software engineering, Component Based Design. Software components have a very specific meaning, a meaning which is different from the one given in this book.
This is important, because if you read this book and say "component" to an IT person, they may not have the same understanding that you got from the book. In a similar channel, the use of "plumbing layer", which is continued throughout the book, is most unfortunate. "Infrastructure" would have been a much better choice. Whatever dubious merit there is in the informal term of "plumbing" is going to be lost the first time a non-tech person uses the word to a technical person, and the technical person thinks "Plumbing? What the heck? Is this guy an idiot?"
At this level, there does have to be a bit of "hand-waving" over the explanations, this is particularly egregious (although the authors themselves acknowledge this somewhat) in the "simple adapter" explanation. Adapters are probably some of the most complex challenges an organization faces in transitioning to SOA.
I did enjoy the brief excursion into "application archaeology", particularly since my original SOA presentation on SOA had a slide or two on this concept, with a nice quote from Vernor Vinge's writings, but I had to take the whole segment out because people said "application archaeology, what the heck were you talking about there?" (I'll try to dig up the quote, it's excellent.)
I'm also not a super big fan of the "black box" terminology used, but it does capture the concept well enough I suppose.
Chapter 3 - Not So Simple SOA
A bit of a jump from components to Web Services, with a bit more handwaving. I also didn't like the strong link between Web Services and SOA. While their statement that SOA is not Web Services enhanced (or I guess I should say "is not Web Services 2.0") is certainly true, it's also true that you can do SOA without Web Services at all. The two are, as we say, loosely coupled.
There's also a bit of a jumble about business processes, I'm not sure it fits well within this chapter.
Finally, while silos are mentioned, I think they really missed an opportunity to emphasize a key failure of siloed software - that it locks up useful services deep inside the silo, where no one can get at them. This aspect is mentioned only in passing.
All those caveats aside, I did feel that Chapters 1-3 are about as good a clear explanation of this extremely layered concept that you can get (well, that is to say, almost as good as my explanation :)
You should really jump from Chapter 3 right to Chapter 5, I'm not sure why they placed Chapter 4 here.
Chapter 4 - SOA Sophistication
Woo boy. Chapter 4. If you are coming from a small organization, or a non-technology background, your service experience of chapter 4 will be akin to hitting a brick wall. Chapter 4 layers on the SOA "essentials" at a breakneck pace, presenting as essential elements of SOA things that are, each of them alone, HUGE complex business activities. We are definitely into the General Motors scale here. The Galactic Motors scale, in fact.
Let me make it immediately clear: I do not personally believe you need all of the stuff they describe in order to use SOA, and to participate usefully in an SOA. The basic SOA methodology can be used by almost any organization that has at least some idea of its basic business functions (and some business decisionmakers) and a small IT group to design and build services.
So, what are the bricks in the giant Chapter 4 wall?
In looking at that list, my first reaction is "hmm, are there any 30 million dollar cheques I haven't cashed yet?" and secondly, that almost anyone, even someone from a large organization, who reads this chapter is at great danger of rejecting SOA out of hand. You're telling me that SOA is going to improve my organization by adding re-use and handling complexity, and you want me to add a gigantic laundry list of complex technologies (many of which don't yet fully exist) to my organization? No thanks. Call me in 10 years when you've got it all figured out.
Now I want to be clear: all of these are important aspects to consider, and many of them will need to be adopted, by very large organizations with huge numbers of services. But I imagine that most libraries will have a handful of services - the basic functions that the ILS performed, plus a few others. That doesn't require anywhere near the scale described in Chapter 4.
I have certainly advocated that the most important piece I think will need to be layered on the base SOA is workflow, the orchestration server (you can listen to my presentation - yes, I will put the slides online eventually too).
I also want to address the SOA Supervisor, and the whole idea of automatically monitoring Service-Level Agreements. I think this is very deep voodoo. I am quite happy to be corrected, but I think this capability does not exist in any meaningful, easy-to-use, standardized way today. To me, for SOA Supervisor, you might as well read SOA Magician, or SOA Magic Box. It simply is not possible to perform most of the supervisory functions they describe, and even attempting to do so would be hugely complex and vastly expensive.
Chapter 5 - Loose Coupling and Federation
The loose coupling part of this chapter is good, but it really should have been separated out and put after chapter 3. The Federation part only applies, as is often the case in this book, to very large organizations, that are so complex and diverse that they need to have multiple, federated, SOA domains, with a different set of rules applying in each one - think IBM HR SOA domain and IBM Supercomputing SOA domain.
(Note to CISTI architects: not the same as our domain concept.)
There is a bit of an overlong excursion into software-as-a-service (SAAS). I think it could have been covered quite well by just saying "you may want to use external services, as well as your own internal ones".
I disagree quite strongly with the concluding section, which asserts that SOA is part of the industrialization of software development. I certainly agree, and I have in fact asserted that SOA is simply our latest understanding of how best to do software development, but industrialization is way, way too strong a term. Software is not constrained by the physical world, that's why you can't build software the way you build a bridge or a car. I'm certain that Toyota knows, second-by-second and gram-by-gram, what is happening as a vehicle moves along its assembly line. That is simply impossible for software, and it will probably never be possible.
Chapter 6 - XML
To be quite honest, I don't really understand the emphasis placed on XML within SOA. Thomas Erl certainly uses it as the keystone of the explanations he gives in his talks and books. To me, you could get by with two sentences: "Software design works best when you can use standards. XML provides the ability to create data standards."
This chapter goes into way way more than I think you need to know (for SOA purposes) about XML, SOAP, WSDL, UDDI and even (oddly thrown in) HTTP. It also seems to express a view of XML as a sort of "magic glue" that ties different programming languages and systems together. Um, we were able to connect systems together in the bad old pre-XML days. We called it "using APIs".
Chapter 7 - Dealing with Adapters
This chapter starts out with a strong statement, unfortunately, one I disagree with
Adapters make SOA possible. No adapters, no SOA -- it's as simple as that.
Um. No. I recognize that not all is green fields, but not all is layers and layers of legacy either. This is coming from big corporation, banking thinking. It is certainly true that when I enter a number on my bank's customer self-service website, it is going through layer upon layer of adapters, ending up at a mainframe Which Must Never Be Touched.
But for example, library catalogues? You'd have to be mad to build adapters to catalogues. Blow them up, replace them with a green field of services.
This chapter also creates a dependency between adapters and service registry and service brokers. This is unnecessary. It also continues the early "simple adapter" nonsense, on page 93
Technically, building an adapter isn't too difficult.
Um. Yes it is. What they're talking about is API Adapters. My mainframe interface takes some ancient binary IBM format, and I wrap it with an XML adapter interface. That isn't too difficult, yes.
But most of the problems facing libraries, at least, is that there is no interface. There's nothing to easily adapt. You have to write an entire interface layer. That is hugely complex. I have to imagine this also applies widely outside the library technology context.
I have to admit my final written note on this chapter reads, simply: this chapter is baloney.
Chapter 8 - The Registry and the Broker
We're now getting into more and more complex technology, and I'm getting more and more dubious about its actual existence and workability. To be quite frank, unless I see lots and lots of demonstrations in the real world of this stuff working, I think it is a lot of wishful technothinking. And I'm not convinced even if it was in place, it wouldn't create a huge agility barrier.
I'm in agreement with the people who say a UDDI service registry simply isn't going to be maintainable in the long term. You might as well call it "the perpetually out-of-date service registry".
Chapter 9 - Enterprise Service Bus (ESB)
I'll willingly admit to some ignorance here. We have certainly been discussion ESB at my organization, but we have yet to be convinced that it is necessary.
Again, there is a scale issue. ESB is basically the reliable message queue idea. You need a reliable message queue if every (service) message in your organization potentially has huge financial impact - that's the only thing that can justify the cost and complexity of an ESB. For a big org, or one doing heavy financial stuff, this certainly applies.
To a library, loaning books for free? If the occasional message drops, I think we can survive.
Chapter 10 - SOA Supervisor
This is currently indistinguishable from magic, which puts the lie to Arthur C. Clarke's famous maxim, because this is certainly far from being an advanced technology.
I'm not even convinced this is an existing technology.
Chapter 11 - SOA Governance
Excellent coverage of the transformational issues raised by moving to an SOA. A must read for anyone doing a serious SOA initiative.
Chapter 12 - SOA Security
I'm not entirely convinced security is part of SOA, it's a separate service or services to be defined within the architecture. But the kind of security they're talking about in this chapter is more "organization-wide complexity protection". Again, the heavy-weight approaches they indicate are appropriate only to large-scale organizations.
To get a taste, the essential pieces they recommend are:
If you're not the scale or type of organization that has the above three concerns, you probably don't need to think too much about the type of security they are describing.
Chapter 13 - Where's the Data?
I have to admit a bias - I don't care about data. I know however, that many EA people, such as Jane Carbone, consider data the essential starting point of analysis. It is of course important that someone in the organization cares about the data, and this chapter has some useful guidance about the topic.
Chapter 14 - SOA Software Development
In this chapter, we get a little bit of "how". But it's not really entirely about software development.
For example, I was again surprised to see yet another way to describe Enterprise Architecture, without saying "Enterprise Architecture". This time they call it "building a business process map".
It then dives into iterative prototype development, which is all well and good, but not essential for SOA. You certainly could develop services using the waterfall model. Iterative prototype is certainly a good software development practice, but it doesn't have any strong connection to SOA in particular.
The last part of the chapter discusses Web Services and SOA testing, and this is an important topic which they cover well.
Chapter 15 - The Repository and the Registry
My concerns are as expressed about all these additional layers before:
Chapter 16 - Do You Need a SOA?
Some relevant items for any organization, but some of their questions, e.g. #1 "Is Your Business Ecosystem Broad and Complex" apply again only to large organizations. I think you can usefully have an SOA even if your business is simple and narrow.
Chapter 17 - Making Sure SOA Happens
Useful information, if a bit generically applicable to any type of major organizational change.
Chapter 18 - SOA Entry Points
Some reasonable guidance.
Chapters 19-26 are supposedly under the rubric "Real Life with SOA". But I think more accurate would be "here are a series of chapters about major SOA vendors". I was hoping for real organizational examples, but this section (which runs from page 225 to page 308) is mostly just "here is the SOA stuff from IBM / HP / BEA / Progress / Oracle / Microsoft / SAP / JBoss". To me, it was a waste of almost 100 pages. A single page pointing to the vendor SOA websites, with perhaps some of their key articles or white papers indicated, would have sufficed.
I think you've already seen from this review that SOA is a complex enough topic that 80 pages could have been usefully deployed for more explanation and for more discussion of the "how" of SOA.
We wrap up with Chapter 27 to 29, where we get some "Part of Tens" guidance and links, including even more vendor links. My written notes conclude this part is "vaguely useful".
Overall, the book started very strong, and really could have been for the Rest of Us if they had continued in that vein, but instead it ends up being SOA for Really Big Business.
If that sounds like a good match for you, or if my review has piqued your interest, you can buy the book from my
SOA Book Store (powered by Amazon.com)
or
SOA Book Store (powered by Amazon.ca)
Wikipedia - Book Sources - ISBN 0470054352
UPDATE 2006-11-27: This conference was cancelled. ENDUPDATE
CISTI's own Daping Tan will be presenting with the EA mentor/guru/consultant whose methodology we use, Jane Carbone, their session is
Integrating EA into the Organization - Roles, Processes, and Policies
The conference runs November 7-9, 2006 in Chicago.
Enterprise Architecture Roundtable 2006
If I have further info about this conference, I will be tagging it EAR2006.
Chunka Mui will be presenting on Emergent Knowledge.
You can view the slides (PDF) from a previous presentation Emergent Knowledge:The Spark for Information Advantage.
He's selling his essay on Amazon.com
but you can read an earlier version on his blog
In a range of industries, including insurance, financial services, healthcare, and complex consumer and industrial products, emergent data is changing the strategic context in which companies manage their organizations, products, customers, and markets. In the right hands, new information about product and process conditions, customer preferences, and other environmental factors are sparking numerous innovations. But, because these data types are just appearing, their strategic significance is often not well understood, so it is usually ignored, and rarely leveraged.
We've been having some internal discussions about service ownership. One of the things we found useful was separating the business-level service description, from the IT-level service implementation.
What we're currently thinking is that the business should own the business-level service ideas, but the IT-level implementation should have an IT owner.
This is important, because I think you can get a bit carried away in the "business drives technology" or "business-driven SOA" enthusiasm. Yes, the business is in charge of generating and validating the business ideas, but they shouldn't be deep within your IT organization making technology decisions.
In addressing the question of service owners, Ali Arsanjani of IBM uses the concept of Service Domains:
Business/IT alignment. Great.
Each business line goes about their way to achieve it, but soon will find out that there is one conspicuous issue: who is the Service Owner? If I build this service and it is used in multiple places, who pays for the changes if the changes are those I do not specifically "care about" in my line of business?
Thus appears the notion of Service Domains, where a domain defines a set of related services that some "one" can own, maintain, support and (obtain) fund (ing).
To make a [change] to a service, contact the Service Domain Owner.
I guess it depends on the size of your organization and the number of services - I can certainly see large organizations with many services needing to take a Service Domain approach.
Ronan Bradley examines this in SOA Architect required: Only superheroes need apply
The key artefacts within a SOA are of course the service definitions (and their related data models). The benefits of reduced cost and increased flexibility rely on these being maintained, evolved and most importantly used as often as possible. So what is the role of the owner and what does he/she own? To my mind ‘owning’ in a business-aligned SOA environment should be in the sense of being responsible for making sure the services are used and used in the way to give most benefit to their business. This person can’t be our super-SOA architect who has all those other jobs!
The solution to me seems quite obvious – if a SOA is business aligned, the business must be more involved in SOA. this suggests that the owners of the service need to be directly aligned to the business owners of the associated business services. How this is done in practise will depend on the organization and its structures and goals. What will be true in every case is that the focus of this role should shift towards business skills and away from technical skills. The service owner can always go ask the architect for help and the architect can then focus again on ensuring that principles and guidelines are followed.
I think this is useful - the owners of the implementation should work with the business owners of the business services. I am not convinced though that this means the implementation owner skillset must inevitable move towards a business focus - I think the implementation owner must retain a deep technical understanding, in order to be able to best advise the business of the impacts of changes. If the implementation service owner is going to come to the architect all the time, you might as well just make the architect the implementation owner - the whole point of having a separate service owner is that the implementation owner has the power and ability to make decisions about the service (in consultation with stakeholders). So to me, implementation owner is a primarily technical and coordination role.
I wonder if we can agree on some terminology.
business service seems fairly clear
but below that? IT component owner? implementation owner? implemented service owner? IT service owner? What are you using in your organization?
Bradley points to a useful Gartner article that exposes some of the transformational impacts a fully-implemented SOA may have on your organization - The Organizational Impact of SOA
Then the idea of quality of service comes into play. Because an organization is going to a service-oriented architecture, one of the biggest transformations is going to be in delivering services that do what they are supposed to do, when they are supposed to do it. In other words, quality of service is more important now than the services that are actually delivered because quality of service is what determines whether someone can actually use what has been delivered.
The organization has to be ready and able to stand up and say, "Ah, we can track, manage, and deliver or guarantee delivery of a certain quality of service." In some organizations that may mean, actually bringing in or using part of the staff to be service providers, to become a service provision organization, as opposed to an application provision organization.
Two other issues come to mind – ownership and distributed transactions. Who owns what? Who owns the data, services, and processes, and how does one bill back for those processes? Who is responsible for the system that those processes run on? In terms of distributed transactions, an enterprise has processes that span organizations, business applications, computing systems, and even companies. And you have to understand when you are doing this transaction across a lot of different boundaries, how are you going to handle that?
Another issue comes as you build capabilities out of BPEL orchestrations of services. Do the orchestrations then have owners? How do you avoid having layer upon layer of owners?
What do you do if you're in a technical library, but you are not, technically, a librarian?
You embrace the joys of cataloguing. Repeatedly.
Our little architecture library has its own page of manually-entered titles on our internal wiki, and now
Steve has manually input them all into LibraryThing.
http://www.librarything.com/catalog.php?view=cistiarchitecture
I had come up with a clever way to get them into Universal Import, but LibraryThing Import is either paralyzingly slow, or broken, at the moment.
Of course, you can't actually borrow the books in our library (unless you're at NRC CISTI), so it's unclear how much public good we're providing. I do plan to write some reviews - perhaps we're a node in an EA, SOA etc. book club?
On June 13 2003 the Danish Ministry of Science, Technology and Innovation published a white paper* on governmentwide enterprise architecture. This whitepaper was the starting point for the succeeding initiatives in the Danish architecture for eGovernment programme which covers all tiers of government. The paper Architecture for e-Government* from sums up the major initiatives up until March 2004.
A National OIO Enterprise Architecture Committee has been established. A National OIO Enterprise Architecture Forum covering the same subjects is open for all interested parties within government as well as within the private sector.
Ressources in english are marked with *.
from http://www.oio.dk/arkitektur/eng
The Interoperability Framework is the Danish e-Government Interoperability Framework.
The framework includes recommendations and status assessments for 609 selected standards, specifications and technologies used in e-government solutions.
This is version 1.2.14 of the framework. If you know the Interoperability Framework already, see What's new?. Newcomers might want to check the survival guide. All users should check the advanced search services, and of course also read the (updated) guidelines for use. Above all, everyone should dig into the 609 standards in the categories below.
from http://standarder.oio.dk/English/
There is a blog in English and Danish - Interoperability Framework Blog. Here's the latest English posting, from February 13, 2006
Another standard has been added to the Framework. This time it is Role-Based Access Control (RBAC).
RBAC is a conceptual model, which uses employee assignments, functions, qualifications, responsibilities and authorizations as its starting point. RBAC is based on users, roles, role hierarchies, relations and limitations (e.g. separation of functions) and is a foundation for controlling who is able to do what, when, where, in what order etc.
As a part of the Danish government wide enterprise architecture a project focusing on a common approach to user and access management has been established. This project is among other things concerned with RBAC.
Previously:
April 30, 2006 Government of Ontario EA and SOA
April 30, 2006 Government of Canada EA and SOA
November 24, 2005 EA and SOA info overload, courtesy of John Gøtze
My colleague Stephen Anthony (contrary to popular belief, we are not the same person) has started an EA blog:
He's already made the Business-Driven Architect blogroll - from newbie to 1337 in record time.
To be an Enterprise Architect is to live in a bit of a temporal disconnect from the rest of the world.
This disconnect is two-fold:
1) Architecture, to my mind, must be looking at the continuously receeding 3-year time horizon.
That creates the critical bridge between the day-to-day and month-to-month activities of the organization, and the organization's stated 5-year strategic plan.
2) The people in your organization may not even be living in the present. They may have a technological viewpoint that lags several years behind.
So you're in the meeting room in 2006, and you're trying to have a conversation based on where you think things will be in 2009, meanwhile, the organization is still back in 2001.
That is a huge challenge.
In fact, I would say one of the biggest single challenges is trying to find ways to move people's thinking forward. And I don't have any solution. So far my recommendation is:
inform and be ready
Tell the organization that based on their goals and the current technology environment, they will need X, do the necessary models to support X and then... wait. Generally in about 9 months, after having completed their journey of discovery, the organization will come to you and say "we've just discovered, we need X".
Is this business-driven?
To me, as long as you are working from the strategic plan created by the business, you are business-driven. A key role of the architect is to be the Keeper of the Plan, and the Great Reminder.
It's not that you can't change the plan, it's just that that should be a conscious decision, not a function of drift. If you said your strategic goal for 2010 was to build a soaring stone cathedral, and you land in 2010 having built only a bunch of wooden houses and two sheds... to me that's the biggest single metric indicating success or failure of the EA.
Anyone else have ideas on key metrics to qualitatively or quantitatively assess the success of the EA?
Any thoughts on bridging the time warp between 3 and 5 year strategic visions and the day-to-day of project and operations execution? Better ideas for moving the organization's technology thinking forward, rather than just "inform and be ready"?
ComputerWorld NZ - March 13, 2006 - Technology gets SOA much better for Ontario government
In Ontario, the new SOA framework will reuse systems for common business processes and make the government's IT infrastructure more flexible to adapt to changes in government programs, and even changes in governments.
Ron Huxter, chief technology officer in Ontario's Ministry of Government Services, used an IBM Canada breakfast on SOA best practices in Toronto last month to outline the government's blueprint. He said one key driver for moving to SOA was the need to be more agile. "We have 200 to 300 different lines of business … of which one-third to one-half get whipped-out every four years."
While SOA may be new to Ontario, Huxter says the idea isn't. The government’s current initiative is a carry-on of what’s described as a "common component strategy" the province tried to implement six years ago. "The reason it wasn't very successful is because it was a technology effort and not a business effort."
That's a mistake they're not planning on making this time.
Huxter says the exercise is being driven as a business initiative, with IT as an equal partner at the table. A Common Components and Applications Services group, which Huxter says will be announced shortly, is responsible for identifying common components that can be reused across the government. "All government programs have unique outcomes but not unique processes."
The Government of Ontario also has an Enterprise Architecture initiative. I did some very quick searching but I couldn't find a specific page for their SOA.
The Government of Canada has an Enterprise Architecture initiative.
Recently as part of this, they have released two papers about Service-Oriented Architecture (SOA):
GC Service Oriented Architecture Strategy - Statement of Direction
This document introduces the Service Oriented approach and the GC SOA reference model that provides all departments and agencies with a statement of direction and orientation to help ensure a consistent adoption of SOA in support of a cohesive approach to service delivery across government.GC Service Oriented Architecture Strategy - Primer
This document provides a foundational overview of the GC SOA and introduces the basic concepts of Service Oriented design from the perspective of the Canadian Federal Government.
As you can see, they are still at an initial and very high-level stage.
Thanks to Steve Anthony for pointing out the Statement of Direction document.
SOA Web Services Journal has an interesting cover story: Focus on the "A" in SOA.
(If you cannot stand their busy, pop-up laden pages, here's the printer version.)
It's quite hard on existing enterprise architecture, but it emphasizes the critical importance of a functioning, relevant, forward-looking EA to support SOA.
The failure to focus on architecture during [the SOA development] process will ultimately dilute the architecture beyond its ability to have positive impact on the business. It is critical, especially with just such a gradual architectural rollout, that the architecture design exists as more than a static document. It must be a living entity that evolves in coordination with changing business needs. This makes the service-oriented enterprise architecture a perpetually moving target. SOA is about anticipating the future, which is as much about where architecture needs to be as it is about where it is today. That is all the more reason to enact measures to ensure that all concerned parties keep that target architecture in their sights at every step.
Balancing Act: Control and Chaos
Effective enterprise architecture requires a balance between control and chaos. It must be dynamic, interactive, and holistic. It must overcome project myopia, spanning space by connecting projects to each other, and spanning time by ensuring that projects are fully aware of what has already been built, what needs to be built now, and what will be built in the future.The funded project model for SOA deployment underscores the need for architects to confer on an ongoing basis with business managers in order to understand their priorities, model the architecture to support critical business needs, and clearly illustrate how technical assets and services directly affect the ability of business managers to do their jobs more effectively.
- Architects will have to translate their plans into the bottom-line language of time and money that is spoken by business managers with little technical expertise. Toward that end, dynamic graphical mapping of project and service dependencies will bring the business implications of IT projects vividly to life.
- Only a robust enterprise registry/repository that links the management of services, business processes, and other software assets with business objectives will provide the kind of long-term impact metrics that IT must supply to guide the business-side decisions that will shape the enterprise architecture (see Figure 1).
Creating the target architecture is only half the battle though. An architecture that no one implements is no architecture at all. Ensuring that developers adhere to the target architecture is absolutely critical to a successful SOA. In this effort, the model for application development in isolation is a recipe for disaster.
Bridging the Silos
Organizations that are adopting SOA must systematically bridge the gaps that separate development silos: those groups of programmers working on projects in almost total isolation, who often insist on building everything from scratch. In the past, the effect of this not-invented-here disease - a malady that afflicts so many corporate development groups - was limited to a severe swelling in the amount of time and money a company needed to spend on any given application. For an SOA, the consequences are life threatening: isolated silos are to SOA what kryptonite is to Superman.
It continues on to emphasize the importance of managing your XML Schema Definitions (XSDs), and concludes with a link to some best practices:
SOA Reality
As yesterday's monolithic IT infrastructure is deconstructed into a dynamic matrix of loosely coupled services, the realization of the dream of the agile, on-demand enterprise inches ever closer to reality. The most important factor in that transformation is that SOA exists within the larger enterprise architecture. The success of the SOA effort depends entirely on the manner in which the enterprise architecture is expressed, communicated, deployed, and governed (See Sidebar).
Some key best practices listed are:
...
As an interesting side note, the journal website has a nice blogging feature: click the blog button (on the regular article, not the print version) and you get text you can use, as well as a trackback URL.
Previously:
I put a big emphasis on the connection between EA and SOA in my presentation to the Manitoba SOA Symposium last year.
The video of the presentations from the SOA Symposium held last December is now available at
UPDATE 2006-03-17: The videos (RealVideo) are now at http://www.mandolin.ca/planning.php - scroll down to the Symposium section - the links may show up as a dark purple, but they are clickable.
Yes, that's me with my hair sticking up.
Also, I'm not sure why they did music that sounds like it's from the X-files, but then again, what is the right musical accompaniment for Service-Oriented Architecture?
Previously:
January 09, 2006 all 4 presentations from Winnipeg SOA Symposium
December 11, 2005 SOA challenges and Web 3.0
December 08, 2005 bookmarks for SOA Symposium
There is a magazine called Enterprise Architect from Fawcette Technical Publications (FTP). It appears to publish about twice a year.
They also have a yearly conference. This year Enterprise Architecture Summit 2006 - Strategies and Best Practices for the Real World - May 15-17, 2006 in Key Biscayne, Florida.
Here are presentations and videos from the previous two years:
Perhaps unsurprisingly, many of the presentations discuss the role of Service-Oriented Architecture within the EA, or other SOA-centric issues.
I am an Enterprise Architect, which can be a bit of a problem when communicating with people, since very few people know what EA is or what we do. Fortunately Brenda Michelson comes through with a detailed posting on
CIOs are determined to invest wisely: in strategies, technology, and people that create an environment for business and IT agility, while keeping project expenditures and run rates in check. Some of the strategies include (but aren’t limited to) service-oriented architecture (SOA), event-driven architecture (EDA), process-based architecture, federated information, enterprise integration, and open source adoption.
At the start of 2006, as IT folks are formulating their hiring plans and professional development plans, I wanted to highlight a critical role for 2006: the enterprise architect. That is, the enterprise architect who practices Business-Driven Architecture.
All four of the presentations about Service-Oriented Architecture are now available for download from http://www.umanitoba.ca/uts/ltc/soa/
I must say I was amazed that all four of us hit many of the same points, without any coordination between our talks (other than some slight communication I did with Thomas beforehand).
My presentation also touches briefly on Enterprise Architecture, because I believe you have to set SOA within an overall architecture context.
I don't know when or if the webcast will be available online.
UPDATE 2006-03-16: The webcast videos are now available, see posting video from Manitoba SOA Symposium.
