Well, I have officially had my 15 minutes of fame.  Back in June, I was invited to participate in a SOA Power Panel for SYS-CON in New York.  I’ve participated in a number of expert panels over the years, but this one was unique.  The discussion took place in the Reuter’s TV studio in Times Square.  The room had a professional camera crew, a sound stage, CNN-style set, and the windows behind the panel overlook Times Square (you can actually see some of the digital billboards in the background).  It was a fantastic experience and we had a really great discussion around the current state of Service Orientation and the future holds for enterprises both large and small with respect to technology innovation and successful SOA adoption.

You can find the videocast here: SOA Power Panel - Live from Times Square. There is a brief commercial and then it takes you right into the panel discussion.

I had a ton of fun participating in it and hopefully you’ll find value in viewing it.

I’ve just published a SOA book through Wiley Press and it is now available on Amazon (hence the catchy title for this post).  Woo-hoo!

The book was a ton of hardwork, but I am quite pleased with the result.  The aim was to address what I considered to be a significant gap in the marketplace — a pragamatic book on SOA that is more business oriented and less technology focused.  So I signed up Bibhas Bhattacharya as my co-author and set out to write such a book. There are vast amounts of resources available for techies that are interested in SOA.  There are even a fair number of resources available for managers.  If, however, you are a business leader, executive, or are actively trying to get the buy-in of one, where do you turn?  What resource is there to help you grapple with important questions like:

• What is SOA really about from a business standpoint?
• When is SOA applicable and when is it not?
• How should an organization go about adopting service orientation?

There are a couple of excerpts available for you to preview the content:

• Excerpt of Chapter 1 “SOA Primer”
• Excerpt of Chapter 3 “SOA Value Proposition”

When you get the opportunity to read through some or all of the book, I would really appreciate your feedback on the content.  I hope that many will find it to be a useful resource to facilitate pragmatic SOA adoption.

Up until now, there has been such a rush to roll out initial SOA services and supporting infrastructure, that little attention has been given to what good service design truly entails.  This is a natural evolution that must occur with any technology or methodology.  Throughout this summer, I plan on releasing several posts regarding proper service design.

Initially, I would like to focus on the subject of ‘composability’.  Composable services have several important qualities:

• composable services stictly adhere to their service interface, Service Level Agreements(SLAs), and supporting metadata
• service operations always leave enterprise data in a valid business state
• service operations can be called indepently, in parallel, or as a part of a larger service orchestration
• service operations can be called within or without a transaction context

When you boil it down to the essentials, composable services clearly define what they do and they do exactly what they have defined.  Consumers of composable services are then at liberty to determine how they want to use that service’s operations, rather than the service having certain built-in assumptions regarding how the service will be used or what the larger business context ought to be.

To learn more about the merits and details of composable services, I recommend the following resources:

• “Composable Services” (MSDN Library)
• “Service Composability” (SOAprinciples.com)
• “A Composable Service is a Good Service” (Presentation at SOAWorld 2008 next week)

There’s another, less tangible benefit of composable services — they are lower stress.  The frustration and stress associated with consuming a service that later fails due to some unknown buisiness logic violation is supremely frustrating.  Consuming services that are not composable is stressful, because they cannot be depended upon to work in whatever context you choose.  Likewise, publishing non-composable services for consumption is stressful because the service provider is constantly in fear that someone will ‘misuse’ a service.  This is no way to run an enterprise or encourage the adoption of service orientation. 

I’ll explore all of these topics next week in my presentation at SOAWorld on composable services.  I’ll post the completed presentation as well as perspectives from the audience next week.  Have a great weekend and I’ll see you in New York!

Primitive man and woman were forced to adapt to their environment to survive. Animal hides for clothes, crude weapons and tools made out of stone or bone, and roots or berries for food (perhaps sabertooth steak or terradactyl ribs if company is coming over). Eventually, some nearby tribe discovers fire, but reports of fire-related injuries and property destruction lead our cave dwellers to avoid this new magic. These primitive humans are doing fine with their current capabilities and they are able to meet their basic needs without dabbling with such things. Over time the nearby tribe’s use of fire expands and one very enterprising woman invents pit fired pottery. The more primitive people have heard and seen the potential of this new breakthrough and are interested in using it. Alas, the obstacles to adopting this innovation are not trivial. They must learn how to create fire, handle fire cautiously, locate and stockpile flint and tender, develop fireproof tools, create fire pits, learn to craft with clay and finally develop a process to fire the clay pots successfully. Lack of education, skills, infrastructure, resources, and processes hinder their ability to progress. Perhaps the greatest barrier for these primitive people is fear of the unknown and risk of failure. Thousands of years later, we are faced with the same technology adoption challenges — education, skills, infrastructure, resources, processes, and fear.

Modernizing legacy information systems is not unlike the modernization efforts that have occurred for thousands of years. In fact, legacy or heritage systems are often referred to as ‘dinosaurs’. Correspondingly, the same types of obstacles must be overcome:

• Education — There is a mindset shift that must take place to understand the service oriented way of thinking. At first glance, services are just fancy objects. The reality is that service orientation requires a much broader, end-to-end view of the enterprise, complete with process-centric alignment, layered architecture, contracted interfaces, standards-based connectivity, and full life cycle governance. I have blogged about the alignment of service orientation and the unique qualities of SOA before. Also, David Ing has some interesting thoughts on service orientation.
• Skills — A well documented skills gap exists around service orientation. Joe McKendrick sites it as one of the two things killing SOA in 2008 and the guys over at ZapThink have been alerting the industry to the dangerous SOA skills gap for quite some time. Why is this? Service orientation skills cannot be acquired by attending a conference or reading a book (although I do have a book that I HIGHLY recommend). Academic learning as well as hands-on mentoring is required. Moreover, there are nuances to effective service orientation, service design, process alignment, and enterprise governance that require time and experience to develop. There are new technologies, design patterns, techniques, and methodologies that must be introduced and ultimately absorbed. This will require instructor-led training, research, hands-on mentoring, and practical experience to develop proficiency in these areas.
• Infrastructure — Contrary to popular belief, a service oriented infrastructure cannot be achieved by purchasing an Enterprise Service Bus (ESB) from a vendor and adding water. There are, in fact, a variety of infrastructure elements (service registry/repository, governance suite, business process engine, policy manager, policy enforcer, and etc.). There is a great little SOA infrastructure white paper that a consortium of vendors put together a while back. Additionally, be sure to avoid the trap of assuming that you need to have your SOA infrastructure fully mature on day one. Eric Newcomer has a great post from a couple years back regarding an incremental adoption of SOA infrastructure.
• Resources – There are several ways to slice the resource issue. For effective adoption of SOA, you will need a pool of human resources (with appropriate education and skills), technical resources (infrastructure, knowledge management and collaboration tools, as well as design and development tools), and you will need expert resources (developed in-house, or brought in from an outside source initially).
• Processes — All the education, skills, infrastructure, and resources in the world won’t amount to a hill of Java beans without effective processes for governing the adoption of service orientation. You will need processes for service selection, service design, quality assurance and testing, policy enforcement, and runtime service management. Effective governance can make or break your adoption of SOA. I have blogged on the importance of service oriented governance numerous times.
• Fear – People fear change. Service orientation appears threatening to many people at first due to the changes required in adopting it. New patterns of thinking are required around how to solve customer problems. Skills must be adapted, other skills developed from scratch. New infrastructure and resource pools must be created and processes must be adapted and new processes created. These changes can be intimidating. A considerable degree of people change management must be accepted as a part of a successful transition toward service orientation.

Service orientation adoption is challenging in its own right. It becomes increasingly challenging when you factor in the dinosaurs that must be modernized as a part of the adoption process. Immediately, questions begin to arise:

• What systems and/or business processes should be modernized?
• Should we migrate, expose, or leverage our existing legacy assets?
• Should adoption occur within a particular line of business or across the whole of the enterprise?
• How will modernization impact existing teams and project roles?

These questions and many more must be explored as part of the adoption of service orientation. I explored several of these topics in a recent public webinar - Enterprise Modernization and SOA Concepts (pdf).

Another resource you might consider is a new course that Web Age has been offering since April of this year. It is a two-day workshop that explores the modernization of legacy applications, SOA concepts, modernization challenges and risks, and various strategies for modernizing legacy systems. It is aimed at team leads, architects, managers, and legacy application support personnel. For more details, check out the course details - WA1657 Application Modernization and SOA Concepts.

Adopting service orientation and “moving out of the Stone Age” is not a simple task. It requires intentional allocation of time, energy, and resources. It cannot and should not be approached in an ad-hoc fashion. Technology adoption has always been challenging, just like when ancient man and woman adopted fire. When it comes to service orientation, be sure that you work through the necessary steps of adopting education, skills, infrastructure, resources, processes, and addressing the fear factor. Otherwise, you just might get burned.

I’ve been on the road a lot lately, working with various clients, writing courseware, writing articles, and basically doing everything EXCEPT for blogging.

Here’s some of what I’ve been up to:

• Article: SOA Governance: Start Small and Build Incrementally (published March 6th by SYS-CON)
  
• Presentation: SOA Adoption Planning (1-hour seminar I delivered in February)
• Presentation: Delivered a virtual presentation for the Dallas SOA Users Group. I had a client engagement in Calgary, so we tried out a virtual delivery and it was quite successful.
  
• Book: Service Oriented Architecture: A Field Guide for Executives (August 2008, Wiley Press)
• Courses: Updates to various courses plus new courses coming soon for Project Managers, Mainframe and Legacy Personnel, and some Best Practices content. For more information, check out Web Age’s SOA curriculum.
• Web-based SOA Training: I’ve been working on a new offering at Web Age in which some of our content is made available as recorded, flash-based audio/visual presentations that can be hosted internally within an organization’s intranet. More details to follow.

I’m going to try and be more diligent about blogging in April. I especially hope to share insights and perspectives that I glean from the IBM SOA IMPACT conference in Vegas. I’ll be attending that conference in early April and conducting a Birds of a Feather session on governance.

One semester in high school, I joined a show choir. After two months, I decided that I had endured all of the step-chasse, step-ball-change, and “jazz hands” that I could stand! I don’t remember any of it now, but I do remember the importance of choreography and coordination among the various participants. What, pray tell, does this have to do with SOA? I’m glad you asked.

Like a well choreographed dance, successful service oriented solutions clearly define roles, responsibilities, and coordinated interaction. Some call it workflow, others choreography, others orchestration, still others talk in more general terms of coordination. There was a time when I thought all of this was a just a semantic game and there was no real difference. While a part of me still questions whether or not we REALLY need all of these different terms, concepts, and standards for the basic idea of ‘making sure everyone and everything does what she/he/it is supposed to’, I am beginning to appreciate the differences among these various concepts.

Orchestration (Executable Business Process)

As organizations mature in their adoption of SOA, they tend to migrate away from a focus upon individual services and toward a focus upon business processes. Orchestration enables an enterprise to fulfill a business process by pulling together a set of service operation calls along with workflow-style business logic and some process-specific variables to maintain persistent state throughout the process. Principally, this is achieved through Business Process Execution Language (BPEL) and a corresponding process engine. BPEL is championed by the likes of Microsoft, IBM, and Oracle (to name a few) and is maintained and actively developed by OASIS.

Choreography (Multi-party Collaboration)

Moving beyond the internal workings of a given organization’s services and orchestrated business processes, choreography is concerned with facilitating the interactions that occur between enterprises. Choreography relates to describing collaboration interfaces between parties. The dominant standard in this space is the W3C’s Web Service Choreography (WS-CDL), a language designed to describe multi-party contracts.

Coordination (Multi-party Context Sharing)

The term ‘coordination’ within the SOA / WS space is far less well understood than either orchestration or choreography. Much of the literature and vendor documentation out there will describe choreography and coordination as being synonymous. This is not quite accurate. To make matters more confusing, the W3C has a Web Services Coordination group. Their job is to coordinate all of the others groups in the WS domain. So what does ‘coordination’ actually mean in the SOA / WS context? It’s really all about sharing context between WS entities, typically to support the prorogation of transaction semantics. OASIS has done a tremendous amount of work in this area under the Web Services Composite Applications Framework (WS-CAF) umbrella. That framework consists of the Web Service Context (WS-CTX), Web Service Coordination Framework (WS-CF), and Web Services Transaction Management (WS-TXM) standards.

Orchestration vs Choreography vs Coordination

So what’s the story here? Are these various workflow-type concepts and corresponding technology standards truly distinct elements that merit unique treatment, or just an example of over-engineering? Although my heart longs for simplicity and clarity in this arena, I do think that there is some value in breaking these notions apart. This is especially true for enterprises that engage in complex, service oriented B2B interactions (typically financial and insurance companies).

Orchestration expresses organization-specific business logic. The entire flow is owned and governed by the organization, even if the flow requires interaction with third parties. Choreography is more collaborative and peer-to-peer in nature, with no central organizing entity. Coordination is a supporting player that can be used in concert with an orchestration or choreography.

So when do you choose one over the other?

• Use orchestration to automate and service orient individual, executable business processes
• Use choreography to describe contractual service interactions between various parties
• Use coordination to define the hand-off of application and business context between various parties

If you’d like to read more on the subject, I recommend the following:

• “Service Orchestration vs Service Choreography”, by John Reynolds
• “Orchestration vs Choreography”, from The ServerSide.com
• “Choreography vs Orchestration” (Orchestra / Dancer Analogy), by Stefan Tilkov
• “Choreography vs Orchestration” (Traffic Light Analogy), by Paul Downey
• “Orchestration, Choreography, Choreostration”, by Jim Alateras

Any thoughts, questions, or war stories to share regarding orchestration, choreography, or similar concepts in SOA?

I spent much of the second day talking with attendees, speakers, and SYS-CON staff about SOA, virtualization, and the conference in general. Overall, there was a general sentiment that the conference was a success and had some valuable content. One presentation from the day that really stood out was a case study from the banking industry — Delivering Big Bank Solutions with Community Banking Intimacy. The bank is Synovus, a regional bank in the southeastern part of the US with about 33 billion in assets spread across multiple lines of business (banking, financial management, brokerage, insurance, and more).

In a thick southern, ‘good ole boy’ style delivery, the speakers from Synovus described their approach as ‘Blue Collar SOA’, explaining: “…it may not be perty, but it works and we get a lot of value out of it.” The Synovus representatives went on to explain that while they did not set out to deploy a SOA solution, their business requirements and technology objectives led them in that direction. They went on to describe several keys to successful SOA adoption, including the following:

• Define and document a set of guiding principles
• Define and document a reference architecture that is consistent with those principles
• Identify standards, technologies, and products that can enable that architecture and communicate and manage this through governance
• Identify a business visible pilot project to convert the strategy and reference architecture into a real implementation
• Define and enforce the implementation strategy through governance processes
• Employ governance toolsets to validate services against guiding principles as well as policies and SLAs

The results of their SOA adoption as well as the importance of service orchestration is outlined in a white paper from Active Endpoints (their service orchestration partner).

The presenters concluded by identifying a huge list of lessons learned. Three things really stood out to me on that list:

1. Identify and document standards and best practices and enforce them through your governance processes
2. SOA changes your team dynamics — you will need more documenters than developers
3. Education is key because SOA is hard

That third one was especially of interest to me given Web Age’s focus upon SOA education. In the Q&A portion I asked for the folks from Synovus to elaborate on the education topic. They were eager to explain that they had discovered a huge gap between legacy skill sets and what it takes to be successful with SOA. They had to figure out how to translate those skills, bring people up to speed on new methodologies and terminologies and get people to think differently about how to solve problems. Synovus also discovered that considerable education and mentoring was necessary to help the leadership that direct their lines of business to “think outside their silo”. Amen!

At Web Age we have seen the same sorts of challenges and often discover that teaching the technologies is fairly simple. Teaching the methodologies is more difficult. Teaching the new mindset (e.g. ‘thinking outside the silo’) and providing business users with new tools and techniques for solving problems is even more difficult. So what is the most challenging? Getting organizations to actually change once they are educated. The technology side of SOA is relatively easy. The human side is where things get tricky.

Overall, the conference was a great experience. If you get a chance to catch the East coast or West coast shows next year, definitely go. I hope to see you there!

The first day of SOA World 2007 - West went very well. Miko Matsumura with Software AG / webMethods kicked things off with the keynote - Time Oriented Architecture: Evolution by Design? And he had some really entertaining 3-D animation and virtual simulations in his presentation. It was pretty cool. The next presentation in the main room was by Theo Beack from BEA - Virtualized SOA: Adaptive Infrastructure for Demanding Applications.

I delivered the third presentation of the day in the main room - A Little SOA Goes A Long Way, discussing the importance of identifying which aspects of your enterprise are ripe for service orientation and which areas are better left alone. The outline for that presentation is as follows:

• Introduction
• Why SOA initiatives fail
• What my kids taught me about SOA
• Adopting SOA selectively
• Bowling for governance
• Review

This presentation seemed to be well-received and I was particularly pleased with the broad acceptance of the Selective SOA Methodology that I presented. This methodology serves as the cornerstone for much of the SOA Adoption and SOA Governance mentoring that Web Age Solutions provides to clients. If you’d like a copy of the presentation, you can download it here — A Little SOA Goes A Long Way (PDF).

After a trip to the Expo Floor and some snacks, we were back in the main room to hear from Mike Pellegrini from Active Endpoints presenting - Your SOA Needs BPEL For Orchestration. He had some great content, but one nugget really stood out to me early in the presentation. Mike was describing services and processes as two kinds of abstractions that are important in SOA. After discussing them separately, he offered the following synthesis: “Services don’t change often, but they are orchestrated and re-orchestrated fairly often to build/modify business processes.” I like that. I would qualify it to say that services SHOULDN’T change often. In other words, I believe that this is indicative of an enterprise that has reached a considerable degree of maturity in their service orientation. Nonetheless, I thought it was a really solid characterization of services and processes.

Following Mike’s presentation, the speaker that was slated to speak in the main room could not be located. After hunting for about ten minutes, SYS-CON got desperate and accepted my offer to deliver another presentation that I had on my laptop. So I hopped up on stage, grabbed a mic, and fired up my laptop with the same presentation that I had delivered at SOAWorld in June early this year in New York - Service Oriented Patterns and Anti-Patterns. Fortunately, I had delivered that presentation about a month ago for a users group in Dallas, so I wasn’t completely unfamiliar with the material. I was a little rushed for time due to the late start, but overall that presentation went rather well.

After a lunch break, Ian Thain of Sybase presented - Model-driven SOA. His presentation included some interesting demonstrations of model-driven SOA using Sybase tools. Next was another Expo Floor break and snacks, followed by the SOA Power Panel.

I had the pleasure of participating on the SOA Power Panel along with Miko Matsumura (Software AG / webMethods), Kevin Hakman (TIBCO), and Sandy Zylka (NextAxiom). Jeremy Geelan moderated and we had some great discussion around SOA, convergence with other trends, economic impacts, and more. The panel session was recorded and I will post a link to it once it is made available.

I don’t know what happened for the rest of the day as I was occupied by various discussions in the late afternoon and then in a bit of sight-seeing in the evening (Coit Tower, Treasure Island, Lombard Street, and more).

Recently, I’ve noticed a theme among a couple of new clients as well as some people that I met at a conference. Governance? Oh sure, we’ll fold that in at some point. I’ve seen this expressed in two ways:

1. Governance is fine, but should only be implemented once some services, a registry, and a few other pieces are in place
2. Governance is a part of the project management life cycle but only once the services have been deployed

I believe that both of these perspectives on governance are flawed. I’ll address each one in turn.

Flawed perspective #1: Governance isn’t needed until after our SOA is up and running

At first glance, this thought process is rather logical. You really need some services (15+) and infrastructure before governance becomes an issue, right? The trouble is that ungoverned services require refactoring once governance is put in place. Additionally, personnel tend to be frustrated when structure is introduced later. They resist the idea that governance is needed today when it wasn’t needed yesterday. The counter to this line of thinking is that instituting governance too early will crush a fledgling service orientation initiative before it gains any traction. The flaw in this line of reasoning is to assume that governance has to be fully-mature at its inception. Governance does not, and in fact should not, be implemented all at once any more than your SOA infrastructure should be deployed in one fell swoop. Experience shows that change should be rolled out incrementally, delivering more mature capabilities gradually within the enterprise.

Flawed perspective #2: Governance isn’t needed until services are deployed
Although I’ve seen this expressed in a variety of ways, no one presented it more clearly than one client I worked with in August when they showed me their SDLC: Design > Build > Deploy > Govern They had successfully avoided the first flawed perspective by recognizing the importance of including governance from the very start. At the same time, however, they fell into another trap in relegating governance to more of an administrative function at the end of the project life cycle. Governance should be a consistent thread throughout the entire project life cycle. Enforcing policies, applying best practices, and ensuring that services and their related artifacts are properly documented begins with the service design phase and continues until service retirement.

Alternative perspective: Lean Governance
Lean governance represents a mindset in which governance is applied as needed. Implement only as much governance as is needed and constantly monitor the environment in order to tweak the degree of guidance and oversight. Governance shouldn’t be implemented as one or more powerless committees, nor as heavyweight bureaucracy. Governance should be focused, lean, and ever-present. Early in the adoption of SOA, governance should be minimal. It could be as simple as a requirement to be WS-I compliant and maintain service contracts for each service. Over time, the governance policies, processes, and procedures, as well as the corresponding infrastructure can be grown incrementally along-side the maturing of the service oriented enterprise. Finally, this governance should be applied throughout the project life cycle. Best practices identify three governance gates - design-time, change-time, and run-time. These three gates serve as check points to ensure that service design, development, and run-time behavior are consistent with enterprise goals and stated best practices.

In my experience, governance is crucial to any significant organizational change and service orientation is no different. I don’t believe that governance has to be at the extremes (powerless committees vs heavyweight bureaucracy), but instead can and should be focused, lean, and ever-present. Initially you don’t need very much governance, but it is much easier to start with an expectation that governance will be present and grow it incrementally than it is to add later down the line. Governance should be planned from the start and it should provide context for the entire project and program life cycle.

It’s been over a month since I’ve posted anything. For the past several weeks I’ve been heavily engaged with clients and several public speaking engagements related to Service Oriented Architecture (SOA), Business Process Management (BPM) and Enterprise Architecture (EA). A resounding theme as of late has been the importance of aligning the business and technology teams along shared enterprise goals. Although architecture and service design patterns are important, more and more enterprises are interested in the cross-functional synergy that SOA enables. In order to truly deliver upon the promise of agility that service orientation offers, it is critical that business and technology personnel work together.

So what exactly does this enterprise alignment look like?

• Cross-functional, enterprise teams must champion the adoption of SOA. Change agents and visionary champions need to understand and clearly communicate the SOA strategy to all levels of the organization. Technology-minded individuals and business-minded individuals need to be involved. This should also be a diverse, enterprise-wide initiative that has input from all lines of business. Some groups choose to establish or leverage an existing Center of Excellence, Competency Center, or similar objective group. Regardless of how formal or informal the team may be, the keys to success are passion, communication, and diversity.
• The Information Technology group must be treated as a strategic partner, not a cost of doing business. IT provides far more value than simply keeping an organization’s servers up and running. They are a key enabler of information and automated systems that allow humans to focus more on analytics and less on low-level details.
• Organizational transparency must be prioritized. The visibility of IT’s business value and internal capabilities must be raised to the business community (minus the technical jargon and useless acronyms). The business capabilities that IT has today and can reasonably make available in the future need to be clearly communicated to the business teams. Visibility is a two-way street though. The business teams need to clearly convey business drivers and relevant metrics so that the technology teams understand where the organization is headed.
• A process-centric approach to business. Business Analysts and Business Engineering teams are gaining increasing credibility within the modern enterprise. In order to truly bridge the gab between business and technology, it is essential for a common frame of reference to exist. Business process models offer a visual mechanism for both business and technology personnel to discuss, explore, and solve business problems without getting caught up in the details. Afterward, each team is then able to translate this into their world. The technology teams, in particular, have a lot of options available to them thanks to standards like Business Process Execution Language (BPEL) and Business Process Modeling Notation (BPMN). On the business-side of things, it is important that business processes become a pervasive element within the enterprise. Business processes should drive budget allocation, project management frameworks, and even human resource strategies.

Service Oriented Architecture has really grown to become an umbrella that encompasses an entire range of enterprise initiatives (service architecture, BPM, EA, governance, etc.). Rather than getting caught up in the technical ‘architecture’ aspect of SOA, I prefer to talk with clients in terms of business and technology alignment along service-oriented business processes. Service orientation is really a people problem more so than it is a technology problem.

As I explore this alignment concept further with clients, I will post some more on this subject. I might even distill my experiences into a white paper later this year. I’ve got a pretty full schedule this summer. If nothing else, I’ll be back in late June / early July to comment on SOAWorld 2007 in NYC.