As is true of service-oriented architecture (SOA) itself, the SOA lifecycle is a wide-ranging topic and even experts...
differ in their definition of it and the areas they believe need to be emphasized. This article provides an overview of the SOA lifecycle, how it differs from the traditional application development lifecycle and how it impacts the tasks an SOA architect is asked to perform. Views of analysts who specialize in SOA are compiled to provide a wide range of informed opinion on the topic.
"This is a boil the ocean subject," said Tony Baer, principal analyst with onStrategies, who offered a quick summary of what the SOA lifecycle entails. "From the hundred thousand foot level, you try to establish a need, you gather requirements, you try to architect a solution and then you develop and you test and then you deploy and manage it. At the end of life, you either modify it or retire it."
What differentiates the SOA lifecycle from the traditional application lifecycle, he added, was the need for governance to maintain order with loosely coupled services.
"The SOA lifecycle is actually pretty complex," agreed Anne Thomas Manes, research director, Burton Group Inc. "You can talk about the lifecycle of a service and it goes through the initial stages of identification and commitment to provide a service in the first place."
Manes' first step in the SOA lifecycle differs from most of the other steps offered. She starts with upfront planning and getting a budget to create services and compose applications from them.
"When you're looking at SOA projects that's where it starts," she said. "You need to decide what needs to be a service. An awful lot of organizations don't go through that scrutiny of identifying where they actually should be building services. Today, a lot of people aren't doing that upfront planning. They are basically saying: 'We've got this cool new technology. We're going to build everything as services.' They don't really go through the process of determining what should be a service and where they should be allocating their efforts. Making the determination of what should be a service is a very important aspect of the lifecycle."
Neil Ward-Dutton, research director, Macehiter Ward-Dutton, takes a big picture view of the SOA lifecycle starting with understanding the context for the SOA application.
"In our view the SOA lifecycle stretches all the way from understanding context, including enterprise architecture and business architecture, through service analysis and modeling. And that modeling isn't just about functional issues, it's also about non-functional issues such as security, performance, auditing and so on. Then there is development, testing, provisioning, monitoring and change management. Even if you're limiting your thinking to the app dev perspective on SOA, this places a set of additional considerations into the picture which probably weren't there before. For example, you need to think about consistent approaches to building business services so that non-functional capabilities are delegated to consistent runtime infrastructure services."
Jason Bloomberg, senior analyst with ZapThink LLC., said the main difference between the SOA lifecycle and the traditional application lifecycle is found in the services.
"The challenge with the SOA lifecycle is that there are essentially two different lifecycles going on, that overlap," Bloomberg said. "On the one hand you still have the traditional software lifecycle where services are interfaces to running software. You have to implement software and that's written in code. All of that is the traditional software lifecycle. But then you also have the service lifecycle and that takes place at the metadata level. As you're updating services, as you're composing services, as you're reconfiguring services that takes place at a different level because it doesn't involve new coding unless you drop into the software level, but the goal is to do all the changing at the metadata level."
Service-orientation thus requires a different lifecycle that focuses on metadata and the contents of the registry/repository.
"One of the key focuses of the architectural design part of the SOA story is building and maintaining the service abstraction," Bloomberg said. "On the one hand, services are interfaces to software and you have to implement the software. On the other hand, services are abstract representations of business capabilities that the business can compose into processes. So you have to be able to support that abstraction in software as well as support it as metadata for the business to give the business the agility that is one of the main goals of SOA."
Basic SOA Lifecycle
An eight-step outline of the basic SOA lifecycle for an application built with services is provided by Bradley F. Shimmin, principal analyst for application infrastructure at Current Analysis LLC. It differs in some aspects from the steps other analysts, but not radically, so it provides a starting point.
Eight steps for building an SOA with services
- Data collection including gathering of business requirements and use cases
- Design including determining service requirements, setting service policies, establishing compliance tasks, building and testing models, and constructing data integration
- Development including developing the service and composing the application from the services
Discussing this outline, Shimmin said, "I look at it as steps on a ladder toward something that actually exists in the world. You have to start somewhere and end somewhere. The lifecycle starts with the business analyst collecting information from the business on what the requirements are for the application or service."
It depends on the company, Shimmin said, but the business analyst will typically create a requirements document that is usually filled with use cases. The use case can be as simple as a user needs to check bank balance. The business analyst will provide detail on what that would exactly entail. This has nothing to do with the final application, it is just about what the requirements are with very specific use cases.
As the collection phase progresses, Shimmin said the business analyst brings the requirements and use cases to IT and systems architects. They work together to determine how the requirements will be translated into final software. At this point there is a differentiation between the traditional application lifecycle and the SOA lifecycle.
"When you are talking about a traditional software development lifecycle, you have the development lifecycle for software, you're building out an application at the end of the day and there's usually one project per application," Shimmin explained. "With SOA, you're talking about two different types of applications you're building. The first one you're building is the service that is a completely self-contained unit that does a specific thing such as 'return value of bank account.' So there's one lifecycle for that service and another lifecycle that applies to compositing that and other services into a final application."
This outline of the SOA lifecycle assumes that services will be built, Shimmin noted. It is not assuming that the services already exist and are available for reuse, which will be covered later.
The basic lifecycle steps apply to both services and the applications being composed from them, but usually these will be two different lifecycle processes encompassing the tasks that are under the SOA architecture umbrella. The initial data collection phase of developing requirements usually does not concern itself with distinctions between what will be a service and what will be the composed application.
But prior to the design phase, the architects will begin to mark out which of the use cases, such as check bank balance, will be services, and what business processes will be encompassed in the SOA application.
At the design phase the business analyst is going to sit down with IT and systems architects and based on the services that were identified, they will sort out what the requirements are going to entail in terms of service policies that are going to be in place and what will need to be adhered to in terms of compliance, including service level agreements (SLA) and public key infrastructure (PKI).
"Some of the PKIs and SLAs can be done ahead of time in the collection phase," Shimmin said. "But from what I've seen most of it gets done when they actually have something more tangible in terms of what the software is going to look like."
The first two phases produce artifacts including use cases, models from the design phase, as well as SLAs and PKIs. This then goes to the developers, who construct the services following the map they have been given.
The development and QA/testing phase will be familiar from the traditional software development lifecycle.
"Development builds the services," Shimmin explained. "They take them to QA in phase four to make sure that the SLAs will be met by the architecture that's been put in place. You also do acceptance testing with the business at this point to make sure the business users' expectations are being met."
From QA and acceptance, the project moves to the deployment phase, which is sometimes called the "publish" phase, Shimmin noted, because that is where the services are published into a repository and registry. The artifacts all go into the repository.
"At that point you move to what I call a nested software lifecycle imbedded in this step," he said. "Because what you are doing at this point is compositing that application, so you may be dragging together the different services that have already been created compositing them into the final application. It may get rolled out as a BPEL or Java WAR [Web ARchive] file that goes up on the server. To compose that application you have to go through those first stages. You have to design it. You compose it as opposed to develop it, so that's a difference. Then you have to go through QA/Test/Acceptance phase. Then you deploy the application."
In phase six, monitoring and management, SOA runtime governance tools are used. There are also design time governance tools that are used in the earlier stages to assure that what is being developed is in line with policies. Monitoring and management makes sure SLAs are being met and PKIs are in line, and that compliance requirements are being met. If problems are encountered it may be necessary to loop back to the design phase for the service to make modifications to a service to put it back into compliance.
Randy Heffner, vice president at Forrester Research Inc. views SOA governance and the SOA lifecycle as being intertwined.
"Lifecycle is the things that deliver your services and you need good governance across the lifecycle," Heffner said. "Governance is an overlay on a lot of things you're doing."
At this point, the lifecycle may loop back to earlier phases, Shimmin explained. For example, in the change phase, it may be necessary to loop back to the collection or design phase to begin work on changes to the services.
"If you need to add capabilities to the service," Shimmin said, "you jump all the way back to number one, collection, and start through the cycle. If you have a bug or a problem, you jump to phase three, which is the development phase. If it's a problem in the logic you jump to number two, the design phase."
Whether the project goes back to phase one, two or three, it will also have to go through the QA and deployment phases again as well. In this way, the SOA lifecycle tends to be an endless loop.
Retirement is the final phase in Shimmin's outline of the SOA lifecycle. Where in the traditional siloed approach to application management, it was possible to simply take an application off line, it is not that easy with a service in an SOA environment because within the enterprise that service may be in use in ways the original developers and architects know nothing about. So before retiring a service, its interdependencies must be checked. If a service is being replaced by a newer version, checking must be done to be sure it will still work for SOA applications that depend on the original version.
SOA service portfolio management
Focusing on SOA services where reuse is a factor, Heffner at Forrester sees three primary components at the top level of the SOA service lifecyle:
- Service consumption
- Service creation
- Service portfolio management
"Everybody will recognize service consumption and service creation as two sub-lifecycles," Heffner said. "Hopefully you are starting out by using things, but you'll have to create things as well. But those should both be done within the context of service portfolio management where we are trying to manage toward a longer term portfolio to make sure we have the right set of services and that they are being used when and where and how they should, and when we're creating things, we're doing it as part of a coherent portfolio."
It might seem that service creation should come before service consumption, but Heffner is focused on service reuse where SOA is already in place. In that situation, the default assumption for SOA application development would be reuse of existing services. However, he acknowledges that it may be determined that the service the application needs has not been developed. That is when you move to step two, service creation.
"Focusing first on consumption puts you in the proper mind set for doing SOA because that should be the default approach," Heffner said. "It puts you in the mindset of first achieving reuse."
In a hierarchy, Heffner believes service portfolio management would be over both service consumption and service creation.
"If you are doing service portfolio management right in relation to the other two then you change the model of how you accomplish reuse in a way that the industry does not yet talk much about," Heffner said. "When the industry talks about reuse it focuses predominantly on discovery, but if you think about it this is a haphazard Christopher Columbus kind of thing."
The discovery approach is the equivalent of saying: "We're going to start a new project, let's go discover a new world of services." That, Heffner said, is an ad hoc approach to SOA development.
"That's what you would have if you just had service consumption and creation," he said.
With service portfolio management, you've got a structured set of major business services organized by business process domain.
"Then when you start a new project, you're not saying let's go off and see what we can find," Heffner explained. "You say this project is in this domain. We have a portfolio there. How should this portfolio come to bear on the project? So reuse is planned into the project. We've got a portfolio. We are doing these major business units of work. We've got people that understand and know this domain. We can validate that we're using services the right way. So it's not a discovery model. It's a planning model. What are we going to consume in this project out of our portfolio?"
It is only in the case where a needed service isn't in the portfolio that the lifecycle moves to creating services.
The SOA service consumer lifecycle
One of the things that makes the SOA lifecycle complex in Manes' view is that it has to deal with two sides, service creation and service consumption. The lifecycle on the consumption side is different from creation, but it is still the responsibility of the SOA architect.
The basic assumption of any service creation lifecycle is that there is at least one consumer for the service being built, but other consumers may later seek to incorporate an existing service into a new application.
Manes identifies three steps in the consumer side of the SOA service lifecycle:
- Retrieving metadata from the repository
- Provision of the service for a new application
- Performance testing and capacity planning
The first step for a potential consumer is to retrieve the metadata from the repository.
"Based on the contracts that are defined with the contracts, they can implement the appropriate connections within the application that is going to use the service," she said.
But before that service consumer can be deployed it has to be properly provisioned. The consumer has to register in the policy management system that this consumer is permitted to access this service. It also includes the required authentication mechanisms for using the service.
The third step is to do performance testing and capacity planning to make sure the service can support the application requirements. That may include negotiating SLAs for that service.
The service provider needs to know the service level loads to anticipate and prepare for, Manes said.
Shortening the SOA Lifecycle
While much of the above has to do with the complexity of the SOA lifecycle, Dana Gardner, principal analyst of Interarbor Solutions LLC., advocates keeping it as short as possible in terms of timeframe in order to maintain the dynamic quality of SOA. He said that should be a major difference between the SOA lifecycle and traditional application lifecycles.
"Your services are going to be dynamic, so you're going to want to have a fairly frequent refresh or replacement of specific services based on requirements," Gardner explained. "At a higher level of abstraction, you're going to have changing business processes that those services support because you're going to want to adjust and amend on a fairly rapid basis for efficiencies. You're going to want to give more people who are close to that business process some say in how that process can be amended and improved iteratively over time."
How the SOA architect's job is different
The SOA lifecycle changes the role of the SOA architect as compared to more traditional enterprise architect, analysts say.
"It's almost a philosophical difference," said Gardner. "An SOA architect should be constantly looking for higher productivity. He should take the role of a town planner rather than a builder of buildings. He should take into account all the moving parts that impact a business' ability to get work done and to enter new markets and produce new products. So the architect in the SOA philosophy is always looking for constant productivity improvement."
In fact, architects can be viewed as the linchpin for SOA implementation.
"The SOA lifecycle impacts architects in some interesting ways," said Ward-Dutton. "Firstly, any architects that were historically just 'designers with a corner office,' and to be honest there are quite a few of them, if they are to step up to the plate with SOA, have to take a much broader view and take on responsibility for coordinating the needs and expectations of service designers and developers, infrastructure managers and systems admin staff. Secondly, architects also need to bridge the gap that can easily spring up in SOA efforts between the service provider community and the consumer community. These are tough challenges because they're not about technical skills, or at least not only about technical skills. Stepping up to the challenges requires architects to be real diplomats and develop negotiation and influencing skills."