News Stay informed about the latest enterprise technology news and product updates.

Web Services Reliable Messaging

Today Web services are inherently unreliable and are not well-suited for conversational-type processes that may run over several days. But unless Web services messages can be sent reliably they will never be adopted for mission-critical, more complex business processes. This article from webMethods takes a detailed look at the emerging WS-Reliability standards and how they build on previous reliable messaging frameworks such as ebXML, BizTalk, and RosettaNet.


1. Introduction

2. Need for Transport Independence

3. Requirements for Reliable Messaging

4. Reliable Messaging Not Unique to Web services

4.1 Reliable Messaging in RosettaNet
4.2 Reliable Messaging in BizTalk Framework 2.0
4.3 ebXML 2.0 Reliable Messaging Scheme

5. Web services Reliable Messaging Specifications

5.1 WS-Reliability
5.2 WS-ReliableMessaging

6. Summary

7. References

1. Introduction

Web services hold the promise of driving down the cost and complexity of application integration both within an enterprise and across the enterprise boundaries. As Web services are deployed in increasing numbers in complex business application integration and collaborate business process scenarios, the reliability of Web services becomes an important factor. Reliability of Web services is impacted by several factors including but not limited to, the reliability of the Web service end-points; reliability characteristics of the communication channel over which the Web services are accessed; performance and fault-tolerance characteristics and the extent to which Web services can handle concurrent client access.

One of the main characteristics of Web services is communication over unreliable communication channels such as the Internet employing unreliable1 data transfer protocols such as HTTP, SMTP and FTP. These are considered unreliable, as they do not offer the reliable messaging services such as guaranteed delivery. But, unless the message exchanges of Web services are made reliable, organizations will not be able to trust them for deployment in industrial-strength business applications and for mission-critical operations such as complex business-to-business transactions or real-time enterprise integrations. Accordingly reliable messaging becomes one of the first problems that need to be addressed for Web services to become truly enterprise-capable technologies.

.As we see later, a number of issues that need to be accounted for come to light as one attempts to devise a reliable messaging solution for Web services. This paper discusses emerging Web services reliable messaging standards and the work done in this space previously by RosettaNet, BizTalk, and ebXML that forms the basis for the advancements in the Web services domain.

webMethods has significant experience in delivering reliable messaging solutions via the implementation of RosettaNet Implementation Framework versions 1.1 and 2.0 and the ebXML messaging service 1.0 and 2.0 specifications. webMethods has co-sponsored the submission of the WS-Reliability specification to OASIS and is driving this effort forward via active involvement in the WS-RM Technical Committee.

2. Need for Transport Independence

There have been attempts at accomplishing reliable messaging of Web services by choosing a reliable transport protocol, over which the SOAP messages are exchanged between the Web services. For example a reliable messaging transport such as message-queues can be used to deliver messages reliably between senders and receivers. Message queuing communication technologies enable applications on different systems to communicate with each other, even if systems and networks occasionally fail, by enabling applications to communicate with each other through messages. Applications send messages to queues and read messages from queues that are persisted across failures for reliability.

There have also been attempts at defining reliable transfer layers over fundamentally unreliable transport protocols. Reliable HTTP or HTTPR is an example of such a protocol. HTTPR defines a reliable messaging protocol layer over HTTP 1.1. HTTPR defines how metadata and application messages are encapsulated within the payload of HTTP requests and responses. HTTPR defines protocol rules, which make it possible to ensure that each message is delivered to its destination application exactly once or is reliably reported as undelivered.

A common problem that plagues both of these approaches is that reliable messaging can be achieved *only* if the specific reliable transport protocol is used for communication between the sender and the receiver. The fundamental nature of Web services calls for independence from specific vendor platform, implementation languages and a specific transport protocol. In a generic case a sender may not be able to transmit a message directly to a receiver using a particular protocol (e.g. receiver does not support the protocol) or the message may need to pass through multiple hops after leaving the sending node, prior to arriving at the destination node. Depending on the nature of connectivity between the two nodes involved in a particular hop, a suitable transport protocol that does not offer reliable messaging characteristics may have to be chosen. In theory at least it is possible that each hop between the source and destination nodes picks a different transport protocol some of which are reliable and others are not. See Figure 1:

Figure 1: Multi-Hop messaging over different protocols

Figure 1: Multi-Hop messaging over different protocols

Obviously the above is an example of an extreme case scenario. However, given the future potential for a message path that involves intermediaries, where an intermediary can make a decision over the choice of next hop to a destination node, say based on dynamic channel availability characteristics and also based on the known capabilities of the receiving node etc., it is typically not possible for a sender to assume that reliable transport protocols will be available for the entire path between the sender and the receiver. Hence there is a need for a reliable messaging solution that is independent of the underlying transport protocol used to transmit the messages.

3. Requirements for Reliable Messaging

At the most basic level, reliable messaging refers to the ability of a sender to deliver a message *once and only once* to its intended receiver. However the key requirements of reliable messaging can be captured more formally as follows:

  1. Support carrying message traffic reliably in support of business processes whose lifetimes commonly exceed the up times of the components on which these processes are realized.
  2. Support quality-of-service assertions such as:
    1. Each message sent be received exactly once (once and only once), at most once, at least once, and so on.
    2. Messages be received in the same order in which they were sent.
    3. Failure to deliver a message be made known to both the sender and receiver.
  3. Accommodate mobility of a reliable business process to different channels or physical machines.
  4. Support message transfer via intermediaries.
  5. Leverage the SOAP extensibility mechanism to achieve reliable messaging.
  6. Enable reliable messaging bindings to a variety of underlying reliable and unreliable transport protocols together with the Message Routing Protocol.
  7. Compose with other protocols to support security and other message delivery services.

4. Reliable Messaging Not Unique to Web services

The concept of reliable messaging in XML-based distributed processing involving network based collaborative application entities is not a new one or unique to Web services. One of the first to recognize and device a solution to this problem that is truly transport-independent was RosettaNet via the RosettaNet Implementation Framework. The same basic mechanism of reliable messaging involving acknowledgement messages (called signals by RosettaNet) sent by the receiver of a message to the sender is used invariably by all reliable messaging schemes introduced in different domains including the BizTalk Framework 2.0, ebXML Messaging Service and the new specifications in the Web services paradigm.

4.1 Reliable Messaging in RosettaNet

The RosettaNet PIP model is primarily based on an asynchronous message exchange mechanism, where reliable messaging is accomplished by means of separate acknowledgment message exchanges sent by the receiver. Figure 2 depicts an example PIP interaction involving Acknowledgement messages:

Figure 2: Sample PIP Interaction

Figure 2: Sample PIP Interaction

In a PIP exchange, positive or negative acknowledgment messages are sent in response to a normal business message such as Purchase Order Request and Acceptance. Only normal business messages are acknowledged, and acknowledgement messages are never acknowledged. The PIP specification also defines the message exchange controls for each of the messages and the acknowledgements involved in the dialog. This includes the specification of time within which an Acknowledgment of Receipt must be sent and the maximum number of times a sender must send a normal message (retry) if an acknowledgement is not received within the timeout period specified above.

The sender achieves reliable message delivery by retrying a message delivery until an acknowledgement is received or the maximum number of retries is exhausted.

The sender assigns a globally unique identifier to the message sent that is used by the receiver to identify which message is being acknowledged. The globally unique identifier of a message is also used by the receiver to detect duplicate deliveries of the same message and discard additional copies of the same message received. Duplicate (multiple) deliveries of the same message could happen for example if the acknowledgement sent by the receiver is somehow lost and not received by the sender or if the acknowledgement did not reach the sender within the timeout period specified.

RosettaNet acknowledgement messages are typically digitally signed for providing the sender with the non-repudiation of the receipt of the message by the receiver.

4.2 Reliable Messaging in BizTalk Framework 2.0

Like RosettaNet, BizTalk Framework facilitates asynchronous exchange of BizTalk documents in different deployment scenarios that generally use transports with lower guarantees of service. Hence to provide a solution for reliable delivery of BizTalk documents, BizTalk Framework 2.0 defines special kind of BizTalk documents called "document receipts" to ensure a defined outcome for BizTalk document delivery. There are two types of document receipts, delivery-receipt and commitment-receipt.

Delivery and commitment receipts provide a way for the source business entity to assure itself that the BizTalk document was received and accepted (delivery receipt), or inspected for correctness of content and committed for processing (commitment receipt).

Given the possibility of multiple transmissions of the same BizTalk document due to retry, the destination business entity may apply idempotent delivery rules to detect and eliminate duplicate documents, using the unique identity of the documents very much like RosettaNet.

4.3 ebXML 2.0 Reliable Messaging Scheme

Reliability requirements defined in the ebXML Messaging Service (ebMS) also relate to the delivery of ebXML messages over unreliable communications channels. Reliability is achieved by a receiving ebMS MSH (Message Service Handler) responding to a message with an Acknowledgment Message. An Acknowledgment Message is any ebXML message containing an Acknowledgment element. Failure to receive an acknowledgment Message by a Sending MSH triggers successive retries until such time as an acknowledgment Message is received or the predetermined number of retries has been exceeded at which time the originator of the sent message is notified of the delivery failure.

It should be noted that reliable messaging elements of the ebMS supply reliability to the communications layer only, while staying independent of a specific communication protocol. That is, even though the reliable messaging protocol defined by ebMS is also centered around acknowledgment, retry and duplicate detection and elimination mechanisms, unlike RosettaNet and BTF the acknowledgements are not intended as business-level acknowledgments to the applications supported by the ebMS as well. For example, the acknowledgment defined in the ebMS specification does not indicate the payload of the ebXML message was syntactically correct. It does not acknowledge the accuracy of the payload information. It does not indicate business acceptance of the information or agreement with the content of the payload. The ebMS reliable messaging is designed to provide the sender with the confidence the receiving MSH has received the message securely and intact. Coupled with the duplicate detection and elimination mechanisms, this results in the destination party receiving the message Once-and-Only-Once.

Though ebXML predates the evolution of Web services, ebMS is one of the very first SOAP based protocols to specify reliable messaging specific SOAP header extension elements in the envelope. Some of the important ones are shown and described below:

<SOAP:Envelope xmlns:SOAP=""
    ............. ..........
      <eb:TimeToLive>2003-03-24T13:12:12</eb: TimeToLive>
...... ....
....... ....
......... ..... 

<MessageId> is a globally unique identifier for each message conforming to RFC2822.

<Timestamp> is a value representing the time that the message header was created conforming to the dateTime construct of XMLSchema and MUST be expressed as UTC.

<RefToMessageId> element when present contains the MessageId of an earlier ebXML message to which this message relates. An Acknowledgement message would typically use this element. If there is no earlier related message, this element is not present.

The <TimeToLive> element is used to indicate the time, expressed as UTC, by which a message should be delivered to the receiving party MSH. Any acknowledgements must be returned within this time.

The <DuplicateElimination> element if present identifies a request by the sender for the receiving MSH to check for and eliminate duplicate messages.

The <AckRequested> element is an extension to the SOAP Header used by the sending MSH to request that a Receiving MSH return an Acknowledgment Message.

The <Acknowledgment> element is an extension to the SOAP Header used by one Message Service Handler to indicate to another Message Service Handler that it has received a message. The RefToMessageId element in an Acknowledgment element is used to identify the message being acknowledged by its MessageId.

An example <Acknowledgment> element is shown below:

  <eb:Acknowledgment SOAP:mustUnderstand="1" eb:version="2.0">
    <eb:RefToMessageId>323210:e52151ec74:[email protected]</eb:RefToMessageId>
    <eb:From> <eb:PartyId></eb:PartyId> </eb:From>
<MessageOrder> <sequenceNumber> <MessageOrder>
  <eb:MessageOrder eb:version="2.0" SOAP:mustUnderstand="1">

The webMethods ebXML 2.0 implementation fully supports the ebMS 2.0 reliable messaging protocol including the duplicate detection in once-and-only-once delivery and message sequencing.

5. Web Services Reliable Messaging Specifications

Just within the last two months two major specifications that address reliable Web services messaging have been made public. The first called Web services Reliability (WS-Reliability) is a specification produced by several vendors including Sun, Oracle, and Fujitsu and the second named Web services Reliable Messaging Protocol (WS-ReliableMessaging) specification from IBM, Microsoft and BEA. Both of these specifications take roots in the reliable messaging protocol and SOAP header extension elements introduced in the ebXML (ebMS 2.0) specification discussed previously and extend them to the Web services domain. Below we take a closer look at each of these two specifications.

5.1 WS-Reliability

Web services Reliability (WS-Reliability) is a SOAP-based protocol for exchanging SOAP messages with guaranteed delivery, no duplicates, and guaranteed message ordering. The WS-Reliability specification borrows extensively from previous work in messaging protocols including SOAP and the ebXML Message Service [ebMS].

WS-Reliability proposes appropriate modifications to apply its work to the Web services domain. For example, WS-Reliability introduces the concept of different levels of quality of message delivery that can be negotiated between Web services and the clients. Within the context of WS-Reliability, reliable message delivery means the ability to ensure delivery of a message with the desired level of "quality of service". Some examples of this quality of service level for message delivery are:

  • Message delivered at least once (guaranteed delivery)
  • Message delivered at most once (guaranteed duplicate elimination)
  • Message delivered exactly once (guaranteed delivery and duplicate elimination)
  • Guaranteed message ordering, within a context delimited using a group id.

WS-Reliability specification defines reliable messaging specific extensions to SOAP Header and Body elements. SOAP 1.1 is currently the base protocol for this specification. However there is nothing really specific to SOAP 1.1 in this and the specification could be updated and defined as a Reliable Messaging Feature to be compliant with the SOAP 1.2 specification when it becomes a W3C Recommendation.

The WS-Reliability Messaging model is very similar to the one used by RosettaNet and ebMS and requires the receiving SOAP node to send back an acknowledgement message in response to a SOAP message sent by the sender SOAP node. In keeping with the conventions of the SOAP protocol however, a SOAP Fault message may be returned in response to a SOAP message received, if necessary (see Figure 3).

Every SOAP message intended for reliable delivery must contain a SOAP header extension element called <MessageID>, whose value must be a globally unique value. The Acknowledgment message must contain a reference to the MessageID of the Acknowledged message, confirming that the receiver SOAP node has received the message.

If the SOAP node sending a Reliable Message does not receive an Acknowledgment message, it must resend the same message with same MessageID to the receiver node until (1) the sender gets an Acknowledgment message from the receiver, or (2) a specified number of resend attempts have been made without success. If the sender SOAP node fails to send the message (i.e., no Acknowledgment is received), the node must report the error to the application layer.

Figure 3: WS-Reliability Message Exchange Mode

A number of conditions may result in the transmission of duplicate message(s), e.g., temporary downtime of the sender or receiver, a routing problem between the sender and receiver, etc. In order to provide at-most-once semantics, the ultimate receiver must eliminate duplicate messages. Messages with the same MessageID element value must be treated as duplicates.

Some applications may need to receive a sequence of messages from the same sender in the same order as they were sent. There could be a means to enforce this at the application level but this is not always possible or practical. To facilitate guarantee of the message order in such cases, WS-Reliability defines SOAP header level extension elements that are used by the SOAP nodes to deliver the messages in the proper order to the receivers even though they were received out of order at the SOAP node. Figure 4 illustrates this scenario:

Figure 4: Message Sequence Preservation Scenario

Figure 4: Message Sequence Preservation Scenario

WS-Reliability uses a sequence number mechanism to track and enforce the order of a sequence of messages having a common grouping identifier value, in a fashion very similar to that of ebMS.

Figure 5 shows the structure of the SOAP message with WS-Reliability extension elements embedded, including the grouped sequence elements described above. The extension elements applicable to the Acknowledgement messages are also shown. As can be seen these map almost one-to-one to the corresponding extension elements defined by the ebMS specification with a few exceptions (e.g. the GroupId).

Figure 5: WS-Reliability SOAP Header Extensions

Figure 5: WS-Reliability SOAP Header Extensions

Shown below is an example WS-Reliability SOAP message and the corresponding Acknowledgement message.

<SOAP:Envelope xmlns:SOAP= ...>
    <rm:MessageHeader xmlns:rm= SOAP:mustUnderstand="1">
      <rm:From>[email protected]</rm:From>
      <rm:To>[email protected]</rm:To>
      <rm:MessageId>[email protected]</rm:MessageId>
    <rm:ReliableMessage xmlns:rm= SOAP:mustUnderstand="1">
      <rm:AckRequested SOAP:mustUnderstand="1" synchronous="false" />
    <rm:MessageOrder xmlns:rm:= SOAP:mustUnderstand="1">
      <rm:GroupId status="Continue">[email protected]</rm:GroupId>
<SOAP:Envelope xmlns:SOAP="" ..>
    <rm:MessageHeader xmlns:rm= SOAP:mustUnderstand="1">
      <rm:From>[email protected]</rm:From>
      <rm:To>[email protected]</rm:To>
      <rm:MessageId>[email protected]</rm:MessageId>
    <rm:RMResponse xmlns:rm= SOAP:mustUnderstand="1">
      <rm:RefToMessageId>[email protected]</rm:RefToMessageId>

The WS-Reliability specification was recently submitted to the OASIS consortium jointly by webMethods, Sun, Oracle, Fujitsu, NEC and others. OASIS has since accepted the submission and formed a Technical Committee with the name WS-RM that will start its work later this month (March 2003). In addition to co-sponsoring the TC webMethods has joined the Technical Committee, and plans to help drive this effort forward.

ebMS also introduced another reliable messaging concept of preserving the order of messages when a sequence of messages is sent from the originator to the receiver. For this, ebMS added an extension header element called with a sub-element called . An example of the element is given below:

5.2 WS-ReliableMessaging

The Web services Reliable Messaging Protocol specification (WS-ReliableMessaging) was introduced to the public domain less than couple of weeks ago (Marc 13, 2003) by IBM, Microsoft and BEA almost in a hurried response to the WS-Reliability specification.

The reliable messaging model used by this specification is pretty much identical to that of the WS-Reliability and the ebMS specifications but for the terms used to model the entities in the Application and the transmission layer.

Figure 6: WS-ReliableMessaging Message Exchange Model

Figure 6: WS-ReliableMessaging Message Exchange Model

The Initial Sender sends a message for reliable delivery. The Source accepts the message and transmits it one or more times. After receiving the message the Destination acknowledges it. Finally, the Destination delivers the message to the Ultimate Receiver.

The XML namespace URI that is used by this specification is:

WS-ReliableMeesaging is a proprietary specification from the authors of the specification who reserve all Intellectual Property rights on the specification, as explicitly stated in the specification. Hence it is not available for anyone to implement on a Royalty Free basis. In addition, the WS-Reliable Messaging specification has dependencies on other proprietary specifications such as WS-Policy. The following namespaces are used by the specification:

Prefix Namespace

Endpoints that implement the WS-ReliableMessaging protocol provide delivery assurances for the delivery of messages sent from the initial sender to the ultimate receiver. There are four basic delivery assurances that endpoints can provide, that have one-to-one correspondence with the ones defined by WS-Reliability (and ebMS):

  • AtMostOnce Messages will be delivered at most once without duplication or an error will be raised. It is possible that some messages in a sequence are not be delivered.
  • AtLeastOnce Every message sent will be delivered or an error will be raised. Some messages may be delivered more than once.
  • ExactlyOnce Every message sent will be delivered without duplication or an error will be raised. This delivery assurance is the logical "and" of the two prior delivery assurances.
  • InOrder Messages will be delivered in the order that they were sent. This delivery assurance may be combined with any of the above delivery assurances.

Figure 7 from the specification illustrates a possible message exchange between two reliable messaging endpoints.

Figure 7: Example Reliable Message Exchange

Figure 7: Example Reliable Message Exchange

The WS-ReliableMessaging specification deviates from the other specifications in one minor but significant way. It models an exchange between two end-points always as a sequence irrespective of only one or a sequence of messages are exchanged. The protocol uses the <Sequence> element to identify and track of a group of messages, which are given a globally unique sequence-identifier, that is common to all messages in the sequence while individual messages in the sequence are identified by an ascending sequence number (unsignedLong). The same <Sequence> construct with overhead elements like <LastMessage/> must be used even if a single message needs to be exchanged. This could be counter-intuitive to many of the typical Web services changes that are not done in a group.

The following pseudo schema fragment defines the <Sequence> element.

<wsrm:Sequence ...>
  <wsu:Identifier> [URI] </wsu:Identifier>
  <wsrm:MessageNumber> [unsignedLong] </wsrm:MessageNumber>
  <wsu:Expires> [dateTime] </wsu:Expires>?
The following example illustrates a Sequence element.

Following the same sequence oriented pattern of originating messages, the acknowledgements are also returned as a sequence acknowledgement. This could cut the number of round trips if a group of messages are acknowledged together. However the optimization is not valuable for many of the typical Web service exchanges that are not group-oriented and even for group-oriented ones in many situations an acknowledgement would be required for each message in the sequence, each with its own expiry period (<Expires>) and a time to acknowledge.

The following pseudo schema defines the <SequenceAcknowledgement> element:

<wsrm:SequenceAcknowledgement ...>
  <wsu:Identifier> [URI] </wsu:Identifier>
  <wsrm:AcknowledgementRange ...
    Lower="[unsignedLong]"/> +

The following examples illustrate <SequenceAcknowledgement> elements:

  • Message numbers 1..10 inclusive in a Sequence have been received by the receiving endpoint.

  • <wsrm:SequenceAcknowledgment>
      <wsrm:AcknowledgmentRange Upper="10" Lower="1"/>
  • Message numbers 1..2, 4..6, and 8..10 inclusive in a Sequence have been received by the receiving endpoint, messages 3 and 7 have not been received.

  • <wsrm:SequenceAcknowledgment>
      <wsrm:AcknowledgmentRange Upper="2" Lower="1"/>
      <wsrm:AcknowledgmentRange Upper="6" Lower="4"/>
      <wsrm:AcknowledgmentRange Upper="10" Lower="8"/>

    Acknowledgements can be sent on return messages or sent standalone. In the case that a return message is not available to send an acknowledgement a destination may wait for up to the acknowledgement interval before sending a standalone acknowledgement. The following pseudo schema defines the corresponding <AcknowledgementInterval> element:

    <wsrm:AcknowledgementInterval Milliseconds="[unsignedLong]" ...> ...

    Policy assertions can be attached to an individual Sequence by embedding a <SequenceRef> element that identifies that Sequence within a <PolicyAttachment> element.

    The following example attaches the policy "profile" with the Sequence with the Identifier of "". Additionally, it requires that the Sequence use this version of the WS-ReliableMessaging specification in a manner that provides the delivery assurances associated with "AtMostOnce":

        <wsrm:DeliveryAssurance Value="wsrm:AtMostOnce"

    Despite any potential merits and new concepts introduced by WS-ReliableMessaging, the specification is plagued by the Intellectual Property issues associated with the specification.

    6. Summary

    Reliable message delivery is very important for the success of Web services, without which organizations will not be able to trust them for deployment in industrial-strength business applications and for mission-critical operations such as complex business-to-business transactions or real-time enterprise integrations. To provide for use of Web services over unreliable networks and to keep them truly transport independent, the reliable messaging solutions must be defined at a level higher than the underlying transport protocols. Protocols such as HTTPR have not seen much of an acceptance if any, precisely for that reason.

    The problem of reliable messaging has been dealt with comprehensively by earlier messaging frameworks such as RosettaNet Implementation Framework and ebXML Messaging Service. Reliability of message delivery is achieved by a receiver responding to a message with an Acknowledgment Message, failure to receive an acknowledgment triggering successive retries until such time as an Acknowledgment is received or the predetermined number of retries has been exceeded at which time the originator of the sent message is notified of the delivery failure. This basic principle and the SOAP header extension mechanisms used by ebXML are also adapted by the reliable messaging specifications in the Web services space, while applying appropriate modifications to the Web services domain.

    webMethods has significant experience in delivering reliable messaging solutions via the implementation of RosettaNet Implementation Framework versions 1.1 and 2.0 and the ebXML messaging service 1.0 and 2.0 specifications. webMethods has co-sponsored the submission of the WS-Reliability specification to OASIS and is driving this effort forward via active involvement in the WS-RM Technical Committee.

    7 References

    Dig Deeper on Topics Archive