WS-Reliable Messaging / WS-Reliability

Reliable Messaging WS-RM / WS-Reliability

WS-Reliable Messaging is good, but it’s not enough.

WS-Reliable Messaging is a retread of existing messaging technology as a Web Service.

WS-Reliable Messaging is supported by IBM and Microsoft. The competing WS-Reliability, supported by Oracle and Sun, is so similar that the world only needs one of them (much like the situation with business transaction specs). We think that IBM and Microsoft are stronger market forces than Oracle and Sun, and so, barring "WS-Rupture", we expect WS-RM to prevail over WS-Reliability, or the two will merge. But we will be happy to use either.

Business Transaction Management + Reliable Messaging provides Guaranteed Delivery and Processing (GDP). That is, you can be assured that your message not only got to the incoming message port, but was correctly processed by the destination business application, and moreover, that a whole sequence of related messages between business applications was processed correctly.

If you have RM and ACID (XA capable) message queues and database containers, you can get GDP. GDP is a cinch with J2EE. What’s not so easy to achieve is GDP where the endpoints are operations on services which come from different places and families. GDP is dependent on a restricted set of assumptions, and becomes more difficult the more heterogeneous the environment.

Even more important is the containment of multiple related messages, even among multiple parties, in the same transactional boundary – in other words, a transactional conversation. You cannot get this larger coordinated unit of work from RM nor J2EE.

Example

Take a Quote-To-Order process, where a buyer requests a quote, the seller sends the quote, the buyer places a firm order, and the seller accepts the order.

Using reliable messaging alone, both parties can be assured that each message was delivered to the counterparty’s incoming message port. But that’s all.

IFF both parties use compatible WS-BPEL processes, both parties can follow the sequence of messages and the state of the process. But they cannot be assured that both parties understand the state of the process in the same way, at the same time, with the same conclusion. This is what we call Mutually Assured Knowledge (MAK): in other words, “I know that you know that I know what you know”.

Some of the requirements for business transaction integrity, or Mutually Assured Knowledge :

• Context: Each party needs to know that they are engaged in the same coordinated process, with a clear beginning and outcome. BPEL Correlation can help, but you cannot depend on the uniqueness of RFQ, Quote and Order IDs from multiple trading partners. Moreover, one BPEL process can contain more than one logical transaction (see Confirmation below), all of which might use the same BPEL Correlation Set. The simplest solution is a unique identifier for the conversation, carried on every message, which in transaction terms is called a Context.
• Outcome protocol: The seller can be assured that the buyer received the Acceptance message, but the seller cannot know that the buyer has processed the Acceptance message correctly or in a timely fashion. In the worst case, the seller could ship goods while the buyer found something wrong with the seller’s response: validation problem, late response, changes from the quote, delayed or partial delivery schedule. You need to agree about who decides the outcome of the process, and add another message round-trip for MAK.
• Confirmation: If the BPEL process contains more than one logical transaction – for example, delivery and payment in addition to quote-and-order – you need closure on the order before the seller delivers the goods. You cannot get closure on anything in BPEL until the end of the process. BPEL provides Compensations – that is, a unilateral undo mechanism – and BPEL Compensations may undo anything, including the agreement on the order, until the end of the whole process. BPEL does not provide Confirmation, which would mean that the order agreement has been securely concluded.
• Time constraints: Messages can be delayed. How long will you wait? You need agreed-upon time-outs. BPEL provides mechanisms, but where should they go in the process? And what should you do when they fire?
• Recovery and resynchronization: If one process goes down, a good BPEL engine can recover and restart that process, but now you need to figure out if and how to re-synchronize with the counter-party’s process. For example, if the buyer goes down after sending the RFQ and the seller gives up on sending the Quote, can the buyer revive the process automatically upon recovery? What if the buyer has more than one seller and one goes down, recovers and sends a better quote, but it’s late. What does the buyer do? These are all process design decisions with business impacts.
• The solutions to these and other coordination problems are interdependent, requiring “transaction-grade” programming skills and a lot of testing.
• These and other coordination problems become more important to the extent the conversation is automated, and especially if the outcome of the order conversation is the trigger for delivery (as explained above under Outcome Protocol and Confirmation).
• These and other coordination problems become much more difficult as more parties are involved in the process (see below).
• If you add a full set of transactional-integrity features to one BPEL process, you will have to do it all over again for the next one.
• And you can be almost certain that your trading partners will not have implemented compatible transactional-integrity features, so you will need to engage in a long design-negotiation and testing process with each trading partner.

By the time you get all of these transactional-integrity features working correctly, you will have re-invented Business Transaction Management software.

All of these problems and can be handled once and for all, in a standardized way, by Choreology Cohesions Business Transaction Management software. Moreover, BTM software can provide transactional integrity with or without RM, XA or BPEL – in other words, with the usual combination of old and new technologies. We enclose the whole conversation in a transaction:

In this case, each business message is accompanied by a transaction signal. (In other cases, business messages may be separate from transaction signals.) Cohesions software interprets the signals and logs the state of the transaction. If one party fails, Cohesions software can recover and resynchronize with the other party (or parties). At each key stage of the transaction, both parties’ states are synchronized by Cohesions software. At the end of the transaction, both parties have come to the same business conclusion, and know that their counterparty will agree.

Cohesions provides more benefits if you have more sellers to coordinate. Coordinating three parties without BTM software is much more difficult than coordinating two. Re-synchronizing with multiple participants becomes especially difficult.

Choreology has documented an experiment where our Chief Architect developed a similar process with our Cohesions software, and without Cohesions but with Reliable Messaging. The productivity gains from Cohesions were 3:1 for a simple process, without considering testing. To achieve the same degree of transactional integrity as Cohesions would make the advantage about 10:1. And that’s 10 times more of a very highly skilled programmer’s time.

We are interested in standardized business protocols for transactions like Quote-To-Order because they will help shorten the negotiations on process design and transaction rules between trading partners. But the basics for business transactional integrity are very similar across many business protocols, and are provided by Business Transaction Management software such as Choreology Cohesions.