TJTST21 Spring 2006 1

MOM->MQ-> ESB ->(Integration server)

Yrityksen tietojärjestelmien integrointi

Petri Maaranen

TJTST21 Spring 2006 2

Topics

• Message oriented middleware

• Message brokers

• Enterprise Service bus

TJTST21 Spring 2006 3

TJTST21 Spring 2006 4

Types of Middleware

• Remote procedure calls

• Message-oriented middleware

• Distributed objects

• Database-oriented middleware

• Transaction-oriented middleware

• Message brokers

TJTST21 Spring 2006 5

Message Oriented Middleware

• Message-Oriented Middleware (MOM) was created to address some of the shortcomings of RPCs (RMI) through the use of messaging. • Like RPCs, MOM provides a standard API across hardware, operating system platforms, and networks. • MOM is also able to guarantee that messages will reach their destination, even when the destination is not available (asynchronous mode).

TJTST21 Spring 2006 6

Message-Oriented Middleware (2)

• The asynchronous model allows the application to continue processing after making a middleware service request. The message is dispatched to queue manager, which makes sure that the message is delivered to its final destination. • This asynchronous paradigm is much more convenient for developers and users because it does not block the application form processing although the model is more complex.

TJTST21 Spring 2006 7

Message-Oriented Middleware (3)

• There are two models supported by the MOM: – point-to-point (both systems must be active) – message queuing (MQ, one end has to be active)

• MQ has several performance advantages over the traditional RPCs

– each participating program proceeds on its own pace without interruption from the middleware layer – a program can publish the same the same message to many remote programs without waiting the programs to be up and running – the queue manager can optimize the performance through priorization, load balancing, and thread pooling

TJTST21 Spring 2006 8

TJTST21 Spring 2006 9

TJTST21 Spring 2006 10

Message-Oriented Middleware (4) Message Brokers

• A message broker is a software system based on asynchronous, store-and-forward messaging. • It manages interactions between applications and other information resources, utilizing abstraction techniques. • Simple operation: an application puts (publishes) a message to the broker, another application gets (subscribes to) the message. The applications do not need to be session connected.

TJTST21 Spring 2006 11

(Message Brokers, MQ)• MQ is fairly fault tolerant in the cases of network or system failure. • Most MQ software lets the message be declared as persistent or stored to disk during a commit at certain intervals. This allows for recovery on such situations. • Each MQ product implements the notion of messaging in its own way. • Widely used commercial examples include IBM’s MQSeries and Microsoft’s MSMQ.

TJTST21 Spring 2006 12

TJTST21 Spring 2006 13

Message Brokers

• Any-to-anyThe ability to connect diverse applications and other information resources – The consistency of the approach – Common look-and-feel of all connected resources • Many-to-many – Once a resource is connected and publishing information, the information is easily reusable by any other application that requires it.

TJTST21 Spring 2006 14

Standard Features of Message Brokers

• Message transformation engines – Allow the message broker to alter the way information is presented for each application.

• Intelligent routing capabilities – Ability to identify a message, and an ability to route them to appropriate location.

• Rules processing capabilities – Ability to apply rules to the transformation and

routing of information.

TJTST21 Spring 2006 15

TJTST21 Spring 2006 16

Services provided by Message Brokers (or Integration servers)

• (A) Message transformation • (B) Rules processing • (C) Intelligent routing • (D) Message warehousing and flow control • (E) Repository services • (F) Directory services • (G) Management • (H) APIs and adapters

TJTST21 Spring 2006 17

(A) Message Transformation Layer

• The message transformation layer understands the format of all messages being passed among the applications and transforms those messages while they move. • It provides a common dictionary that contains information on how each application communicates outside itself, as well as which pieces of information have meaning to which applications.

TJTST21 Spring 2006 18

Message Transformation Layer (2)

• Generally contain parsing and pattern-matching methods that describe the structure of any message format.

• Message formats are then constructed from pieces that represent each field encapsulated within a message.

• Once the message has been broken down to its components parts, the fields may be recombined to create a new message.

• Two kinds of transformations: schema conversions and data conversions.

TJTST21 Spring 2006 19

Schema and Data Conversions

•A schema conversion is the process of changing the structure of a message, and thus re-mapping the schema, so that it is acceptable for the target system.

• In other words, general re-mapping of data formats.

•A data conversion is the process of determining the data format of source and target applications, assessing the differences between them.

• That is, changes within an individual data field.

TJTST21 Spring 2006 20

TJTST21 Spring 2006 21

(B) Rules Processing

• The rules processing engine provides the ability to create rules to control the processing and distribution of messages.

• It is an application development environment supported by the message broker to address the special requirements of integrating applications.

• By using rules engine, it becomes possible to implement intelligent message routing and transformation.

• The logic for sharing information.

TJTST21 Spring 2006 22

The Rules

• Most often, rules engines use traditional Boolean logic (IF, ELSE, and OR) with an higher level language to associate the rule with appropriate actions.

• Generally relies on scripting languages rather than more complex programming languages.

• The implementation of the rules generation varies from entering commands to the broker to Wizard-assisted management of rules repositories.

TJTST21 Spring 2006 23

(C) Intelligent Routing

• A message broker can identify a message coming from the source application and route it to the proper target application, transforming it if required.

• Builds on the capabilities of both the rules layer and the message transformation layer.

• This happens virtually instantaneously, with as many as a thousand of these operations simultaneously.

TJTST21 Spring 2006 24

(D) Message Warehousing

• A message warehouse is a database that, as an option, is able to store messages that flow through the message broker.

• This feature is provided for

– Message mining

– Message integrity

– Message archiving

–Auditing

TJTST21 Spring 2006 25

Message Mining and Integrity

• Data mining – Allows extraction of business data to support decision

making – For example, the amount and characteristics of new

customer information processed through the broker – Rules can be applied to alter the messages prior the

storing, e.g., for the data to make sense to business users

• Message integrity – A persistent buffer or queue for messages on the event of

system failures (or for other reasons) – The messages may be resent or compared with other

warehouses on the network to insure integrity

TJTST21 Spring 2006 26

Message Archiving and Auditing

• Message archiving – Ability to store months (or more) of message traffic – For auditing or other purposes

• Auditing – Using the warehouse to determine the health of the EAI solution – One can monitor message traffic loads, message content variations, the number of messages requiring transformation, etc.

TJTST21 Spring 2006 27

(E) Repository Services

• A message broker repository is a database of information about source and target applications.

• A sophisticated repository is capable of keeping track not only rudimentary information (such as directory data), but more sophisticated information about the systems:

– Location of system – Transformation information – Security parameters – Design and architecture

information– Rules and logic for message – Object information processing – Message schema information-- Metadata – Enabling technology

TJTST21 Spring 2006 28

Value of the Repository

• A common reference point for all connected processes and databases.

• Greatly simplifies integrating data and methods: it is just about finding their equivalents and joining them together.

• Possibility to track all the rules applied within the EAI problem domain.

• By containing the schema of both source and target system, it also contains information for proper transformation of messages flowing from source to target.

TJTST21 Spring 2006 29

(F) Directory Services

• Message brokers deal with distributed systems, and thus require directory services to

– locate, – identify,–use, and– authorize network resources for those systems

• Directory services provide a single point of entry for applications and middleware – directories know where to find resources on behalf of applications.

TJTST21 Spring 2006 30

Directory Services (2)

• Directory services are nothing more than a way to classify resources on the network in a way consistent with any other method of classification.

• A multitude of widely available directory services exists: – Domain Name System (DNS) – Novell’s NetWare Directory System and Directory

Services – Netscape Communications’ Directory Server – Microsoft’s Active Directory – X.500

TJTST21 Spring 2006 31

(G) Management

• Administration and management of the EAI problem domain is primarily the responsibility of the management layer of the message broker.

• General purpose management tools, such as IBM Tivoli Framework, Microsoft Systems Management Server, HP OpenView, Novell ZENworks and CA Unicenter, lack the ability to monitor message traffic, message integrity, and the coordination of the distribution of messages among target applications.

TJTST21 Spring 2006 32

(H) Adapters and APIs

• API is nothing more than the mechanism that allows an application to access the services of a message broker.

• Adapters link deeply into an application without having to invoke the native interfaces. – Layers between the message broker interface and the source or target application – For example, set of libraries that map the differences between two distinct interfaces – Adapters for different source and target applications(SAP R/3, Baan, Oracle, PeopleSoft,…) – Adapters for certain types of databases (Oracle, Sybase, DB2,..)

TJTST21 Spring 2006 33

Enterprise Service Bus

• Enterprise Service Bus • Found in a category of middleware

infrastructure products usually based on Web services standards, that provides foundational services for more complex service-oriented architectures via an event-driven and XML-based messaging engine (the bus).

• An enterprise service bus generally provides an abstraction layer on top of an Enterprise Messaging System which allows integration architects to exploit the value of messaging without writing code.

TJTST21 Spring 2006 34

ESB common characteristics 1. it requires the clear separation of message headers and message body 2. it is usually operating system and language independent; it should work between Java and .Net

applications, for example 3. it (often) uses XML and Web services to transport messages 4. it includes adapter standards (such as J2C) for incorporating existing applications into the bus 5. it includes support for asynchronous processing 6. it includes intelligent, content-based routing 7. it includes a standardized security model to authorize, authenticate, and audit use of the ESB 8. it includes transformation services (such as XSLT) between the format of the sending application

and the receiving application, including the transformation of data formats 9. it includes validation against schemas for sending and receiving messages 10. it can uniformly apply business rules, enrichment of the message from other sources, splitting

and combining of multiple messages, and the handling of exceptions 11. it can conditionally route or transform messages based on a central policy 12. it is monitored for message latency and other characteristics described in a Service Level

Agreement 13. it (often) facilitates "service classes," responding appropriately to higher and lower priority users 14. it supports queuing, holding messages if applications are temporarily unavailable 15. it handles a "publish and subscribe" messaging model, including event handling

TJTST21 Spring 2006 35

ESB / XML

• An alternative view, particularly for high performance enterprise service buses, is that "standard" message formats should flow across the bus, not just XML.

• Generating XML and parsing it can be costly in terms of processing and memory, and high volume scenarios may not be viable.

TJTST21 Spring 2006 36

ESB-related vendors

• Strictly speaking ESB is an architectural concept which may be implemented by one or more products. Most vendors naturally argue that ESB is a product — their product — and more and more are putting "ESB" in their product names and literature. There are a number of vendors who offer ESB-oriented products.

TJTST21 Spring 2006 37

Vendors•

Adea S

olutions[2]: Adea E

SB

Fram

ework

•S

erviceMix[3]: S

erviceMix (A

pache) •

[4]: Synapse (A

pache Incubator) •

BE

A: A

quaLogic Service B

us •

BIE

: Business integration E

ngine •

Cape C

lear Softw

are: Cape C

lear 6 •

Cordys: C

ordys ES

B

•F

iorano Softw

are Inc. Fiorano E

SB

™ 2006 •

IBM

: WebS

phere Platform

(specifically W

ebSphere M

essage Broker or W

ebSphere

ES

B)

•IO

NA

Technologies: A

rtix •

iWay S

oftware: iW

ay Adaptive F

ramew

ork for S

OA

•

Microsoft: .N

ET

Platform

M

icrosoft BizT

alk Server [5]

•O

bjectWeb: C

eltix (Open S

ource, LGP

L) •

Oracle: O

racle Integration products •

Petals S

ervices Platform

: EB

M

WebS

ourcing & F

ossil E-C

omm

erce (Open

Source)

•P

olarLake: Integration Suite

•LogicB

laze: ServiceM

ix ES

B (O

pen Source,

Apache Lic.)

•S

oftware A

G: E

ntireX

•S

onic Softw

are: Sonic E

SB

•

Sym

phonySoft: M

ule (Open S

ource) •

TIB

CO

Softw

are •

Virtuoso U

niversal Server

•w

ebMethods: w

ebMethods F

abric

TJTST21 Spring 2006 38

Conclusions

• Message oriented middleware ->

Message brokers-> ESB

• Services provided by Message Brokers

• Common characteristics of ESB

• Products and vendors