A very common scenario nowadays is to use SaaS application like Salesforce or SugarCRM for CRM. Then to use SaaS ERP like NetSuite or on-premise ERP. There could also be Quickbooks used for accounting. Besides these there are the HelpDesk apps.

All of these applications have their own internal databases, schemas and representation of your data. A change in one place needs to be reflected in other apps as well. This results in lot of difficulties on how to integrate the apps. Conventional star topology or app-to-app integration fall short.

This is where a Messaging Oriented Middleware (MOM) solution like ESB is very useful. I am presenting a generic architecture for multiple apps to apps integration.

Brief explanation of components in the proposed architecure -

• Purchase Order – This is the input document that comes to the system. We generally need to update multiple systems based on the docuemnt. The format could be cXML or any custom XML schema. It could be transformed into a standard or canonical format accepted internally.
• ActiveMQ JMS – The document is put on Topic of any messaging application. I have assumed ActiveMQ here, but it could be any MQ system that supports publish-subscribe model.
• Transformers – We have 3 subscribers to this Topic, however each of them accepts different formats. To compensate for this, we have a trasnfromer for each of the subscriber. e.g., the PO to CRM transformer could transform the message from CXML format to SalesForce.com Schema.
• Subscribers – All 3 subscribers receive the message document and update their respective system with the data.

According to Enterprise Integration Patterns,

The Dynamic Router is a Router that can self-configure based on special configuration messages from participating destinations.

We can realize this pattern in OpenESB using Dynamic Addressing which uses WS-Addressing to dynamically determine the service location at runtime.

To check this in action we need 3 Projects -

• EJB Module implementing the Web Service – you can use any WSDL based web service, not necessary to use EJB.
• BPEL – This will contain the actual code that determines the endpoint at runtime.
• JBI / CASA – This is a composite application necessary to deploy the BPEL.

Step 1 - We create a web service with 2 operations – addition and subtraction. Both operations take 2 numbers as input and return result.
Here’s the xsd –

<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema"
    targetNamespace="http://xml.aurorite.com/schema/numberBase"
    xmlns:tns="http://xml.aurorite.com/schema/numberBase"
    elementFormDefault="qualified">

    <xsd:element name="NumbersAdditionRequest" type="tns:AdditionType"/>
    <xsd:element name="NumbersAdditionResponse" type="tns:OperationResultType"/>
    <xsd:element name="NumbersSubtractionRequest" type="tns:SubtractionType"/>
    <xsd:element name="NumbersSubtractionResponse" type="tns:OperationResultType"/>

    <xsd:complexType name="AdditionType">
        <xsd:sequence>
            <xsd:element name="number1" type="xsd:int"/>
            <xsd:element name="number2" type="xsd:int"/>
        </xsd:sequence>
    </xsd:complexType>

    <xsd:complexType name="SubtractionType">
        <xsd:sequence>
            <xsd:element name="number1" type="xsd:int"/>
            <xsd:element name="number2" type="xsd:int"/>
        </xsd:sequence>
    </xsd:complexType>

    <xsd:complexType name="OperationResultType">
        <xsd:sequence>
            <xsd:element name="result" type="xsd:int"/>
            <xsd:element name="processor" type="xsd:string"/>
        </xsd:sequence>
    </xsd:complexType>
</xsd:schema>

and here’s the wsdl -

<?xml version="1.0" encoding="UTF-8"?>
<definitions name="NumbersOperations"
    targetNamespace="http://websvcs.aurorite.com/wsdl/NumberOperationSvcs/NumbersOperations"
    xmlns="http://schemas.xmlsoap.org/wsdl/"
    xmlns:wsdl="http://schemas.xmlsoap.org/wsdl/"
    xmlns:xsd="http://www.w3.org/2001/XMLSchema" 
    xmlns:tns="http://websvcs.aurorite.com/wsdl/NumberOperationSvcs/NumbersOperations"
    xmlns:ns="http://xml.aurorite.com/schema/numberBase"
    xmlns:plnk="http://docs.oasis-open.org/wsbpel/2.0/plnktype"
    xmlns:soap="http://schemas.xmlsoap.org/wsdl/soap/">
    <types>
        <xsd:schema targetNamespace="http://websvcs.aurorite.com/wsdl/NumberOperationSvcs/NumbersOperations">
            <xsd:import namespace="http://xml.aurorite.com/schema/numberBase" schemaLocation="numberBase.xsd"/>
        </xsd:schema>
    </types>
    <message name="additionRequest">
        <part name="part1" element="ns:NumbersAdditionRequest"/>
    </message>
    <message name="additionResponse">
        <part name="part1" element="ns:NumbersAdditionResponse"/>
    </message>
    <message name="subtractionRequest">
        <part name="part1" element="ns:NumbersSubtractionRequest"/>
    </message>
    <message name="subtractionResponse">
        <part name="part1" element="ns:NumbersSubtractionResponse"/>
    </message>
    <portType name="NumbersOperationsPortType">
        <operation name="addition">
            <input name="input1" message="tns:additionRequest"/>
            <output name="output1" message="tns:additionResponse"/>
        </operation>
        <operation name="subtraction">
            <input name="input2" message="tns:subtractionRequest"/>
            <output name="output2" message="tns:subtractionResponse"/>
        </operation>
    </portType>
    <binding name="NumbersOperationsBinding" type="tns:NumbersOperationsPortType">
        <soap:binding style="document" transport="http://schemas.xmlsoap.org/soap/http"/>
        <operation name="addition">
            <soap:operation/>
            <input name="input1">
                <soap:body use="literal"/>
            </input>
            <output name="output1">
                <soap:body use="literal"/>
            </output>
        </operation>
        <operation name="subtraction">
            <soap:operation/>
            <input name="input2">
                <soap:body use="literal"/>
            </input>
            <output name="output2">
                <soap:body use="literal"/>
            </output>
        </operation>
    </binding>
    <service name="NumbersOperationsService">
        <port name="NumbersOperationsPort" binding="tns:NumbersOperationsBinding">
            <soap:address location="http://localhost:8080/NumbersOperationsService/NumberService"/>
        </port>
    </service>
    <plnk:partnerLinkType name="NumbersOperations">
        <!-- A partner link type is automatically generated when a new port type is added. Partner link types are used by BPEL processes. 
In a BPEL process, a partner link represents the interaction between the BPEL process and a partner service. Each partner link is associated with a partner link type.
A partner link type characterizes the conversational relationship between two services. The partner link type can have one or two roles.-->
        <plnk:role name="NumbersOperationsPortTypeRole" portType="tns:NumbersOperationsPortType"/>
    </plnk:partnerLinkType>
</definitions>

Here is the implementation of the webservice operations -

@WebService(serviceName = "NumbersOperationsService", portName = "NumbersOperationsPort", endpointInterface = "com.aurorite.websvcs.wsdl.numberoperationsvcs.numbersoperations.NumbersOperationsPortType", targetNamespace = "http://websvcs.aurorite.com/wsdl/NumberOperationSvcs/NumbersOperations", wsdlLocation = "META-INF/wsdl/NumberService/NumbersOperations.wsdl")
@Stateless
public class NumberService {

    Logger logger = Logger.getLogger("soa.aurorite.com.NumberService");

    public com.aurorite.xml.schema.numberbase.OperationResultType addition(com.aurorite.xml.schema.numberbase.AdditionType part1) throws UnknownHostException {
        logger.info("Running this service at " + InetAddress.getLocalHost().getHostName() + " on port 8080");
        com.aurorite.xml.schema.numberbase.OperationResultType result = new com.aurorite.xml.schema.numberbase.OperationResultType();
        result.setProcessor(InetAddress.getLocalHost().getHostName());
        try {
            result.setResult(part1.getNumber1() + part1.getNumber2());
        } catch (Exception e){
            result.setResult(0);
        }

        return result;
        //throw new UnsupportedOperationException("Not implemented yet.");
    }

    public com.aurorite.xml.schema.numberbase.OperationResultType subtraction(com.aurorite.xml.schema.numberbase.SubtractionType part1) throws UnknownHostException {
        logger.info("Running this service at " + InetAddress.getLocalHost().getHostName() + " on port 8080");
        com.aurorite.xml.schema.numberbase.OperationResultType result = new com.aurorite.xml.schema.numberbase.OperationResultType();
        result.setProcessor(InetAddress.getLocalHost().getHostName());
        try {
            result.setResult(part1.getNumber1() - part1.getNumber2());
        } catch (Exception e){
            result.setResult(0);
        }
        return result;
        //throw new UnsupportedOperationException("Not implemented yet.");
    }

}

Note that I’m returning the machine name in the response. This’ll help us to determine who actually processed a particular operation.

Now to the BPEL. We’ll create a simple process looking like this -

In the BPEL, we add these 2 namespace references -

xmlns:sref="http://docs.oasis-open.org/wsbpel/2.0/serviceref" 
xmlns:wsa="http://schemas.xmlsoap.org/ws/2004/08/addressing"
    <import namespace="http://docs.oasis-open.org/wsbpel/2.0/serviceref" location="ws-bpel_serviceref.xsd" importType="http://www.w3.org/2001/XMLSchema"/>
    <import namespace="http://schemas.xmlsoap.org/ws/2004/08/addressing" location="addressing.xsd" importType="http://www.w3.org/2001/XMLSchema"/>

And now the most important part – Adding the Service Endpoint Reference

<assign name="assign_endpoint">
    <copy>
                        <!--<from>ns2:doXslTransform('urn:stylesheets:wrap2serviceref.xsl', $EPRVariable.eprValue)</from>-->
        <from>
            <literal>
                <sref:service-ref>
                    <wsa:EndpointReference>
                        <wsa:Address>http://machine2:8080/NumbersOperationsService/NumberService</wsa:Address>
                        <wsa:ServiceName PortName="NumbersOperationsPort"
                                        xmlns:serv="http://websvcs.aurorite.com/wsdl/NumberOperationSvcs/NumbersOperations">serv:NumbersOperationsService
                        </wsa:ServiceName>
                    </wsa:EndpointReference>
                </sref:service-ref>
            </literal>
        </from>
        <to partnerLink="NumberService"/>
    </copy>
</assign>

The portname and Service name must match to the ones in target service.

According to Enterprise Integration Patterns,

The Content-Based Router examines the message content and routes the message onto a different channel based on data contained in the message. The routing can be based on a number of criteria such as existence of fields, specific field values etc. When implementing a Content-Based Router, special caution should be taken to make the routing function easy to maintain as the router can become a point of frequent maintenance. In more sophisticated integration scenarios, the Content-Based Router can take on the form of a configurable rules engine that computes the destination channel based on a set of configurable rules.

It can be achieved in OpenESB using “If” component.

In this example it simply checks for the value in “num:operation” element.

<soapenv:Envelope xsi:schemaLocation="http://schemas.xmlsoap.org/soap/envelope/ http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:num="http://xml.aurorite.com/schema/NumberOperationSchema">
  <soapenv:Body>
    <num:OperationRequest>
      <num:operation>ADDITION</num:operation>
      <num:data>
        <num:number1>20</num:number1>
        <num:number2>30</num:number2>
      </num:data>
    </num:OperationRequest>
  </soapenv:Body>
</soapenv:Envelope>     

At a high Level the answer is Yes, however not completely.

SOA requires thinking about the system in terms of

• Services (well-defined business functionality)
• Components (discrete pieces of code and/or data structures)
• Processes (Service orchestrations. Generally using BPEL)

Being able to compose new higher level services or business processes is a basic feature of a good SOA. XML, SOAP based Web Services and related standards are good fit for realizing SOA.

Also SOA has a few accepted principles – http://en.wikipedia.org/wiki/Service-oriented_architecture#Principles

• Standardized service contract – Services adhere to a communications agreement, as defined collectively by one or more service-description documents.
• Service Loose Coupling – Services maintain a relationship that minimizes dependencies and only requires that they maintain an awareness of each other.
• Service Abstraction – Beyond descriptions in the service contract, services hide logic from the outside world.
• Service reusability – Logic is divided into services with the intention of promoting reuse.
• Service autonomy – Services have control over the logic they encapsulate.
• Service granularity – A design consideration to provide optimal scope and right granular level of the business functionality in a service operation.
• Service statelessness – Services minimize resource consumption by deferring the management of state information when necessary.
• Service discoverability – Services are supplemented with communicative meta data by which they can be effectively discovered and interpreted.
• Service composability – Services are effective composition participants, regardless of the size and complexity of the composition.

A SOA based architecture is expected to have Service Definition. Since RESTful web services lack a definitive service definition (similar to wsdl), it is difficult for a REST based system to fulfill most of the above principles.

To achieve the same using REST, you’d need to have RESTful Web Services + Orchestration (possible using some lightweight ESB like MuleESB or Camel)

Please also see this resource – From SOA to REST

Orchestration is required to compose processes. That’s what provides the main benefit of SOA.

Say you have a order processing application with operations like
-

• addItem
• addTax
• calculateTotal
• placeOrder

Initially you created a process (using BPEL) which uses these operations in sequence. You have clients who use this Composed Service. After a few months a new client comes who has tax exemption, then instead of writing new service, you could just create a new process skipping the addTax operation. Thus you could achieve faster realization of business functionality just by re-using existing service. In practice there are mutiple such services.

Thus BPEL or similar (ESB or routing) technology is essential for SOA. Without business use, a SOA is not really a SOA.

Part 1:Enterprise Integration Patterns (EIP) with OpenESB Part 1 : Content-Based Router

Create a new SoapUI project and add the WSDL as initial WSDL. SoapUI will create a new project with placeholder requests for all operations. Let’s open ContactQueryPage operation from SoapUI.

The endpoint path will be something like - https://secure-ausoxxxxx.crmondemand.com/Services/Integration
This path is  unique to your instance of Siebel COD.

Next lets remove the body of input and replace it with -

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Header/>
<soapenv:Body>
     <con:ContactQueryPage_Input xmlns:con="urn:crmondemand/ws/ecbs/contact/" xmlns:quer="urn:/crmondemand/xml/Contact/Query">
        <quer:ListOfContact startrownum="0" pagesize="100" recordcountneeded="true">
            <quer:Contact searchspec="[ContactFirstName] LIKE 'John'">
                <quer:Id/>
                <quer:ContactFirstName/>
                <quer:ContactLastName/>
                <quer:AccountId/>
            </quer:Contact>
        </quer:ListOfContact>
    </con:ContactQueryPage_Input>
</soapenv:Body>
</soapenv:Envelope>

This results in Soap Fault response with Siebel error code SBL-ODU-01006 and message : “Internal Error: Session is not available. Aborting.”
This is because we still haven’t added username/password for authentication. There is another option to first get the session and then pass that value in header of the request. However I prefer to use the Stateless version above, as it separates the responsibility of handling session.

Let’s make 2 changes to fix this error.
1) Replace first line of the SOAP request with -

<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:wsse="http://schemas.xmlsoap.org/ws/2002/07/secext">

2) Add these lines just below the SOAP Envelope element -

<soapenv:Header>
   <wsse:Security>
      <wsse:UsernameToken>
         <wsse:Username>your_siebel_username</wsse:Username>
         <wsse:Password Type="wsse:PasswordText">your_siebel_password</wsse:Password>
      </wsse:UsernameToken>
   </wsse:Security>
</soapenv:Header>

What we’ve done here is added WS-SECURITY headers to the SOAP request.
Now the request should run just fine.

Note -
1) You need to add any fields you need to the request before they can be returned by Siebel CRM. e.g., <ContactFirstName/>
2) Its better to confirm data actually exists by logging through web-console.

SOAP is just a protocol for passing information, important is WSDL.

A WSDL(Web Service Description Language) file defines the contract for web service. It is just like the menu in a restaurant or a table of contents of a book. It tells about what is offered.

Without wasting more time, lets just get to work – understanding WSDL.

A WSDL file may be provided to you or it can generally be accessed by appending “?wsdl” at the end of a web service address.

A WSDL file is best-read bottom-up. It consists of abstract as well as concrete parts. Important parts to focus are (concrete)-

• wsdl:service – this is the tag which tells you about the name of the service. Consider it synonymous with class file.
• wsdl:port or port – this tag tells about the place where you can connect to. There can be multiple ports. Each port will have an “address” element. The location attribute specifies the endpoint address of the service. Pay attention the binding attribute of the port.
• wsdl:binding – this tag refers to the actual implementation of the service. It’ll be further up in the wsdl document. It contains the methods of the service.
• wsdl:operation – this tag refers to the operations provided by the port. wsdl:operation will contain soap:operation or similar tag depending on the binding.
• wsdl:input and wsdl:output – specifies the form of input/output. This is because it can be either literal or encoded.

Now focus on the Type attribute of the wsdl:binding, and trace it up in the document. It leads you to portType element. portType is the abstract part of the wsdl.

• portType – is something like an Interface, in fact it is called as interface in wsdl 2.0. It’ll have a no. of operations.
• operation – specifies input and output message variable types. These types are specified further up in the wsdl.

The first thing to do is add xdb.jar and xmlparserv2.jar file to your application lib. These contain the required class files for Oracle XML API. the jars can be found under your installation of oracle client lib folders. Also don’t forget to add oracle jdbc driver files – ojdbc6 or ojdbc14.

The code I setup was this -

XMLType reqInXml;
XMLType reqOutXML;
String atpInStr = "";
OpResponse output = null;

try {
   reqInStr = jaxbMarshalRequestToString(input);
   System.out.println("Input : " + reqInStr);
} catch (JAXBException ex) {
   logger.log(Level.SEVERE, null, ex);
} catch (IOException ex) {
   logger.log(Level.SEVERE, null, ex);
}

//input.getHeader().setNotes("THIS VALUE RETURNED BY FACADE : " + XXDS);
try {
   Connection con = XXDS.getConnection();
   //The IN parameter for stored proc is Oracle XDB XMLType
   reqInXml = XMLType.createXML(con, reqInStr);
   OracleCallableStatement stmt = (OracleCallableStatement) con.prepareCall("call DEMO_PROC.ProcessXMLRequest(?, ?, ?, ?)");
   stmt.setObject(1, reqInXml);

   //set out parameters
   stmt.registerOutParameter (2, OracleTypes.OPAQUE,"SYS.XMLTYPE");
   stmt.registerOutParameter(3, Types.INTEGER);
   stmt.registerOutParameter(4, Types.VARCHAR);

   stmt.executeQuery();
   int resultCode = stmt.getInt(3);
   String resultMsg = stmt.getString(4);
   System.out.println("result code : " + resultCode);
   System.out.println("result msg : " + resultMsg);
   if (resultCode == 101 || resultCode == 100){
      reqOutXML = XMLType.createXML(stmt.getOPAQUE(2));
      System.out.println("Output from ERP :" + reqOutXML.getStringVal());
      output = jaxbUnmarshalFromString(reqOutXML.getStringVal());
   }
}
....

This gave me “java.lang.ClassCastException: com.sun.gjc.spi.jdbc40.ConnectionHolder40 cannot
be cast to oracle.jdbc.OracleConnection” Exception. The connection was being returned from GlassFish JDBC ConnectionPool and was instance of OracleConnectionPooldataSource.
The FIX proved tricky, but in the end it was simple -
Do this -

OracleConnection oraCon = con.unwrap(OracleConnection.class);
//The IN parameter for stored proc is Oracle XDB XMLType
atpInXml = XMLType.createXML(oraCon, atpInStr);

Pass the cast instance of OracleConnection to the XMLType API.

Now, ActiveMQ uses Camel ESB for internal message routing, this proved useful. However try as I might, I couldn’t find a simple solution to what I wanted to achieve.

My target was to configure ActiveMQ broker which’ll act as a message router and decouple the 2 systems. Both would be unaware of its presence. Sounds really good in theory providing loose coupling.

But the difficult part was finding documentation. All the documentation available was about using ActiveMQ as JMS provider in GlassFish. Not what I was looking at.

So finally after much difficulty, I found the solution. It is simple but difficult to find.

Step 1) Copy imq.jar to ActiveMQ lib directory. The jar is found in GLASSFISH_HOME/imq/lib

Step 2) Add below lines to your activemq configuration file -

This line creates a default connectionFactory using host localhost and port 7676

<bean id="connectionFactoryOpenMQQueue"/>

This part creates a bean with credentials

<bean id="openMQQueueCredentials">
<property name="targetConnectionFactory">
<ref bean="connectionFactoryOpenMQQueue"/>
</property>
<property name="username">
<value>admin</value>
</property>
<property name="password">
<value>admin</value>
</property>
</bean>

This part created the actual scheme, namespace identifier for the connection.

<bean id="openMQQueue">
<property name="connectionFactory" ref="openMQQueueCredentials"/>
</bean>

Step 3) Add the routing instructions

This’ll consume message from GlassFishMQ and put it on ActiveMQ’s queue. You can check the messages at http://localhost:8161/admin

<route>
<from uri="openMQQueue:queue:q_user_info" />
<to  uri="activemq:queue:amq.temp.userinfo.queue"/>
</route>

Referred to this info on how to configure the ConnectionFactory - http://docs.sun.com/source/817-0355/adminobj.html

Using SVN is one option; other is use of Database Change Management / Database Diff
Some tools for that are –

1) Oracle Change Management Pack
From the docs –
There are three core capabilities of Change Management that allow developers and database administrators to manage changes in database environments:
• Baseline – A point in time of the definition of the database and its associated database objects.
• Comparison – A complete list of differences between a baseline or a database and another baseline or a database.
• Synchronization – The process of promoting changes from a database definition capture in a baseline or from a database to a target database

It allows us to take a baseline(snapshot) at a fixed time and then later we can see how the DB schema and objects have changed. The CMP can also generate DDL scripts, though I am not sure we would want to use it.

Details
• http://download-east.oracle.com/docs/cd/B19306_01/em.102/b31949/change_management.htm
•http://www.oracle.com/technology/products/oem/pdf/change-management-pack-11g-datasheet.pdf

2) PL/SQL Developer Compare User Objects feature
This is available from Tools -> Compare User Objects

3) Oracle SQL Developer Database Diff feature
This is available from Tools -> Database diff
http://www.oracle.com/technology/products/database/sql_developer/files/what_is_sqldev.html#copy See “Schema Copy and Compare”

#1 looks to be most versatile and flexible but DBA rights may be necessary.
#2 & 3 can be used by any developer. I think Oracle SQL Developer is easier and provides more options.

Using any of the above option can help in –
1) Identifying the changed objects and may also serve as a Check List before requesting deployment.

2) The developers concerned can take ownership of specific changed objects.