Emitting Real-Time Events from IBM BPM for Custom BAM Solution

Overview

Business activity monitoring (BAM) and reporting in real time are some of the essential features that most of the firms around the world (including large banking/financial, healthcare, insurance, automotive, retail, manufacturing, and entertainment organizations) are looking for.

BAM plays a vital role in BPM world. Among the BPM products in the market, IBM BPM is one of the most successful and widely used. IBM Business Monitor is a real-time BAM product provided by IBM. It is mostly used by customers alongside IBM BPM to monitor business processes.

However, some customers who are using IBM BPM to implement the business processes might be interested in implementing their own custom BAM solution. The key part in implementing a custom BAM solution is really getting the monitoring data in real time from the business processes in IBM BPM.

This blog primarily showcases how to send/extract the real-time monitoring data from IBM BPM by leveraging its out-of-the-box eventing mechanism using Dynamic Event Framework (DEF) to implement a custom BAM and reporting solution.

Configure BPM to Emit Real-Time Events

Follow the instructions to configure IBM BPM infrastructure to emit/send default events from BPM processes to MQ for implementing a custom BAM and reporting solution.

Note:

This solution is applicable for IBM BPM version 8.5.5 and above.

This solution uses MQ for messaging purposes. MQ is the recommended messaging provider here for real-time eventing (instead of using the default messaging provider with internal SIB queues). This is mainly due to the fact that being an external messaging infrastructure, it significantly reduces the load/resource utilization in BPM infrastructure. The custom event processing modules for BAM (which essentially read messages from MQ) can also be externalized from BPM.

  1. If the version of IBM BPM is 8.5.5, check if the following APAR is applied:

https://www-01.ibm.com/support/docview.wss?uid=swg1JR51784

 

If not, follow the steps in the above link and apply the APAR.

 

Note: BPM v8.5.6 and above should contain this APAR by default.

 

Note: Following steps should be performed in the deployment manager in a clustered environment.

 

  1. Create a JMS queue connection factory.
  1. Log into the WebSphere administrative console of DMGR to create a cell scoped queue connection factory.
  2. Create a queue connection factory named “BPMEventEmitterMQCF” and JNDI “jms/BPMEventEmitterMQCF.” Set the provider to “WebSphere MQ messaging provider.” Assign the correct cell scope.

        

  1. Enter the following MQ connectivity details.

 

 

  1. Create a JMS queue.
  1. Create a cell scoped JMS queue with name set as “BPMEventEmitterMQ,” the JNDI name set as “jms/BPMEventEmitterMQ,” and the provider set as “WebSphere MQ messaging provider.”

Refer to the screenshots below.

 

  1. Copy the jython scripts SampleConfigureEventsToJMS.py and SampleReloadDEF.py from the following location to a /tmp location:

<BPM-install_root>/BPM/Lombardi/tools/def

 

  1. Modify SampleConfigureEventsToJMS.py to define queue connection factory, queue, and subscriptions as shown below:
     

defListenterId – jmsBPMEventEmitterListenerId_MQ1

This is a string value that uniquely identifies this listener.

 

eventQueueJndiName – jms/BPMEventEmitterMQ

This is a string value that refers to the JNDI name of the queue resource created in WebSphere.

 

eventQueueCFJndiName - jms/BPMEventEmitterMQCF

This is a string value that refers to the JNDI name of the queue connection factory resource created in WebSphere.

 

eventQueueCF_AuthorizationAlias - CellAdminAlias

This is a string value that refers to the authorization alias created in WebSphere.

 

Each subscription in the subscriptions array is a single string with a “/” separator for each of the seven part keys. A comma is used to separate each subscription. The seven-part keys are:

 

Process Application Acronym / Version / ComponentType / Component Name / Element Type /Element Name / Nature

 

To listen for every event for all applications, use the wildcard character as shown in the following example:

 

subscriptions=[

'*/*/*/*/*/*/*'

]

 

The following example shows how you might register to receive events for the Hiring Sample:

 

subscriptions=[

'HSS/*/BPD/*/PROCESS/*/*',

'HSS/*/BPD/*/ACTIVITY/*/*',

'HSS/*/BPD/*/GATEWAY/*/*',

'HSS/*/BPD/*/EVENT/*/*'

]

 

Note: If events need to be emitted and subscribed for multiple applications, then those application details need to be added in the Subscriptions array similar to HSS.

 

  1. Execute the SampleConfigureEventsToJMS.py script using the command

“wsadmin –lang jython –f \tmp\ SampleConfigureEventsToJMS.py” (use the credentials for the cell administrator in lieu of celladmin).

 

Once it is executed, validate if the defconfig.xml file has been created in the following location:

<BPM-install root>/profiles/<DMGR>/config/cells/<cellname>/

 

  1. Execute the SampleReloadDEF.py script using the following command:

“wsadmin –lang jython –f \tmp\SampleReloadDEF.py” (use the credentials for the cell administrator in lieu of celladmin). 

 

Note that the SampleReloadDEF.py command does not need to be modified.

Note: Normally SampleReloadDEF.py should dynamically change the DEF configuration and events should start being emitted. If for some reason they are not, then do a full sync from DMGR and restart the clusters (App Target, Support and Messaging clusters).

 

Note:

If eventing needs to be turned off for some reason (in case of any infrastructure issues), then update the SampleConfigureEventsToJMS.py script to empty the subscription array, as shown below:

 

subscriptions=[

]

 

After updating the SampleConfigureEventsToJMS.py script, execute step 6, then delete the defconfig.xml file (from <BPM-install root>/profiles/<DMGR>/config/cells/<cellname>/).

 

Then execute step 7, which should turn off eventing dynamically. For starting it again, follow steps 2 through 7.

Next Steps (to Implement a Custom BAM Solution)

While BPM infrastructure is configured to emit/send real-time events by following the steps above, there are three major activities that must be done to implement a custom BAM solution.

1. Enabling real-time eventing will emit the default process and task lifecycle events. However, for emitting the application data for every process instance, add the tracking events in the BPD wherever needed (based on the monitoring and reporting requirement) to capture the application data from the process implementation side. This needs to be done for all the processes to be monitored.

2. Implement a custom event processor (a Java module with MDB) to listen/read messages from MQ, parse the XML messages/events by understanding the XML message structure/schema (see the documentation available in the link below), and write the process and task/activity level data to separate tables (the table definition should include the fields for process/task metadata and application/business data based on the reporting requirements).

https://www.ibm.com/support/knowledgecenter/en/SSFPJS_8.5.5/com.ibm.wbpm.main.doc/topics/cmon_monitor_eventoverview.html

This event processor needs to be continuously running to drain the events in MQ.

3. Once the event processor is implemented, the next important activity is to use any reporting tool like Cognos, Tableau, etc. to build and display the reports by reading the data from the database tables (the process and activity tables mentioned in step 2) based on the reporting requirements.

 

Parthasarathi Jayapathi
Solution Architect

Parthasarathi Jayapathi is a Solution Architect at Prolifics specializing in BPM, SOA and Java/JEE technologies. He has over 11 years of consulting experience in the IT industry in domains including Healthcare, Banking and Media/Entertainment. Partha has extensive experience across different project lifecycle phases such as architecture, design, development, deployment and production support.