A Powerful Data Audit Combination – IBM Streaming Queues and Prolifics GTM

What are streaming queues, and what is GTM?

Released in July of last year, IBM MQ 9.2.3 added the “streaming queues” feature to IBM MQ queue managers. This feature allows you to configure a queue to put a near-identical copy of every message to a second queue (the streaming queues). 

As noted by IBM, “streaming queues can be useful in certain scenarios, where you need to create a copy of your messages.” For example: 

• Performing analysis on the data going through the system 
• Storing messages for recovery at a later time 
• Capturing a set of messages to use in development and test systems 

Users can configure streaming queues to ensure that the original messages remain unaffected by the streaming process. This means your core business applications are not impacted by streaming. Additionally, the streaming queues feature only delivers messages to stream queues if the original message was also delivered. 

Prolifics’ Global Transaction Monitor (GTM) offers a customized, document-centric, end-to-end transaction tracking framework and automated managed file transfer. GTM captures, records, and exposes events, made visible via customized dashboards and tracked in accordance with user-defined performance indicators. In addition, it offers reporting and filtering to help with self-service requests to track transactions and documents. 

The GTM solution is built with the latest and newest technologies: IBM MQ Advanced, Mongo document-centric database, IBM Transformation Extender Advanced (ITXA), and Python programming language. 

A match made in “audit heaven”

Honda Bhyat is Prolifics Sr Advisor, Integration and Digital Transformation (DX) Practice. “(Streaming queues) is effectively a record of anything going through the system. You just make a definition change and it’ll give you an exact copy of a message that you placed on a queue for whatever purpose. And so we can, using GTM, simply define that streaming queue and get a copy logged into a Mongo database. I now have an immutable copy of the data stored away that I can analyze and report on to get more insights”  

Bhyat notes that the real benefit of this IBM/Prolifics combination is in its audit capabilities. “I believe almost every organization will jump at the opportunity to understand what is moving through their MQ network for high value messages. An auditor can have eyes on everything going out or coming in for certain message queues. All kinds of metrics that can be measured and accounted for in an audit, or in any exercise that wants to understand the providence of data and its movement. 

“The streaming queues feature makes it so simple. There’s really very little overhead because it’s just a message that it’s already being created – there’s already the same copy of the message. And storing it in a database makes these messages – that we’ve communicated back and forth – so much more accessible. So, we’re coming up with implementations that will be valuable to many different use cases” 

If you’re an IBM MQ user and want to understand how the streaming queues/GTM combination can work for you, you can email Honda Bhyat at honda.bhyat@prolifics.com or solutions@prolfics.com. 

About Prolifics

Prolifics is a global digital engineering and consulting firm helping clients navigate and accelerate their digital transformation journeys. We deliver relevant outcomes using our systematic approach to rapid, enterprise-grade continuous innovation. We treat our digital deliverables like a customized product – using agile practices to deliver immediate and ongoing increases in value. 

We provide consulting, engineering and managed services for all our practice areas — Data & AI, Integration & Applications, Business Automation, DevXOps, Test Automation, and Cybersecurity across multiple industries — at any point our clients need them. 

Vision to Value. Faster. It’s not just the Prolifics’ tagline, it’s what drives us. Email us at solutions@prolifics.com or visit us at prolifics.com.