IBM MQ Little Gem #31: Publisher's CorrelId

This is part of a series of small blog posts which will cover some of the smaller, perhaps less likely to be noticed, features of IBM MQ. Read other posts in this series.

Sub Message Correl ID.jpgA question that comes in fairly regularly on MQ fora is about the message ID and the correlation ID in a message delivered to a subscriber.

Message ID

The message ID in a message delivered to a subscriber is unique to that message, so that every message in the system is uniquely identified. That is, even if there are several subscribers that each receive a copy of the same published message, the message ID in each copy will be different.

Correlation ID

The way the correlation ID works can follow one of two patterns. Either the correlation ID is the subscriber's unique correlation ID for every message, or the correlation ID is the one set by the publishing application. The pattern which is in effect is controlled by the creator of the subscription.

Subscriber's Correlation ID

This pattern is the default behaviour. When a subscription is created it is assigned a unique correlation ID which is then placed into each message delivered to that subscriber. This can be useful if several subscribing applications are sharing the same queue as they can each do an MQGET by CorrelId to retrieve their own messages.

An application can discover what the correlation ID assigned to it is by using the MQSD.SubCorrelId field on output from an MQSUB call.

MQSUB(hConn,
      &SubDesc,
      &hSub,
&hObj,
&CompCode,
&Reason);
if (CompCode == MQCC_OK)
{
memcpy(MsgDesc.CorrelId, SubDesc.SubCorrelId, MQ_CORREL_ID_LENGTH);
gmo.MatchOptions = MQMO_MATCH_CORREL_ID;
:
MQGET(hConn,
hObj,
&MsgDesc,
&gmo,
...

You can also see the correlation ID assigned to a subscription using the DISPLAY SUB command, and inspecting the DESTCORL attribute.

SUBID(414D51204D5147312020202020202020AF9C6E5A20002303)           SUB(ApplePrices)
TOPICSTR(Price/Fruit/Apples)                TOPICOBJ( )           DISTYPE(RESOLVED)
DEST(APPLE.FRUIT.Q)   DESTQMGR(MQG1)        PUBAPPID( )           SELECTOR( )
SELTYPE(NONE)         USERDATA( )
PUBACCT(1601051500000083E2174C2DDA25A59C18E568EE03000000000000000000000B)
DESTCORL(414D51204D5147312020202020202020AF9C6E5A20002303)        DESTCLAS(PROVIDED)
DURABLE(NO)           EXPIRY(UNLIMITED)     PSPROP(MSGPROP)       PUBPRTY(ASPUB)
REQONLY(NO)           SUBSCOPE(ALL)         SUBLEVEL(1)           SUBTYPE(API)
VARUSER(FIXED)        WSCHEMA(TOPIC)        SUBUSER(morag)        CRDATE(2018-01-29)
CRTIME(17:04:13)      ALTDATE(2018-01-29)   ALTTIME(17:04:13)

Publisher's Correlation ID

Alternatively a subscription can request that it gets to see the correlation ID set by the publisher. In order to do this it must zero out it's subscription correlation ID. Here's how you'd do that in an application.

MQSD   SubDesc = {MQSD_DEFAULT};

SubDesc.ObjectString.VSPtr = “Price/Fruit/Apples”;
SubDesc.ObjectString.VSLength = MQVS_NULL_TERMINATED;
SubDesc.Options               = MQSO_CREATE
                              | MQSO_SET_CORREL_ID;
memcpy(SubDesc.SubCorrelId, MQCI_NONE, MQ_CORREL_ID_LENGTH);

MQSUB(hConn,
...

And here's how you would do that with an administratively defined subscription:-

DEFINE SUB('ApplePrices') +
       TOPICSTR('Price/Fruit/Apples') +
       DEST(APPLE.PRICE.Q) +
       DESTCORL(0)

Now if you need to correlate between publishing and subscribing applications you know how to get hold of that ID.


Morag Hughson is an MQ expert. She spent 18 years in the MQ Devt organisation before taking on her current job writing MQ Technical education courses with MQGem. She also blogs for MQGem. You can connect with her here on IMWUC or on Twitter and LinkedIn.
8 Comments
3 Likes

Same corelation id in DESTCORL attribute for all API sub

January 29, 2018 03:07 AM by Mangesh Khandale, S/w Developer

Hi Morag,

I have tested this scenario in our test environemnt.  However I have observed one strange thing . When I checked for total subscription present , below things I have observed :

For all subscriptions SUBTYPE(API)
- All API subscriptions has different SUBID but have similar DESTCORL as :
 DESTCORL(414D51580000000000000000000000000000000000000000)

- All these API subscriptions are made by same Application conneted to the QM. Is this the reason that all these subs have same COREL ID?

For all ADMIN subs

- All the admin subs has different SUBID & DESTCORL ids.

However each individual ADMIN sub has same SUBID & DESTCORL id.
Is this obvious behaviour ?

 

Thanks ,

Mangesh

 

Reply to Mangesh

January 29, 2018 03:56 PM by Morag Hughson

Hi Mangesh,

It is normal for the SUBID to match the DESTCORL for both SUBTYPE(ADMIN) and SUBTYPE(API) subscription, if the queue manager has been left to generate the correlation ID. It already has a unique string generated for the SUBID, so it uses the same one for the correlation ID.

If the correlation ID is different from the SUBID, it is probably because the application is setting it to a value it needs. Perhaps it needs to do an MQGET by CorrelID to retrieve these messages. What is the application in question? Can you ask the developers why they have set a CorrelID? You could run Activity Trace for that application and see whether it is using the MQSO_SET_CORREL_ID option if you're curious.

Cheers,
Morag

Version dependencies?

January 30, 2018 01:16 PM by Peter

Does this feature require any particular version of MQ? When was the DESTCORRELID option introduced?

Reply to Peter

January 30, 2018 03:19 PM by Morag Hughson

DESTCORL has always been part of DEFINE SUB. It was added in V7.

SubCorrelId is a field in the V1 MQSD. It was added in V7.

Reply To Morag

February 1, 2018 06:30 AM by Mangesh Khandale, S/w Developer

Thanks Morag.

Your exaplaing is neat and clear to me now.

Menwhile as you asked me , the application we are using in above case is , MQ connector state  in Datastage tool . I have checked all its properties and nowhere its stating that it is to set corelation id. I think this is the tool's out-of-the-box behaviour that can be changed.

ALL API SUB HAS TOPICOBJ(SYSTEM.BROKER.DEFAULT.SUBPOINT)

February 1, 2018 06:40 AM by Mangesh Khandale, S/w Developer

Hi Morag,

I am now being keenly interested to know the PUBSUB mechanisam.
During my further analysis I found , all the API subscriptions that our applicaiton creates has TOPICOBJ(SYSTEM.BROKER.DEFAULT.SUBPOINT) automatically.
When I see what this topic exactly points to , below is the definition :

display topic(SYSTEM.BROKER.DEFAULT.SUBPOINT)
    45 : display topic(SYSTEM.BROKER.DEFAULT.SUBPOINT)
AMQ8633: Display topic details.
   TOPIC(SYSTEM.BROKER.DEFAULT.SUBPOINT)   TYPE(LOCAL)
   TOPICSTR()
   DESCR(Default RFH2 subscription point for queued Pub/Sub interface)
   CLUSTER( )                              CLROUTE(DIRECT)
   DURSUB(ASPARENT)                        PUB(ASPARENT)
   SUB(ASPARENT)                           DEFPSIST(ASPARENT)
   DEFPRTY(ASPARENT)                       DEFPRESP(ASPARENT)
   ALTDATE(2017-06-06)                     ALTTIME(12.33.20)
   PMSGDLV(ASPARENT)                       NPMSGDLV(ASPARENT)
   PUBSCOPE(ASPARENT)                      SUBSCOPE(ASPARENT)
   PROXYSUB(FIRSTUSE)                      WILDCARD(BLOCK)
   MDURMDL( )                              MNDURMDL( )
   MCAST(DISABLED)                         COMMINFO( )
   USEDLQ(ASPARENT)                        CUSTOM( )

Where as if for ADMIN subscriptions that I have created , TOPICOBJ() its blank becasue I doint spcfied any during creation.

By looking ate the description of the above topic definition its look like something to do with queued pu/sub. But not able to understand the significance of it.
It will be great if you can explain this .

Thanks ,
Mangesh

Please explain one more default subscsripiton

February 1, 2018 06:56 AM by Mangesh Khandale, S/w Developer

Can you please also explain the significance of below subscription ? When exactly  this subscription will be useful.

display sub('QM1 SYSTEM.BROKER.INTER.BROKER.COMMUNICATIONS 414D51590101000000000000000000000000000000000000 SYSTEM.BROKER.ADMIN.STREAM MQ/QM1                                      /StreamSupport')
   TOPICOBJ(SYSTEM.BROKER.ADMIN.STREAM)    DISTYPE(RESOLVED)
   DEST(SYSTEM.BROKER.INTER.BROKER.COMMUNICATIONS)

As per my understanding if any publish  a message on this Queue Manager (on any topic) , it will be first subscribed by this subscription.
Then this message will go to the destination queue SYSTEM.BROKER.INTER.BROKER.COMMUNICATIONS.
To this destination queue , there is a process  pubsub controller - 'amqfqpub' always listening .
This pubsub controller then read message from this queue and do the below :
#1. Check which topic it has mentioned in the messages.
#2. Check if any active subscripitons or durable subscripitons are there.
#3. If any , then copy the messages to these numbers & send it to these subscripitons

Not sure if my understanding is correct .

Reply to Mangesh

February 2, 2018 03:15 AM by Morag Hughson

Before MQ V7, the Message Broker (aka IIB) had a Pub/Sub engine which was interfaced via an RFH2 format message. When MQ added native pub/sub in V7 support for the "Queued Pub/Sub interface" was maintained so that the pub/sub engine could be removed from IIB, and it could use the MQ pub/sub engine/ The SYSTEM.BROKER.DEFAULT.SUBPOINT is releated to that interface. You can read more about subpoints in Knowledge center at Subscription points.

Inter Broker Pub/Sub is the mechanism by which publications are sent through a topology of queue managers to satisfy subscriptions elsewhere in the network. For example if an application subscribes to Topic1 on QM1 and another application subscribes to Topic1 on QM2, and then on QM3 an application publishes to Topic1, you'd probably want the subscriptions on QM1 and QM2 to receive copies of that message.

There are two different ways you can achieve this, inter-broker pub.sub which is compatible with the queued pub/sub interface, and topic clustering which was new in MQ V7. The former is what the default subscription you see is used for.

Cheers,
Morag

Recent Stories
IBM MQ Little Gem #31: Publisher's CorrelId

IBM MQ Little Gem #30: MQ_CONNECT_TYPE

IBM MQ Little Gem #29: STOP CHANNEL parameters