Table of Contents generated with DocToc
- MQEndPoint Policy
- Alternative to MQEndPoint Policy
- MQInputNode mode. How your messages will be processed
- MQOutputNode mode. How your messages will be processed
- Scenario full queue MQOutputNode node
- MQ Output
- MQ Reply
- Publication Node
- MQRFH2 Tree
In ACE you create an MQEndPoint Policy. If the flow within the Application contains an MQ Input node then add the MQ Policy to it's properties in the Toolkit. The same applies to the MQ Output node. This way the Application can then reference this policy. You need to deploy the Application ALONG with the MQEndPoint Policy or deploy the Policy prior to deploying the Application.
If you forget to update MQ nodes with an MQEndpoint policy and the Integration server server.conf.yaml does not have a default queue manager then how will it connect to the queue manager? You will see the message:
> BIP1361E Message flow node 'MQ Input',
> 'com.udemy.ace12_26.MQEndpointTest#FCMComposite_1_1' in Message flow
> 'com.udemy.ace12_26.MQEndpointTest',
> 'com.udemy.ace12_26.MQEndpointTest' requires Policy 'Default' of
> type 'MQEndpoint' which is not deployed.
If you opt to not using a policy your Integration Server OR Integration Node should associate with a QMGR if your flow contains MQ nodes (MQ Input, MQ Output, MQ Get).
You need to add the Queue Manager details to the server.conf.yaml or node.conf.yaml in order associate the Integration Server/Node with the Queue Manager.
$ grep defaultQueueManager iServer/server.conf.yaml
defaultQueueManager: 'QM1'
Then restart iServer.
If you then deploy and see this error again:
> BIP1361E Message flow node 'MQ Input',
> 'com.udemy.ace12_26.MQEndpointTest#FCMComposite_1_1' in Message flow
> 'com.udemy.ace12_26.MQEndpointTest',
> 'com.udemy.ace12_26.MQEndpointTest' requires Policy 'Default' of
> type 'MQEndpoint' which is not deployed.
That means your MQ Input or MQ Output nodes properties still reference an MQEndpoint policy. Remove those property values.
If you restart the IServer, the deployment works fine.
Transaction Mode of MQ Input Node (Part1) Under Properties then Advanced there is a Transaction mode which relates to how your messages are going to be processed.
This could be Yes (message is received but if there is an exception the original message will go to a backout queue; which if defined is at the queue level or DLQ; defined at QMGR level else it will loop), No or Automatic. Imagine there is a problem in the compute node or suppose the queue on the MQ output has a problem then the flow returns the message to the MQInput node. If the IN queue has a backout queue defined the message ends up on it. If not defined, then it goes to DLQ defined at QMGR level.
If no DLQ then continuous loop. If while it loops I define a DLQ for the QMGR then looping stops as the message goes to the DLQ.
If a DLQ or backout queue is defined but I change the Default persistence of the IN queue to Not persistent then the message is discarded.
If a DLQ or backout queue is defined but I change the Default persistence of the IN queue to Persistent then the message is discarded if the sending application has set persistence to No thereby overriding the queue setting.
An exception could occur if the wrong data source is defined at the Compute Node in between the MQInputNode and MQOutputNode. The MQInputNode has a Basic tab under properties which has a Queue Name field. If say IN.LQ is the value of that field then you need to define the backout queue for that queue using MQ Explorer.
If the Transaction mode is Automatic then it will inherit from the MQInput node.
If the Transaction mode is Yes then it will wait for the whole flow (Transaction) to complete before the MQPUT results in a message existing on the output queue. The whole transaction is rolled back.
If the Transaction mode is No then it will NOT wait for the whole flow (Transaction) to complete. The MQPUT will complete and a message put on the output queue
Look for "Resolving implementation problems when developing message flows" in IBM's online ACE page. Then look for Messages enter the message flow but do not exit
DEC212020-IIB FixHTTPProdCon_ESQLThrow 33:52 minutes shows you how the flow looks.
FUNDAMENTAL: In a flow even if you have 30 nodes in a message flow, if any of the nodes throws an error, the transaction is rolled back to node 1 in your message flow.
When a rollback occurs the exception is in the ExceptionList tree at the end. If the first node's catch terminal is connected then the Message and ExceptionList trees are sent to the node connected to catch. That node could be a transaction node that reads the exception and maybe an MQOutput node that puts the message on a queue. That way the message is not lost.
If catch is not connected then it will try the Failure node. It will send the Message tree but not the ExceptionList. Instead of the ExceptionList it will send the default exception which will say "you did not handle the Catch terminal".
Another scenario is when the MQPUT command to the output queue that is defined on the MQOutput node may not successful because the queue is FULL. The Failure terminal of the MQInput node has not been connected. Non-transactional messages are discarded. Whenever Transaction mode of the MQInput node is automatic, then transactionality is defined by persistence of the queue for the node. When queues are defined the Default persistence is 'Not Persistent'.
So the behaviour is the same as when Transaction mode is 'No'. This means that, when the exception is not handled, the non-transactional messages are discarded.
When Transaction mode is Automatic and Default Persistance property on the IN queue is Persistent then (if DLQ on queue manager is defined) then the message persists to the DLQ.
Sometimes the app sending the message to the flow has a Persistent Msg setting:
If you select No then it will override the queues "Persistent" setting and the message is lost.
When the transaction mode setting for MQInput node is Yes, MQOutput node is Automatic (transactionality derived from the Input node), Input queue IN's Default persistence is Not persistent, OUT queue is same as IN.
If I change the persistence to Persistent for the OUT queue, it makes no difference so I reverted. If I change the persistence to Persistent for the IN queue then I see in the syslog:
"BIP2643W: Unable to commit a WebSphere MQ transaction; MQCMIT
failed; queue manager=QM3, MQCC=1, MQRC=2003."
"BIP2648E: Message backed out to a queue; message flow node
'EsqlThrowStmt.MQ Input'."
The message has been backed out to the DLQ for the QMGR.
DEC292020-IIB Timeout_LabelRouting_MultiQueues
You can send the message to multiple queues in ESQL. You need to change the Compute mode to LocalEnvironment and Message in the Compute node properties as well as the MQ Output's Destination mode to Destination List. Use the command:
SET OutputLocalEnvironment.Destination.MQ.DestinationData[1].queueName='OUT';
SET OutputLocalEnvironment.Destination.MQ.DestinationData[2].queueName='OUT1';
- Reference
- LocalEnvironment tree
JAN042021-IIBFileNodeBARMQReplyTryCatchFlowOrder 24 minutes.
The MQ Reply node and the MQ Output nodes both PUT messages on a queue. Whereas the queue for the latter is defined at the IIB layer, for the latter the client system sets the name of the queue the flow has to reply to in the HEADER. RFHUTIL allows you to set the ReplyToQ header setting.
This node handles publishing to the Topic within MQ. MQ can contain subscriptions that will listen to what is published if the topic string matches.
- REFERENCE MQRFH2 Udemy
The MQRFH2 header is used to pass messages to and from an integration node that belongs to IBM® Integration Bus.
It allows downstream systems (or other IIB flows) to understand the message domain and properties.
In a message, the MQRFH2 header follows the WebSphere® MQ message descriptor (MQMD) and precedes the message body, if present.
What is the use of MQRFH2 header? You can pass application data to other flows if the protocol is MQ.
The Environment Tree is in memory and holds temporary data used within a flow (not transmitted), inside the message flow context whereas MQRFH2 is in the message itself and carries metadata and message properties across systems (in transit).
In the example below:
Environment variable scope is in the message flow. The environment variables defined in the first Compute node are available to the Compute1 node. But these environment variables are not available to the Compute in the msglow that the 'call Service' calls (another message flow in the same Application called HTTPService.msgflow). The MQ Output puts a message to a queue called test2. That then triggers another message flow called SecondFlow.msgflow which has an MQ Input to read the message PUT on test2. Again SecondFlow.msgflow cannot access the Environment variables from FirstFlow.msgflow. So how can you pass the data from one flow to another? The answer is MQRFH2 header.
CREATE COMPUTE MODULE FirstFlow_Compute
CREATE FUNCTION Main() RETURNS BOOLEAN
BEGIN
CALL CopyMessageHeaders();
SET Environment.Variables.orderPerson = InputRoot.XMLNSC.shiporder.orderperson;
Then in the later Compute1:
CREATE COMPUTE MODULE FirstFlow_Compute1
CREATE FUNCTION Main() RETURNS BOOLEAN
BEGIN
CALL CopyMessageHeaders();
DELETE FIELD OutputRoot.HTTPResponseHeader; -- this came from the HTTPService.msgflow and is not needed downstream.
SET OutputRoot.XMLNSC.Acknowledgment = 'Sent to Downstream';
PROPAGATE TO TERMINAL 'out1'; -- goes to HTTP Reply so the Acknowledgment goes back to the client that sent the HTTP request. This will also delete all the messages in the OutputRoot unless you add DELETE NONE to the PROPAGATE
SET OutputRoot.Properties = InputRoot.Properties; -- need to construct the tree again after the PROPAGATE deletes the message tree above.
SET OutputRoot.MQMD = InputRoot.MQMD;
SET OutputRoot.MQRFH2.usr.oPerson = Environment.Variables.orderPerson; -- Latter was set in the first Compute. Upto usr is standard syntax. The usr is for storing User-defined application data.
SET OutputRoot.MQRFH2.usr.shipto = InputRoot.XMLNSC.ServiceResponse; -- ServiceResponse comes from the Compute in the HTTPService.msgflow.
In the HTTPService.msgflow:
SET OutputRoot.XMLNSC.ServiceResponse = InputRoot.XMLNSC.shiporder.shipto;
Once the 2nd message assembly reaches the 2nd flow via MQ protocol a new Output message assembly is created using the MQRFH2 header values.
CREATE COMPUTE MODULE SecondFlow_Compute
CREATE FUNCTION Main() RETURNS BOOLEAN
BEGIN
-- DELETE FIELD InputRoot.MQRFH2;
IF EXISTS(InputRoot.MQRFH2[]) THEN
SET OutputRoot.XMLNSC.FinalResponse.orderPerson = InputRoot.MQRFH2.usr.oPerson;
SET OutputRoot.XMLNSC.FinalResponse.ShipTo = InputRoot.MQRFH2.usr.shipto;
END IF;