WHINT AMQP Adapter (On-Premise)

Functionality

This JCA Adapter for SAP PI / SAP Process Orchestration (PRO) supports the Advanced Message Queuing Protocol (AMQP), a reliable way to establish a properly decoupled application integration using message queues and topics. AMQP defines both the network protocol and the server-side services through a defined set of message capabilities called AMQ model, which consist of a set of components that route and store messages within the broker service and a network wire-level protocol, that lets client applications talk to the server and interact with the AMQ model it implements.

It connects asynchronously to the followings Brokers by sending and receiving messages in a reliable way:

  • Microsoft Azure Service Bus (Cloud)
  • RabbitMQ (On-Premise/Cloud)

Prerequisites

  • Works with SAP PI 7.1 and higher (incl. SAP Process Orchestration 7.50)
  • Deploy the SCA file provided by Whitepaper InterfaceDesign using NWDS/JSPM/SUM/Telnet
  • Import the TPZ file provided by Whitepaper InterfaceDesign into the ESR

Usage

  • Create a new sender or receiver communication channel and select the adapter “AMQP” from namespace “http://whint.de/xi/AMQ” (Software Component WHINT_AMQ 2016.04 of whitepaper-id.com)
  • Transport Protocol
    • Select one of the following transport protocols
      • MS Azure – to connect with a Microsoft Azure Service Bus
      • RabbitMQ – to connect with a RabbitMQ Broker
  • Message Protocol
    • The message procotcol is preselected based on the transport protocol. The following are available so far:
      • AMQP 1.0 – This is the message protocol for MS Azure
      • AMQP 0-9-1 – This is the message protocol for RabbitMQ
  • Other Options
    • Host & Port
    • Virtual Host (RabbitMQ)
    • SSL/TLS enabled
    • Queue / Exchange Name
    • Topic Support
      • Subscription ID (MS Azure)
      • Exchange Type: direct/headers/fanout/topic (RabbitMQ)
      • Routing Key (RabbitMQ)
    • Consuming Interval (secs/msecs)
    • Authentication based on User/Password
    • Quality of Service (EO/EOIO with queue name)
    • Set/Use Adapter-specific Message Attributes (ASMA)
      • Namespace: http://whint.de/xi/AMQ
      • externalMessageID – Message Id received from broker
      • CreateDateTime – Timestamp of message creation
      • QueueExchangeID – Name of the Queue / Exchange Name
      • RoutingKey (only RabbitMQ)

Example

AMQo1

AMQo2

AMQo3

WHINT MessageMissing Alert

Functionality

How can we make sure if a message was processed at all with SAP PI/PO? Today, no alert means no error.
But what if the sender system did not place the file or send the message? You will find out, when the business will ask for missing data…
WHINT MessageMissing Alert is a solution that runs pro-actively as a job on your system (e.g. every hour during business hours or every month on the first) and checks in the message (performance) monitor if the interface ran or not. In case no message was found, an e-mail will be sent to a receiver.


Prerequisites

  • Works with SAP NetWeaver PO (PI single-stack) version 7.3 EHP1 and higher (including 7.50)
    • If you read from more than one Adapter Engine and you enforce SSL, you need to apply SP15 for 7.31 / SP10 for 7.40
  • Create a new User with the following standard role (or equivalent custom role): SAP_XI_MONITOR_J2EE
  • Activate the performance monitoring in NWA
    • Configuration > Infrastructure > Java System Properties
    • Services > XPI Service: AF Core
    • Properties > profile.performance.runtime = true
    • Configure the interval to keep the last 31 days of the message processing by executing the following URL on your PI/PO system: http(s)://host:port/mdt/performancedataqueryservlet?PeriodConfig=DAILY=31
  • Import the Software Component into your ESR provided by Whitepaper InterfaceDesign
  • Deploy the SCA/EAR file provided by Whitepaper InterfaceDesign using NWDS/JSPM/SUM/Telnet
  • Configure the Scenario by installing the Integration Scenario in NWDS (via iFlows) or in Integration Directory Swing Client. Per default “WHINT_MMA” is defined as the sender business component.

MMA1


Usage

  • Schedule the job from NWA -> Operations -> Jobs -> Java Scheduler
  • Add Task with Job name MessageTriggerJob
  • The solution is using WHINT MessageTrigger Job which can generically send an XML message to the PI/PO runtime.
  • Parameters:
    • ScenarioID: WHINT_MMA (actually this value can have an
    • ScenarioSender: MMA_TEST1 (this value has to match the IntegratedConfiguration/iFlow configuration)
    • Param01: Interface=<the interface name of the message you look for. “*” is the wildcard.>
    • Param02: InterfaceNS=<the interface namespace of the message you look for. “*” is the wildcard.>
    • Param03: SenderComponent=<the sender system of the message you look for. “*” is the wildcard.>
    • Param04: SenderParty=<the sender party of the message you look for. This parameter is optional.>
    • Param05: ReceiverComponent=<the receiver system of the message you look for. “*” is the wildcard.>
    • Param06: ReceiverParty=<the receiver party of the message you look for. This parameter is optional.>
    • Param07: Period=<DAY/HOUR/15MIN>
    • Param08: Frequency=<e.g. the amount of hours within the message was processed>. Possible values / Unit:
      • DAY: 1..31
      • HOUR: 1..24
      • 15MIN: 1..4
    • Param09: EmailRecipient=<E-Mail Address or list: receiver1@company.com;receiver2@company.com>
  • The sequence of the parameters is not important.
  • The scheduling should be done according to your business needs (hourly, Mon-Fri between 8:00 and 18:00)

MMA2

MMA3


Example

Finance data for reconciliation from a payment service provider shall be sent monthly from an SFTP server and posted into SAP accounting. The solution makes sure that the interface is running every month at least once or at a given day.

Another example is that the warehouse interface shall be active during business hours. Stock movement messages should be sent at least once every 30 minutes from SAP ERP to the AS/400 warehouse system. An immediate alert is sent out if no messaging takes place.

MMA4


 

WHINT MessageTrigger Job

Functionality

In case you want to start an interface without scheduling a polling sender adapter (e.g. file) and you need a reliable trigger for a message transfer, this solution helps to set up any scenario easily. An XML message with a generic structure is sent to the messaging runtime with the parameters you have defined in the trigger job. Based on the parameters you can proceed with any interfacing scenario (call a web service/REST/HTTP URL, read a file using WHINT FileReader Adapter, …).


Prerequisites

  • Deploy the SCA/EAR file provided by Whitepaper InterfaceDesign using NWDS/JSPM/SUM/Telnet
  • Define an Integrated Configuration (ICO) or iFlow that reflects the scenario

Usage

  • Schedule the job from NWA -> Operations -> Jobs -> Java Scheduler
  • Add Task with Job name MessageTriggerJob
  •  Parameters:
    • Scenario: <name/description of the scenario>
    • System: WHINT_MTJ
    • Further scenario parameters as name/value pairs
    • The scheduling should be done according to your business needs (hourly, Mon-Fri between 8:00 and 18:00)


Example

You need to pick up exchange rates without deleting them from the source folder, or you do not want to send a request message from SAP ERP to receive the latest updates (e.g. master data from a CMS). Set up the MessageTrigger Job in NWA and define the scheduling according to your needs.

Simple MFT with SAP PI/SAP PRO

MFT (Managed File Transfer) scenarios are not clearly described by SAP, so please follow this short description to set up a simple MFT scenario:

Scenario

Polling a source directory/folder via standard FILE/FTP(S)/SFTP adapter, delete or archive the file and write the file(s) to a target directory/folder.

Design Time

No activities needed/mandatory for SLD or ESR (!).
The Service Interfaces (Outbound/Inbound) can be designed of course, but this is not mandatory for the configuration/runtime.
As the file content is not XML, the modeling of a data type is not obligatory.
However, to apply a proper naming and reuse, it would be recommended to create a Software Component (e.g. MFT 1.0 of whitepaper-id.com) and create Dummy Service Interfaces (e.g. File_Out / File_In in a namespace e.g. urn:whint.de:mft).

Configuration

In the Integration Directory the routing configuration can be done based on the created Service Interfaces or by using Dummy Interfaces explicitly (iFlow in NWDS) or by manually entering the name of the Service Interface in the Integrated Configuration.
For the sender channel configuration make sure you read the Adapter Specific Message Attributes (ASMA) for the File Name to transport it accordingly as meta data to the target communication channel.

And then you are done, no rocket science here…

 

MFT – Managed File Transfer

Whenever possible, please try to avoid file-based integration (flat or not, going through the file system is not a good integration pattern), see blog post: Avoid using flat files!. However, sometimes it is still the best fit when the connected system does not provide a clean (RESTful or WebService-based) API. This blog series is about how to set up MFT scenarios with SAP PI/SAP NetWeaver Process Orchestration (PRO) in a smart way.

SAP PI/SAP PRO for MFT scenarios

WHINT FileReader Adapter (On-Premise)

Functionality

This Receiver Adapter is able to read a file from a directory (NFS/FTP/SSH) and process it further.

The standard SAP File Sender Adapter is only able to continuously poll a directory and delete/archive the file or re-send the file from a source folder. There are many scenarios where it makes sense to read a file from a directory event-driven or triggered by a sender system where the file has to remain at its source.

The FileReader Adapter works in a synchronous mode on the receiver side.
Queries can be done either

  • synchronously where the file is returned to the sender directly as a response message or
  • asynchronously where the file can be sent to any receiver by using the RequestResponseBean

WHINT FRA


Prerequisites

  • Deploy the SCA file provided by Whitepaper InterfaceDesign using NWDS/JSPM/SUM/Telnet
  • Import the TPZ file provided by Whitepaper InterfaceDesign into the ESR

Usage

  • Create a new receiver communication channel and select the adapter “FileReader” from namespace “http://whint.de/xi/FRA” (Software Component WHINT_FRA 2015.08 of whitepaper-id.com)
  • Transport Protocol
    • Select one of the following transport protocols
      • NFS – to connect with a local or network file system directory
      • (S)FTP – to connect with an FTP or SSH server
  • Message Protocol
    • You can choose the access method by selecting one of the Message
      • Channel Configuration – The file name and directory is configured in the communication channel directly
      • Query XML message – The file name is sent with an XML message. The format is visible in software component WHINT_FRA, message type FileReaderQuery (http://whint.de/xi/FRA). The directory can be sent along with the XML message or maintained in the channel configuration.
      • Dynamic Configuration – The file name is provided as an adapter-specific message attribute (ASMA): FileName (http://sap.com/xi/XI/System/File).  The directory can be provided as an adapter-specific message attribute (ASMA): Directory (http://sap.com/xi/XI/System/File) or maintained in the channel configuration.
  • Other Options
    • List Directory – The directory content (based on the given file pattern) is returned as a response XML message.
    • Append multiple files -If multiple files are found, the adapter returns them appended. XML files are enveloped in a response XML message, TXT files are simply appended. Binary files can not be appended.
    • Error message if no file is found – An error message is raised if the file can not be found. Otherwise the information is returned in a response XML message.

Example

WHINT_FRA_Config1 WHINT_FRA_Config2

Sync-Async Connectivity (SOAP 2 MAIL) with SAP PRO (w/out BPM)

Sync-Async Patterns with BPM on SAP NetWeaver Process Orchestration (PRO) are quite easy to implement.
However, when you want to design the integration flow as lean as possible, this document might be useful for you as the usage of the sync/async adapter modules is a bit tricky.


Scenario:

  • SOAP Sender (in our example WebShop) is calling SAP PRO synchronously
  • The request is forwarded asynchronously via E-Mail to the receiver (typically it would be e.g. IDoc to SAP ERP of course)

This is an example scenario of course. HTTP/SOAP on the (sync) sender side can be combined with any channel on receiver side (IDOC/SOAP/FILE/SFTP/MAIL/JMS/JDBC/…)


Adapter Modules used:

We are using the following (standard) adapter modules:

  • AF_Modules/RequestOnewayBean (to convert the synchronous message to an asynchronous one)
  • AF_Modules/DynamicConfigurationBean (to store the original message id in the ASMA of the sender channel and read it in the receiver channel to set the correlation)
  • AF_Modules/WaitResponseBean (to wait for the reply of the receiver channel)
  • AF_Modules/NotifyResponseBean (to send back the message to the waiting sender channel using correlation)

ESR Content (sender side):

  • Service Interface Sync Out
  • Service Interface Sync In (Dummy)
  • Operation Mapping (Service Interface Sync Out -> Sync In)
  • Message Mapping (Input Message Type -> Output Message Type)

DIR Configuration:

  • Sender System
    • SOAP Sender Channel
    • SOAP Receiver Channel
  • Receiver System
    • MAIL Receiver Channel

Screenshots:

It is of course not necessary to do this in NWDS but I really started to like it…

SA_Mapping

SA_SOAP_SND_1SAP_SOAP_SND_2

SA_SOAP_RCV_1 SA_SOAP_RCV_2

SA_ICO


Test & Monitoring:

  • Call WebService from WS Navigator
  • ASMA (DynamicConfiguration) with MessageId
  • E-Mail received from SAP PRO

SAP_WSNavi2

SAP_Moni1 SAP_Moni2 SAP_Moni3 SAP_Moni4

SAP_Mail

AE-AE / ICO-ICO Connectivity

ICO_ICO_Challenge

Challenge

You want to set up a communication across two Adapter Engines (AEs), because you have each AE in a different network zone. For Dual-Stack Systems you can choose different adapter engines for the sender and receiver channels, but an ICO (Integrated Configuration) is always running on one AE.

Solution

You have to set up two ICO´s, one for each AE.

ICO_ICO_Scenario

  • For the AE-AE connectivity you use the SOAP adapter in XI 3.0 mode to ensure reliable messaging and keep all the meta data of the SOAP message (e.g. DynamicConfiguration as FileName, Sender IP-Address, etc.).
  • For the second ICO do not forget to set the Virtual Receiver in the ICO header data (you can remove the receiver when sending the message from the first ICO via Header Mapping, but we recommend keeping it for transparency reasons).
  • Use Channel Templates to facilitate the handling (C2D = central to decentral)
    • SOAP_RCV_C2D / SOAP_SND_C2D / SOAP_RCV_D2C / SOAP_SND_D2C

There are two approaches possible that make sense from an interface point of view:

  • Out->In=>In->In
    • The first ICO is performing the mapping (Out->In) and
    • the second ICO is using a Dummy Sender Interface as an Inbound Message
  • Out->Out=>Out->In
    • The first ICO is only doing the routing keeping the Outbound Interface as the Receiver Message and
    • the second ICO is performing the mapping (Out->In)
  • The recommendation is to perform the mapping on the Central Adapter Engine, especially if you are using Lookups, then you definitely need both approaches

Example

Scenario 1:

ICO_ICO_OIII

This is a Out->In->In->In case from Central to Decentral Adapter Engine. The mapping takes place in the first ICO.

The interfaces must exactly match (outgoing message of ICO 1 and incoming message of ICO 2). Same applies for Sender and Receiver. The SOAP Sender channel of ICO 2 simply compares the SOAP header data and matches them.

ICO´s:

ICO_1

ICO_2


Scenario 2:

ICO_ICO_OOOI

This is a Out->Out->Out->In case from Central to Decentral Adapter Engine. The mapping takes place in the second ICO.

ICO´s:

ICO_1

ICO_2_O

The setup works in the same way for Decentral => Central Communication (e.g. for incoming EDI messages).

 

Monitoring

In the Message Monitor of the Adapter Engines you will see:

Central Adapter Engine

  • Sender:  SAP_ERP_P    PurchaseOrder_Out (urn:test.com.pi)
  • Receiver: Whitepaper:EDI    EDI_Order_In (urn:test.com.edi)

Decentral Adapter Engine

  • Sender:  SAP_ERP_P   EDI_Order_In (urn:test.com.edi)
  • Receiver: Whitepaper:EDI    EDI_Order_In (urn:test.com.edi)

WHINT DynamicQueue Writer

Functionality

This Adapter Module is able to set the EOIO Queue/Sequence Id by putting it together from existing DC values or by accessing the XML payload using an XPATH.

Prerequisites

  • Deploy the SCA/EAR file provided by Whitepaper InterfaceDesign using NWDS/JSPM/SUM/Telnet
  • Check if the deployment was successful in JNDI browser (NWA -> Troubleshooting -> Java -> JNDI Browser) It should appear under Folder WHINT with name DynamicQueueWriter

Usage

  • Add the Adapter Module in the Module Processor (Sender or Receiver Channel)
  • Name: WHINT/DynamicQueueWriter
  • Type: Local
  • Parameters (Input):
    • in.varX = xpath({XPATH on XML payload}) or dc({Key of Dynamic Configuration: NAME NAMESPACE})
    • in.search.attachment = {true/false} (search also with xpath in attachment)
    • in.remove.specialchar = {true/false} (remove special characters)
  • Parameters (Output):
    • out.dq.name = {combination of in-Variables with constants/strings}
    • Please note that the maximum length of the queue must not exceed 16 characters
    • Substring-Values to build the queue name:
      • $var1.substring(startOffset,endOffset)
      • $var2.substring(startOffset)
      • $var1.substring-after(“TEXT“)
      • $var2.substring-before(“TEXT“)

Example

The Queue is built by concatenating the IDoc number with the file name (without “.xml”) separated by “_”:

  • in.var1: xpath(//EDI_DC40/DOCNUM)
  • in.var2: dc(FileName http://sap.com/xi/XI/System/File)
  • in.search.attachment: true
  • in.remove.specialchar: false
  • out.dq.name: $var1+_+$var2.substring-before(“.xml“)

NWDS: Download SAP NetWeaver Developer Studio

NWDS Update Site

…you will need your S-User…

There is no 7.4 version, you have to use the 7.31 version for 7.4 (SP-Level minus 5)

Update: For 7.50 there is still no Eclipse plugin available… It has to be downloaded from the Softwarecenter.