Runtime Variables Overview for SAP Cloud Platform Integration (HCI)

This is an overview of useful and commonly used (dynamic) variables and runtime parameters of SAP Cloud Platform Integration (f.k.a. SAP HCI) based on Apache Camel´s expression language.

SAP Help: Headers and Exchange Properties

Those variables can be used in Integration Flows (e.g. in artifacts as Content Modifier, Router, Channels, …):

ParameterDescription
${in.body} or ${body}Message Payload
${header.var1}Message Header Variable (var1)
${property.prop1}Exchange Property (prop1)
${date:now:yyyy-MM-dd}Current Date/Time with format
${property. SAP_MessageProcessingLogID}Message Guid
${CamelFileName}File Name
${CamelHttpUri}HTTP URI (URL with QueryString)
${CamelDestinationOverrideUrl}SOAP Adapter URL
${exception.message}Error Message
${exception.stacktrace}Error Stacktrace

WHINT AWS SQS Adapter (On-Premise)

Functionality

This JCA Adapter for SAP PI / SAP Process Orchestration (PO) connects to Amazon Web Service´s (AWS) Simple Queue Service (SQS) as well as Simple Notification Service (SNS) in a reliable way. It enables businesses to establish a properly decoupled application integration using message queues and topics like known queue-based protocols as JMS or AMQP.

“Amazon SQS is a distributed queue system that enables web service applications to quickly and reliably queue messages that one component in the application generates to be consumed by another component. A queue is a temporary repository for messages that are awaiting processing.” (Source: Amazon)

Our AWS SQS Adapter enables SAP Process Orchestration/SAP PI to act as a publisher and/or consumer of SQS messages.

The current release uses the version 1.11.143 of the aws-java-sdk.


Features

  • Consume Messages from Amazon SQS
  • Send Messages to Amazon SQS Queue
  • Publish Messages to Amazon SNS Topic
  • Dead Letter Queue Support
  • Adapter-specific Message Attributes, also custom attributes
  • Internet Proxy Support
  • Quality of Service: EO/EOIO

Prerequisites

  • Works with SAP PI 7.1 and higher (incl. SAP Process Orchestration 7.50)
  • Deploy the SCA file provided by Whitepaper InterfaceDesign using NWDS/JSPM/SUM/Telnet
  • Import the TPZ file provided by Whitepaper InterfaceDesign into the ESR

Usage

  • Create a new sender or receiver communication channel and select the adapter “AWS_SQS” from namespace “http://whint.de/xi/ASQ” (Software Component WHINT_ASQ 2016.10 of whitepaper-id.com)
  • Transport Protocol: AWS SQS
  • Message Protocol: HTTP
  • Other Options
    • SQS Queue Name
    • AWS Region (select from Drop-Down)
    • Authentication based on Access/Secret Key
    • Internet Proxy (Host, Port, User, Password)
    • Sender Only:
      • Consuming Interval (secs)
      • Quality of Service (EO/EOIO with queue name)
    • Receiver Only:
      • SNS Topic ARN (Receiver only) as alternative to SQS Queue Name
      • Default Visibility Timeout (secs)
      • Message Retention Period (secs)
      • Maximum Message Size (1-256 KB)
      • Delivery delay (secs)
      • Receive Message Wait Time (secs)
      • SQS Dead Letter
        • Use Redrive Policy
        • Dead Letter Queue Name
        • Maximum Receives (1-1000)
    • Set/Use Adapter-specific Message Attributes (ASMA)
      • Namespace: http://whint.de/xi/ASQ
      • externalMessageID – Message Id received from broker
      • createDateTime – Timestamp of message creation
      • SqsQueueName – Name of the SQS Queue
      • SqsTopicName – Name of the SNS Topic ARN
      • Custom Message Attributes (up to 10, free defined)

Example Configuration

Sender:

 

Receiver:

 

How to expose a CSV file as a Service with SAP PI/PRO

Imagine you need to read a file content from a remote application and you do not want to set up a replication?
This scenario explains how to expose a CSV file (here a product list) as a WebService using the standard SOAP Adapter and the WHINT FileReader Adapter.

FRAS2

CSV File:
FRAS1


Design (ESR):

  • Create Service Interfaces (Sender and Receiver side)
  • For the receiver side you can also use the synchronous Inbound Service Interface FileReaderQueryResponse_In which is shipped with the Adapter
  • Optional: Create a Mapping if you do not want to expose the result of the CSV-to-XML conversion of the MessageTransformBean

FRAS9

FRAS10

FRAS11


Configuration (Eclipse/NWDS):

iFlow:
FRAS4

Receiver Channel (FileReader):
FRAS5

SFTP Connectivity (of course you can read the file from NFS or FTP as well):
FRAS6

Module Configuration (MessageTransformBean: CSV->XML):
FRAS7


Upload File to SFTP Server:
FRAS8

Test from SOAP UI:
FRAS3

Update: I think it is obvious that the Files can be also queried with other sender channels, not only from SOAP (like e.g. REST providing the response in JSON format):

FRAS12

WHINT WebDAV Adapter (On-Premise)

Functionality

This JCA Adapter for SAP PI / SAP Process Orchestration (PRO) supports the WebDAV protocol (Web-based Distributed Authoring and Versioning) which is based on http(s) and allows read/write access to any WebDAV provider that implements the protocol, such as e.g. Telekom MagentaCloud, Box.com, ownCloud, … 


Prerequisites

  • Works with SAP PI 7.1 and higher (incl. SAP Process Orchestration 7.50)
  • Deploy the SCA file provided by Whitepaper InterfaceDesign using NWDS/JSPM/SUM/Telnet
  • Import the TPZ file provided by Whitepaper InterfaceDesign into the ESR

Usage

  • Create a new sender or receiver communication channel and select the adapter “WebDAV” from namespace “http://whint.de/xi/WDA” (Software Component WHINT_WDA 2016.04 of whitepaper-id.com)
  • Transport Protocol: WebDav / HTTP
  • Message Protocol: File
  • Other Options
    • URL (http/https)
    • Proxy (optional)
    • Authentication (Username/Password)
    • Filename
    • Directory
    • Set/Use Adapter-specific Message Attributes (ASMA)
      • Namespace: http://whint.de/xi/WDA
      • FileName
      • Directory
      • Size (sender only)
      • Timestamp (sender only)
      • Host (sender only)
    • Sender Only
      • Poll-Interval in seconds (Sender)
      • Delete file (Yes/No)
      • Duplicate file checking (Yes/No)
      • Quality of Service (EO/EOIO)
      • Archiving on WebDAV Server (Directory/Filename)

Example

WDA1

WDA2

 

How to integrate with Microsoft Azure Service Bus using WHINT AMQP Adapter

This article describes the necessary steps you have to perform to connect with Microsoft Azure Service Bus using WHINT AMQP Adapter (On-Premise).

Azure Service Bus is a generic cloud-based messaging platform that enables you to send data asynchronously between decoupled systems – applications, services and devices – wherever they are.

For the message exchange component types are connected into processing chains within the service bus. The following components are actually supported by the WHINT AMQP Adapter.

  • Service Bus Namespace – A namespace provides a scoping container for addressing Service Bus resources within your application.
  • Service Bus Queue – A Service Bus queue is an entity in which messages are stored.
  • Service Bus Topics and Subscriptions – A topic can be visualized as a queue and when using multiple subscriptions, it becomes a richer messaging model, essentially a one-to-many communication tool.

(Source: https://azure.microsoft.com)


Publishing
Messages send to the SAP PI Adapter framework are published asynchronously to the message broker by an AMQP receiver communication channel.


Consuming
Asynchronously, a message consumer (receiving application) pulls the message from the queue and processes it. The producer does not have to wait for a reply from the consumer in order to continue to process and send further messages. Queues offer First In, First Out (FIFO) message delivery to one or more competing consumers. That is, messages are typically received and processed by the receivers in the order in which they were added to the queue. A sender communication channel is used to connect with the specified queue/topic and consume messages.


How-To: Create Service Bus

Create Namespace (New->Enterprise Integration->Service Bus)

AMQa1

AMQa2

The namespace is created.


How-To: Integration via Queues

The AMQP adapter does not support partitioned queues, so make sure that the option “Enable partitioning” is set to false when creating a queue on Azure Service Bus.

Create Queue:

AMQa3

Add Shared access policy (credentials):

AMQa4

AMQa5

Display Policy and copy Key:

AMQa6


How-To: Send to Queue

Configure Receiver Channel:

AMQa7

AMQa8

Create iFlow/Integrated Configuration (e.g. with Dummy Interfaces):

AMQa9

Test iFlow/ICO:

AMQa10

AMQa11

Monitoring (PI/PRO):

AMQa12

AMQa13

Monitor/check queue in Azure:

AMQa14


How-To: Integration via Topics

Create Topic:

AMQa15

Also create a Shared Access Policy (see above for queues).


How-To: Send to Topic (publish)

Configure Receiver Channel:

AMQa16

Create/Change iFlow/ICO and send message.

Monitor/check queue in Azure: Topics are using a Publish & Subscribe mechanism (Pub-Sub), so when there is no subscriber to a topic, the message is successfully transmitted, but not visible. So in order to see a successful transmission, we also consume the message topic by creating a AMQP Sender Channel that subscribes to the topic.


How-To: Receive Topic (consume a subscribed topic)

Azure: Create Subscription

AMQa17

AMQa18

Configure Sender Channel:

AMQa19

Send message (again).

Monitoring (PI/PRO):

AMQa20

AMQa21

 

WHINT AMQP Adapter (On-Premise)

Functionality

This JCA Adapter for SAP PI / SAP Process Orchestration (PRO) supports the Advanced Message Queuing Protocol (AMQP), a reliable way to establish a properly decoupled application integration using message queues and topics. AMQP defines both the network protocol and the server-side services through a defined set of message capabilities called AMQ model, which consist of a set of components that route and store messages within the broker service and a network wire-level protocol, that lets client applications talk to the server and interact with the AMQ model it implements.

It connects asynchronously to the followings Brokers by sending and receiving messages in a reliable way:

  • Microsoft Azure Service Bus (Cloud)
  • RabbitMQ (On-Premise/Cloud)

Prerequisites

  • Works with SAP PI 7.1 and higher (incl. SAP Process Orchestration 7.50)
  • Deploy the SCA file provided by Whitepaper InterfaceDesign using NWDS/JSPM/SUM/Telnet
  • Import the TPZ file provided by Whitepaper InterfaceDesign into the ESR

Usage

  • Create a new sender or receiver communication channel and select the adapter “AMQP” from namespace “http://whint.de/xi/AMQ” (Software Component WHINT_AMQ 2016.04 of whitepaper-id.com)
  • Transport Protocol
    • Select one of the following transport protocols
      • MS Azure – to connect with a Microsoft Azure Service Bus
      • RabbitMQ – to connect with a RabbitMQ Broker
  • Message Protocol
    • The message procotcol is preselected based on the transport protocol. The following are available so far:
      • AMQP 1.0 – This is the message protocol for MS Azure
      • AMQP 0-9-1 – This is the message protocol for RabbitMQ
  • Other Options
    • Host & Port
    • Virtual Host (RabbitMQ)
    • SSL/TLS enabled
    • Queue / Exchange Name
    • Topic Support
      • Subscription ID (MS Azure)
      • Exchange Type: direct/headers/fanout/topic (RabbitMQ)
      • Routing Key (RabbitMQ)
    • Consuming Interval (secs/msecs)
    • Authentication based on User/Password
    • Quality of Service (EO/EOIO with queue name)
    • Set/Use Adapter-specific Message Attributes (ASMA)
      • Namespace: http://whint.de/xi/AMQ
      • externalMessageID – Message Id received from broker
      • CreateDateTime – Timestamp of message creation
      • QueueExchangeID – Name of the Queue / Exchange Name
      • RoutingKey (only RabbitMQ)

Example

AMQo1

AMQo2

AMQo3

How to check if a file exists with SAP PRO/SAP PI

Sometimes we have to check if a file has been written on a file system and perform actions on that. With PI we usually poll a directory using the File Sender Adapter (e.g. in TEST mode if we can not delete the file). Here´s a smarter way:

  1. Schedule a Trigger that runs the File Check periodically (based on your requirement)
  2. Read the directory of the NFS/FTP/SFTP using the FileReader Adapter

Example Configuration

  • Sender: SOAP Message coming from WHINT MessageTrigger Job
  • Receiver: FileReader Query using WHINT FileReader Adapter
  • Both sides use BusinessComponent “FileChecker” here

Create Business Component:

FileChecker 1

Define iFlow/IntegratedConfiguration (SOAP to FileReader):

FileChecker 2

Configure the FileReader Adapter Channel with the file name you look for:

FileChecker 7

As the FileReader Adapter reads synchronously, we have to switch from Async to Sync using the RequestResponseBean and back to Async using the RequestOnewayBean:

FileChecker 8

If the file is found, we need to have a second (Dummy) iFlow/ICO that is receiving the Response Message from the adapter but without processing it:

FileChecker 9


Set up a MessageTriggerJob in NWA / Operations / Jobs according to your needs (the period is configurable very flexible: daily / every 10 minutes / …):

FileChecker 4FileChecker 6

If the file can not be found, the message goes into error:

FileChecker 5

VOILÁ – here is the message in error (System Error).
If you have defined alerting, an automatic E-Mail is triggered!

In case the file is found, no additional message is being generated. This behaviour can be changed if you change the routing of the Dummy iFlow.


This blog is using the following stand-alone solutions:

SAP Plain J2SE Adapter Engine in Proxy Mode

In case you want to track your HTTP requests (e.g. to SOAP / HTTP / REST receivers), the SAP Plain Adapter Engine might offer a nice surprise to you.

You simply have to reactivate the adapter type “httpwatcher” and then you can set up your own Proxy Server easily:

J2SE Proxy 1 J2SE Proxy 2

In the Log of this HTTP Watcher Proxy you can see the complete HTTP (not HTTPS) traffic that is going through!

 

WHINT MessageMissing Alert

Functionality

How can we make sure if a message was processed at all with SAP PI/PO? Today, no alert means no error.
But what if the sender system did not place the file or send the message? You will find out, when the business will ask for missing data…
WHINT MessageMissing Alert is a solution that runs pro-actively as a job on your system (e.g. every hour during business hours or every month on the first) and checks in the message (performance) monitor if the interface ran or not. In case no message was found, an e-mail will be sent to a receiver.


Prerequisites

  • Works with SAP NetWeaver PO (PI single-stack) version 7.3 EHP1 and higher (including 7.50)
    • If you read from more than one Adapter Engine and you enforce SSL, you need to apply SP15 for 7.31 / SP10 for 7.40
  • Create a new User with the following standard role (or equivalent custom role): SAP_XI_MONITOR_J2EE
  • Activate the performance monitoring in NWA
    • Configuration > Infrastructure > Java System Properties
    • Services > XPI Service: AF Core
    • Properties > profile.performance.runtime = true
    • Configure the interval to keep the last 31 days of the message processing by executing the following URL on your PI/PO system: http(s)://host:port/mdt/performancedataqueryservlet?PeriodConfig=DAILY=31
  • Import the Software Component into your ESR provided by Whitepaper InterfaceDesign
  • Deploy the SCA/EAR file provided by Whitepaper InterfaceDesign using NWDS/JSPM/SUM/Telnet
  • Configure the Scenario by installing the Integration Scenario in NWDS (via iFlows) or in Integration Directory Swing Client. Per default “WHINT_MMA” is defined as the sender business component.

MMA1


Usage

  • Schedule the job from NWA -> Operations -> Jobs -> Java Scheduler
  • Add Task with Job name MessageTriggerJob
  • The solution is using WHINT MessageTrigger Job which can generically send an XML message to the PI/PO runtime.
  • Parameters:
    • ScenarioID: WHINT_MMA (actually this value can have an
    • ScenarioSender: MMA_TEST1 (this value has to match the IntegratedConfiguration/iFlow configuration)
    • Param01: Interface=<the interface name of the message you look for. “*” is the wildcard.>
    • Param02: InterfaceNS=<the interface namespace of the message you look for. “*” is the wildcard.>
    • Param03: SenderComponent=<the sender system of the message you look for. “*” is the wildcard.>
    • Param04: SenderParty=<the sender party of the message you look for. This parameter is optional.>
    • Param05: ReceiverComponent=<the receiver system of the message you look for. “*” is the wildcard.>
    • Param06: ReceiverParty=<the receiver party of the message you look for. This parameter is optional.>
    • Param07: Period=<DAY/HOUR/15MIN>
    • Param08: Frequency=<e.g. the amount of hours within the message was processed>. Possible values / Unit:
      • DAY: 1..31
      • HOUR: 1..24
      • 15MIN: 1..4
    • Param09: EmailRecipient=<E-Mail Address or list: receiver1@company.com;receiver2@company.com>
  • The sequence of the parameters is not important.
  • The scheduling should be done according to your business needs (hourly, Mon-Fri between 8:00 and 18:00)

MMA2

MMA3


Example

Finance data for reconciliation from a payment service provider shall be sent monthly from an SFTP server and posted into SAP accounting. The solution makes sure that the interface is running every month at least once or at a given day.

Another example is that the warehouse interface shall be active during business hours. Stock movement messages should be sent at least once every 30 minutes from SAP ERP to the AS/400 warehouse system. An immediate alert is sent out if no messaging takes place.

MMA4


 

WHINT Proxy FileTransfer

PFT_SAP

Whenever possible, please try to avoid file-based integration (flat or not, going through the file system is not a good integration pattern), see blog post: Avoid using flat files!. However, sometimes it is still the best fit when the connected system does not provide a clean (RESTful or WebService-based) API….


Functionality

This solution makes FTP/SFTP(SSH) communication with your SAP Backend system obsolete. Your ABAP programs can still use file-based integration (Import/Export), but the communication with PI/PO is done with HTTP(S). This is especially useful, when you do not want ANY system to connect to the file system of your SAP landscape or you want to make sure only the SAP system itself is accessing the (local/remote) file system, no other system or user.

The integration is done through the ABAP Proxy runtime, which supports multipart messages. The XML data of each ABAP proxy contains the meta data (file name, directory, job/program name) and the data file itself is transferred as an attachment.
WHINT_PFT


Prerequisites

  • Import the TPZ file provided by Whitepaper InterfaceDesign into your ESR
  • Import the ABAP transport containing the objects into your SAP Backend system via Transaction SAINT

Usage

From SAP

  • Proxy: ManagedFileTransfer_Out [http://whint.de/xi/PFT]
  • Define an iFlow/Integrated Configuration (ICO) to connect your SAP Backend with a receiver system
    • To map the data from the attachment into the main payload and set the DynamicConfiguration values (ASMA) for FileName and Directory, you can run Operation Mapping FileMessageFromProxy
    • Alternatively you can use your own mapping of course
  • Run: Execute ABAP Program /WHINT/MFT_SEND
    • Enter the file name and source directory
    • Optional: Decide if the file shall be deleted after processing
    • Optional: Enter a scenario id to make routing in the IntegrationDirectory easier. The value will be part of the meta data (XML) as well as a context object (PFTScenarioID)

To SAP

  • Proxy: ManagedFileTransfer_In [http://whint.de/xi/PFT]
  • Define an iFlow/Integrated Configuration (ICO) to connect your sender system with your SAP Backend
    • To map the main payload into the attachment and (optionally) read the DynamicConfiguration values (ASMA), you can run Operation Mapping FileMessageToProxy.
      • To set the FileName and Directory from DynamicConfiguration, set mapping parameter to “ASMA”, otherwise enter the value in the Configuration (iFlow/ICO)
      • If you want to execute a program or schedule a job (with variant), you can also specify those with the mapping parameters. To leave the parameters empty, set the parameter value to “IGNORE”.
    • Alternatively you can use your own mapping of course
  • Run: Send the data from your source system