WHINT Proxy FileTransfer

PFT_SAP

Whenever possible, please try to avoid file-based integration (flat or not, going through the file system is not a good integration pattern), see blog post: Avoid using flat files!. However, sometimes it is still the best fit when the connected system does not provide a clean (RESTful or WebService-based) API….


Functionality

This solution makes FTP/SFTP(SSH) communication with your SAP Backend system obsolete. Your ABAP programs can still use file-based integration (Import/Export), but the communication with PI/PO is done with HTTP(S). This is especially useful, when you do not want ANY system to connect to the file system of your SAP landscape or you want to make sure only the SAP system itself is accessing the (local/remote) file system, no other system or user.

The integration is done through the ABAP Proxy runtime, which supports multipart messages. The XML data of each ABAP proxy contains the meta data (file name, directory, job/program name) and the data file itself is transferred as an attachment.
WHINT_PFT


Prerequisites

  • Import the TPZ file provided by Whitepaper InterfaceDesign into your ESR
  • Import the ABAP transport containing the objects into your SAP Backend system via Transaction SAINT

Usage

From SAP

  • Proxy: ManagedFileTransfer_Out [http://whint.de/xi/PFT]
  • Define an iFlow/Integrated Configuration (ICO) to connect your SAP Backend with a receiver system
    • To map the data from the attachment into the main payload and set the DynamicConfiguration values (ASMA) for FileName and Directory, you can run Operation Mapping FileMessageFromProxy
    • Alternatively you can use your own mapping of course
  • Run: Execute ABAP Program /WHINT/MFT_SEND
    • Enter the file name and source directory
    • Optional: Decide if the file shall be deleted after processing
    • Optional: Enter a scenario id to make routing in the IntegrationDirectory easier. The value will be part of the meta data (XML) as well as a context object (PFTScenarioID)

To SAP

  • Proxy: ManagedFileTransfer_In [http://whint.de/xi/PFT]
  • Define an iFlow/Integrated Configuration (ICO) to connect your sender system with your SAP Backend
    • To map the main payload into the attachment and (optionally) read the DynamicConfiguration values (ASMA), you can run Operation Mapping FileMessageToProxy.
      • To set the FileName and Directory from DynamicConfiguration, set mapping parameter to “ASMA”, otherwise enter the value in the Configuration (iFlow/ICO)
      • If you want to execute a program or schedule a job (with variant), you can also specify those with the mapping parameters. To leave the parameters empty, set the parameter value to “IGNORE”.
    • Alternatively you can use your own mapping of course
  • Run: Send the data from your source system


WHINT MessageTrigger Job

Functionality

In case you want to start an interface without scheduling a polling sender adapter (e.g. file) and you need a reliable trigger for a message transfer, this solution helps to set up any scenario easily. An XML message with a generic structure is sent to the messaging runtime with the parameters you have defined in the trigger job. Based on the parameters you can proceed with any interfacing scenario (call a web service/REST/HTTP URL, read a file using WHINT FileReader Adapter, …).


Prerequisites

  • Deploy the SCA/EAR file provided by Whitepaper InterfaceDesign using NWDS/JSPM/SUM/Telnet
  • Define an Integrated Configuration (ICO) or iFlow that reflects the scenario

Usage

  • Schedule the job from NWA -> Operations -> Jobs -> Java Scheduler
  • Add Task with Job name MessageTriggerJob
  •  Parameters:
    • Scenario: <name/description of the scenario>
    • System: WHINT_MTJ
    • Further scenario parameters as name/value pairs
    • The scheduling should be done according to your business needs (hourly, Mon-Fri between 8:00 and 18:00)


Example

You need to pick up exchange rates without deleting them from the source folder, or you do not want to send a request message from SAP ERP to receive the latest updates (e.g. master data from a CMS). Set up the MessageTrigger Job in NWA and define the scheduling according to your needs.

WHINT Payload SaveAttachment

Functionality

This Adapter Module is adding the current payload as an attachment to the SOAP XI 3.0 message of SAP Process Orchestration/PI. It is useful, when the content is changed afterwards (e.g. via content conversion) or when the target system would like to see the message which was originally received (e.g. when the target system is an SAP ABAP Proxy runtime that supports multipart messages).


Prerequisites

  • Deploy the SCA/EAR file provided by Whitepaper InterfaceDesign using NWDS/JSPM/SUM/Telnet
  • Check if the deployment was successful in JNDI browser (NWA -> Troubleshooting -> Java -> JNDI Browser) It should appear under Folder WHINT with name PayloadSaveAttachment

Usage

  • Add the Adapter Module in the Module Processor (Sender or Receiver Channel)
  • Name: WHINT/PayloadSaveAttachment
  • Type: Local
  • Parameters:
    • FileName (optional): to set the file name of the attachment if needed

Example

Here, a File Adapter is converting a CSV file into XML and sending the data (both files, converted and unconverted) via Mail adapter to a receiver.

Channel Configuration:

PSA_Channel

Message Log:

PSA_LOG

Message Monitor:

PSA_Multipart

Message Editor:

E-Mail:

PSA_Mail

WHINT AggregateMessages byReceiver BPM Process

WHINT_AMR

Functionality

This BPM process collects (aggregates) messages for a system and then (after 5 minutes collecting) forwards them to a receiver (system/partner). It is a generic process that can collect any XML data (not only IDocs) and forward it to a receiver.


Compatibility

  • SAP Process Orchestration 7.3 EHP 1
  • SAP Process Orchestration 7.4
  • SAP Process Orchestration 7.5

Prerequisites

  • Deploy the SCA file provided by Whitepaper InterfaceDesign using NWDS/JSPM/SUM/Telnet
  • Import the TPZ file provided by Whitepaper InterfaceDesign into ESR

Usage

  • ICO/iFlow 1 (Sender to BPM): Map your single message XML to the generic XML Structure and define the correlation under which the messages have to be aggregated (e.g. receiver partner id, vendor number, order number, …)
  • ICO/iFlow 2 (BPM to Receiver): Map the aggregated message XML to the final target structure before sending it to the receiver

Simple MFT with SAP PI/SAP PRO

MFT (Managed File Transfer) scenarios are not clearly described by SAP, so please follow this short description to set up a simple MFT scenario:

Scenario

Polling a source directory/folder via standard FILE/FTP(S)/SFTP adapter, delete or archive the file and write the file(s) to a target directory/folder.

Design Time

No activities needed/mandatory for SLD or ESR (!).
The Service Interfaces (Outbound/Inbound) can be designed of course, but this is not mandatory for the configuration/runtime.
As the file content is not XML, the modeling of a data type is not obligatory.
However, to apply a proper naming and reuse, it would be recommended to create a Software Component (e.g. MFT 1.0 of whitepaper-id.com) and create Dummy Service Interfaces (e.g. File_Out / File_In in a namespace e.g. urn:whint.de:mft).

Configuration

In the Integration Directory the routing configuration can be done based on the created Service Interfaces or by using Dummy Interfaces explicitly (iFlow in NWDS) or by manually entering the name of the Service Interface in the Integrated Configuration.
For the sender channel configuration make sure you read the Adapter Specific Message Attributes (ASMA) for the File Name to transport it accordingly as meta data to the target communication channel.

And then you are done, no rocket science here…

 

MFT – Managed File Transfer

Whenever possible, please try to avoid file-based integration (flat or not, going through the file system is not a good integration pattern), see blog post: Avoid using flat files!. However, sometimes it is still the best fit when the connected system does not provide a clean (RESTful or WebService-based) API. This blog series is about how to set up MFT scenarios with SAP PI/SAP NetWeaver Process Orchestration (PRO) in a smart way.

SAP PI/SAP PRO for MFT scenarios

WHINT FileReader Adapter (On-Premise)

Functionality

This Receiver Adapter is able to read a file from a directory (NFS/FTP/SSH) and process it further.

The standard SAP File Sender Adapter is only able to continuously poll a directory and delete/archive the file or re-send the file from a source folder. There are many scenarios where it makes sense to read a file from a directory event-driven or triggered by a sender system where the file has to remain at its source.

The FileReader Adapter works in a synchronous mode on the receiver side.
Queries can be done either

  • synchronously where the file is returned to the sender directly as a response message or
  • asynchronously where the file can be sent to any receiver by using the RequestResponseBean

WHINT FRA


Prerequisites

  • Deploy the SCA file provided by Whitepaper InterfaceDesign using NWDS/JSPM/SUM/Telnet
  • Import the TPZ file provided by Whitepaper InterfaceDesign into the ESR

Usage

  • Create a new receiver communication channel and select the adapter “FileReader” from namespace “http://whint.de/xi/FRA” (Software Component WHINT_FRA 2015.08 of whitepaper-id.com)
  • Transport Protocol
    • Select one of the following transport protocols
      • NFS – to connect with a local or network file system directory
      • (S)FTP – to connect with an FTP or SSH server
  • Message Protocol
    • You can choose the access method by selecting one of the Message
      • Channel Configuration – The file name and directory is configured in the communication channel directly
      • Query XML message – The file name is sent with an XML message. The format is visible in software component WHINT_FRA, message type FileReaderQuery (http://whint.de/xi/FRA). The directory can be sent along with the XML message or maintained in the channel configuration.
      • Dynamic Configuration – The file name is provided as an adapter-specific message attribute (ASMA): FileName (http://sap.com/xi/XI/System/File).  The directory can be provided as an adapter-specific message attribute (ASMA): Directory (http://sap.com/xi/XI/System/File) or maintained in the channel configuration.
  • Other Options
    • List Directory – The directory content (based on the given file pattern) is returned as a response XML message.
    • Append multiple files -If multiple files are found, the adapter returns them appended. XML files are enveloped in a response XML message, TXT files are simply appended. Binary files can not be appended.
    • Error message if no file is found – An error message is raised if the file can not be found. Otherwise the information is returned in a response XML message.

Example

WHINT_FRA_Config1 WHINT_FRA_Config2

Sync-Async Connectivity (SOAP 2 MAIL) with SAP PRO (w/out BPM)

Sync-Async Patterns with BPM on SAP NetWeaver Process Orchestration (PRO) are quite easy to implement.
However, when you want to design the integration flow as lean as possible, this document might be useful for you as the usage of the sync/async adapter modules is a bit tricky.


Scenario:

  • SOAP Sender (in our example WebShop) is calling SAP PRO synchronously
  • The request is forwarded asynchronously via E-Mail to the receiver (typically it would be e.g. IDoc to SAP ERP of course)

This is an example scenario of course. HTTP/SOAP on the (sync) sender side can be combined with any channel on receiver side (IDOC/SOAP/FILE/SFTP/MAIL/JMS/JDBC/…)


Adapter Modules used:

We are using the following (standard) adapter modules:

  • AF_Modules/RequestOnewayBean (to convert the synchronous message to an asynchronous one)
  • AF_Modules/DynamicConfigurationBean (to store the original message id in the ASMA of the sender channel and read it in the receiver channel to set the correlation)
  • AF_Modules/WaitResponseBean (to wait for the reply of the receiver channel)
  • AF_Modules/NotifyResponseBean (to send back the message to the waiting sender channel using correlation)

ESR Content (sender side):

  • Service Interface Sync Out
  • Service Interface Sync In (Dummy)
  • Operation Mapping (Service Interface Sync Out -> Sync In)
  • Message Mapping (Input Message Type -> Output Message Type)

DIR Configuration:

  • Sender System
    • SOAP Sender Channel
    • SOAP Receiver Channel
  • Receiver System
    • MAIL Receiver Channel

Screenshots:

It is of course not necessary to do this in NWDS but I really started to like it…

SA_Mapping

SA_SOAP_SND_1SAP_SOAP_SND_2

SA_SOAP_RCV_1 SA_SOAP_RCV_2

SA_ICO


Test & Monitoring:

  • Call WebService from WS Navigator
  • ASMA (DynamicConfiguration) with MessageId
  • E-Mail received from SAP PRO

SAP_WSNavi2

SAP_Moni1 SAP_Moni2 SAP_Moni3 SAP_Moni4

SAP_Mail

Decoupling and service-orientation at Amazon

Great article about Amazon´s transformation about a decade ago from an online book-seller into a billion-dollar, IaaS/cloud computing leader at API Evangelist

It is a clear and direct mandate, issued by Jeff Bezos (CEO and founder) to make sure all teams interact through well-defined service interfaces and do not interact on a point-to-point level.

My favorite sentence: “Anyone who doesn’t do this will be fired.  Thank you; have a nice day!