How to expose a CSV file as a Service with SAP PI/PRO

Imagine you need to read a file content from a remote application and you do not want to set up a replication?
This scenario explains how to expose a CSV file (here a product list) as a WebService using the standard SOAP Adapter and the WHINT FileReader Adapter.

FRAS2

CSV File:
FRAS1


Design (ESR):

  • Create Service Interfaces (Sender and Receiver side)
  • For the receiver side you can also use the synchronous Inbound Service Interface FileReaderQueryResponse_In which is shipped with the Adapter
  • Optional: Create a Mapping if you do not want to expose the result of the CSV-to-XML conversion of the MessageTransformBean

FRAS9

FRAS10

FRAS11


Configuration (Eclipse/NWDS):

iFlow:
FRAS4

Receiver Channel (FileReader):
FRAS5

SFTP Connectivity (of course you can read the file from NFS or FTP as well):
FRAS6

Module Configuration (MessageTransformBean: CSV->XML):
FRAS7


Upload File to SFTP Server:
FRAS8

Test from SOAP UI:
FRAS3

Update: I think it is obvious that the Files can be also queried with other sender channels, not only from SOAP (like e.g. REST providing the response in JSON format):

FRAS12

Sync-Async Connectivity (SOAP 2 MAIL) with SAP PRO (w/out BPM)

Sync-Async Patterns with BPM on SAP NetWeaver Process Orchestration (PRO) are quite easy to implement.
However, when you want to design the integration flow as lean as possible, this document might be useful for you as the usage of the sync/async adapter modules is a bit tricky.


Scenario:

  • SOAP Sender (in our example WebShop) is calling SAP PRO synchronously
  • The request is forwarded asynchronously via E-Mail to the receiver (typically it would be e.g. IDoc to SAP ERP of course)

This is an example scenario of course. HTTP/SOAP on the (sync) sender side can be combined with any channel on receiver side (IDOC/SOAP/FILE/SFTP/MAIL/JMS/JDBC/…)


Adapter Modules used:

We are using the following (standard) adapter modules:

  • AF_Modules/RequestOnewayBean (to convert the synchronous message to an asynchronous one)
  • AF_Modules/DynamicConfigurationBean (to store the original message id in the ASMA of the sender channel and read it in the receiver channel to set the correlation)
  • AF_Modules/WaitResponseBean (to wait for the reply of the receiver channel)
  • AF_Modules/NotifyResponseBean (to send back the message to the waiting sender channel using correlation)

ESR Content (sender side):

  • Service Interface Sync Out
  • Service Interface Sync In (Dummy)
  • Operation Mapping (Service Interface Sync Out -> Sync In)
  • Message Mapping (Input Message Type -> Output Message Type)

DIR Configuration:

  • Sender System
    • SOAP Sender Channel
    • SOAP Receiver Channel
  • Receiver System
    • MAIL Receiver Channel

Screenshots:

It is of course not necessary to do this in NWDS but I really started to like it…

SA_Mapping

SA_SOAP_SND_1SAP_SOAP_SND_2

SA_SOAP_RCV_1 SA_SOAP_RCV_2

SA_ICO


Test & Monitoring:

  • Call WebService from WS Navigator
  • ASMA (DynamicConfiguration) with MessageId
  • E-Mail received from SAP PRO

SAP_WSNavi2

SAP_Moni1 SAP_Moni2 SAP_Moni3 SAP_Moni4

SAP_Mail

WHINT Webhook JSON Connector

WHINT_WJC

Functionality

This solution enables you to push data via an ABAP structure to a Webhook using JSON.

Prerequisites

  • Works with any SAP ABAP Based system (Basis 6.10 and higher)
  • You install the Whitepaper InterfaceDesign ABAP components through a transport
  • The software comes in an isolated, namespace-specific environment that basically provides one function group and a sample report to show the usage

Usage

  • You pass the data to be transferred from any context
    • BAdI (Business Add-In)
    • User-Exit (CMOD)
    • BTE (Business Transaction Event)
    • BOR Event
    • Enhancement Spot
    • Customer-specific ABAP program
  • to a function module:  /WHINT/WJC_ABAP2JSON

Example

You want to add subscribers from your SAP CRM system to your MailChimp E-Mail distribution list.

See our blog entry: “SAP integration with Webhooks and JSON through elastic.io

SAP integration with Webhooks and JSON through elastic.io

WHINT_WJC_elastic

This article shows how to integrate any SAP ABAP system with MailChimp through elastic.io using a solution of Whitepaper InterfaceDesign, the WHINT Webhook JSON Connector.


elastic

Our Partner elastic.io is an iPaaS (Integration Platform as a Service), a cloud connectivity solution to enable business integration between on-premise and on-demand applications. The platform is hosted in Germany´s most secure cloud environment and fulfills highest security standards.

MailChimp is a cloud solution to send marketing emails, automated messages, and targeted campaigns.

Whitepaper InterfaceDesign provides professional services and software solutions around SAP integration and cloud connectivity. We are a SAP Application Development Partner since 2014.

SAP Partner


Step-by-step guide: Prerequisites

  1. Install the add-on into your SAP ABAP system
  2. Create an account at elastic.io
  3. Create an account at MailChimp
  • No additional middleware needed (!)

Configure MailChimp

  • Create a list

MailChimp Create List


Configure elastic.io (1)

  • Add new flow
  • Save Webhook URL

elastic 1 elastic 2


Configure SAP

  • Configure the connectivity
  • Select the data structure to be sent
  • Retrieve the JSON structure by invoking the WHINT WJC (Webhook JSON Connector)

Transaction SM59: Create an RFC destination (HTTP) and select SSL

sap 1 sap 2

Transaction STRUST: Load the SSL certificates (CAs) needed for the HTTPS connection

sap 3

Transaction SE11: Select or create a DDIC structure to pass the information to MailChimp

sap 4

Transaction SE38: Create program that invokes the WHINT WJC and run:

sap 5 sap 7


Configure elastic.io (2)

  • Insert JSON String into Webhook sample
  • Define field mapping
  • Connect your MailChimp account

elastic 3

elastic 4

elastic 5

Start the flow!


Monitoring & Results

  • SAP
  • elastic.io
  • MailChimp

sap 8

Monitoring on elastic.io:

elastic 7 elastic 8

Verify result in MailChimp:

mailchimp list 2 mailchimp list 3

Avoid using flat files !

This is about why we should not use flat files anymore:


Integration across systems in the 1990´s looked like this:
– Sender System runs export program in batch to produce flat file
– Transfer script sends flat file via FTP to Receiver System
– Receiver System runs import program in batch to update database


What is so bad about it:

– STORAGE: to store files on a (local) file system is not good from a Security and Auditing point of view. Any intruder who is able to access the file system, can read the data (unless they are right away encrypted, e.g. via PGP). It is better to persist your (especially sensitive) data in a database.
– TRANSFER: the transmission protocol FTP is not very secure and sometimes can not be so reliable (depending on the FTP server implementation). To secure the transmission use at least SSH/SSL encryption via SFTP/FTPs
– FORMAT: there are issues with flat files in general. The change management is horrible if you want to extend something (all senders and receivers have to switch their export/import programs in one shot) – and believe me: there will always be changes…

The only advantage is: File Size.
Nothing is leaner than a flat file, especially if you deal with high volumes/large messages.
But especially in SAP projects we learned already: Sometimes it is better to go a structured and organized way instead of choosing the light-weight and easy approach.


Options to improve existing integration flows (keeping the flat file format for the time being or for message size reasons):
– Clean up the SAP side first where you can instead of redesigning the flow completely: use ABAP Proxy with flat file as attachment to avoid the STORAGE problem and solve the TRANSFER issue by using http(s):
– TRANSFER: No more FTP to/from SAP servers, only RFC or HTTP
– STORAGE: Change your programs to write files into memory only and then pass the data to an ABAP proxy. This is of course only possible for you own programs, but SAP is also modernizing the way to transfer standard-based files.

For new integrations simply try to avoid flat files. They are not state-of-the-art anymore.

J2SE Adapter Engine – Setup/Installation

How to use the J2SE Adapter Engine:
The J2SE AE aka “Adapter Engine (Java SE)” is still supported and being maintained by SAP. You can use it if you need a light-weight middleware component on any system close to database, JMS queue, WebService or NFS filesystem.
Mainly it is used to solve the UNIX-Windows NFS dilemma (you want to access a network drive but your PI/PO installation is in UNIX).
Another network/security use case is to install it into the DMZ and use the J2SE AE as a reverse proxy (application gateway) for inbound communication from the internet.

Here you go:

  1. Get J2SE software from SAP (Service Marketplace Alias /swdc -> Support Packages and Patches-> Browse -> SAP NetWeaver and complementary products -> SAP NetWeaver -> SAP NETWEAVER 7.4 -> Entry by Component -> PI Adapter Engine (Java SE) => Downoad XI CONNECTIVITY SE 7.40
  2. Download JRE (e.g. from http://www.oracle.com/technetwork/java/index.html according to the required version) – Java 7 works fine
  3. Install JRE and extract AE archive into file system (only archive com.sap.xi.techadapters.sda is needed)
  4. Copy file servlet.jar (can be downloaded e.g. from http://www.java2s.com/Code/Jar/s/Downloadservletjar.htm) into extracted folder tech_adapter
  5. Install Service under Windows/Unix (use admin rights “run as administrator” when executing the script)
  6. Start the Admin UI from Browser with default port 8200 and user sap password init
  7. Configure the channels you need and communicate via HTTP(S) to your PI/PO integration server

Btw: This approach basically exists since XI 2.0 and is still valid for PI/PO 7.5! See the official documentation: SAP Help.