Messaging Principles for Interface Designers

This article is part of our Integration Cookbook.

Quality of Service

If you can influence the application design of the sender or receiver (doesn’t apply if an external partner/service provider with an existing service is involved):

QoS = Quality of Service

  • BE: Best Effort (Synchronous: the sender waits for immediate response. Basic integration which every beginner understands.)
  • EO: Exactly Once (Asynchronous: decoupled integration, a response might be answered by another message)
  • EOIO: Exactly Once in Order (Asynchronous, but serialized with specific queues)

Recommendations

  • Asynchronous Communication where possible, to reduce dependencies between sender and receiver applications and enable decoupling. Real-time Integration can also be achieved asynchronously, e.g. by using events to trigger an immediate message exchange. (Asynchronous does not mean batch mode!)
  • Synchronous Communication only where necessary: Quick Implementation (Error Handling is pushed to the sender/consumer), but strong dependency regarding development/maintenance as well as for the runtime behaviour. There should be a good reason to go for synchronous integration. Examples: Creditlimit-Check, ATP-Check.
  • BUT: Don´t be afraid, if the best fit is a synchronous setup – in the past there was a rule of thumb, to avoid synchronous interfaces in PI (for technical reasons). This does NOT apply anymore. PI serves this pattern very well by exposing SOAP/REST based services, e.g. to connect with databases (via JDBC) and decouple them from the front-end.

Usage of Queues (Async Only)

  • Dynamic Queue Assignment where possible using the object id  (Order, Material, Customer, …) for single messages
  • Static Queue Names for bulk messages (if the throughput allows it, see below)
  • No Queues, if high volume requirements exist (parallelization and mass processing with bulk messages). To ensure correct serialization (sequencing of objects), use timestamps inside the messages.

Message Throughput

For mass data scenarios, consider the following principles:

  • The ideal message size is within 500 KB and 10 MB in SAP PI
  • EO Messaging (Avoid EOIO with static queue names to allow  parallelization)
  • Consider direct (P2P) vs. mediated communication (EAI/HUB)

Design Principles for Interface Designers

This article is part of our Integration Cookbook.

When we build integration solutions, the following characteristics (principles) of application integration should be clear and defined as goals:

  • Enable Connectivity by integrating heterogeneous applications (protocol/format, security, routing, reliability)
  • Transparency through centralization of integration content
  • Control of interface operations through centralized monitoring
  • Efficiency through a standardized and pattern-oriented development process
  • Decoupling of application to minimize dependencies (separate sender from receiver)
  • Reuse of integration artefacts to reduce development effort and double maintenance
  • Simplification of complexity through a harmonized/unified modeling

Outside-In vs. Inside Out

The main question is: Do both applications use existing (import/export) interfaces/APIs or shall new ones be created?

  • Outside-In: Interfaces are modeled from scratch (typically in ESR) and both applications implement the interface structure (e.g. defined by a WSDL)  or
  • Inside-Out: Connecting existing interfaces via mappings. Most of the time we find existing structures like XSDs or WSDLs, IDoc/RFC, DTDs as well as File/CSV structures and database tables. The best is to import those structures as External Definitions. However, it is much more pragmatic to simply create a Data Type and Message Type in the ESR (although this is not an outside-in approach) based on an existing file as we do not have an equivalent XSD available.

Usually mixed versions exist, e.g. the sender side has an existing CSV export format and the receiver is an SAP sytem with an Enterprise Service (ABAP Proxy)

To consider:

  • Where does the business logic and integration logic take place?
  • Are enrichments necessary in the sender system/middleware?
  • Does the middleware have to calculate values in the mapping?
  • Are look-ups needed (e.g. to convert code lists)?

Integration Considerations

Which aspects are relevant for an interface?

  • Processing Time
  • Realtime (Event-based/manually) vs. Batch (Job)
  • Data-/Message-Volume
  • How business critical is the interface? (high availability)
  • Queuing/Serialization
  • Error-Handling
  • Usage of Acknowledgements
  • Retry & Correction methods
  • Archiving of messages

Business Logic in Interfaces 

We hear this sentence many times: “the business logic should not be in PI / in the middleware”. Why? Where does it come from? It is a 1990´s mentality where all the integration logic was programmed into the (file) export or import program. This time is over!
It makes absolutely sense to put intelligence and integration logic into a middleware.

Sender and receiver systems/applications are typically not harmonized, so it is key to have a smart middleware to covert whatever is necessary and make them speak to each other.

Examples:

  • Simple Currency/Language Code conversions
  • Status Code derivation (e.g. SAP knows 80, XYZ knows only 5)
  • Account determinations (G/L account, cost center, etc.) based on different criteria only the sender knows
  • Apply 20% discount on an amount
  • Summarize quantities of line items into a total value
  • Aggregate/Split positions of (bulk/compound) messages

Those are typical mapping tasks of a middleware, not only moving field values from one structure to another and make sure the data is transported.

Another aspect is to have a clearly defined responsibility on the maintenance of a mapping table. That´s why it makes sense to have the functional IT team controlling the value mapping table (and the middleware is accessing the table e.g. remotely via lookups).

WHINT Interface Monitoring: Concierge Service for SAP

Interface Monitoring

WHINT® Interface Monitoring is a consulting service to pro-actively monitor your SAP interface landscape.

We generate daily snapshots (if you wish more often) of your SAP middleware and backend systems and perform level-1/level-2 support via follow-up activities. Our pricing is super-competitive. Please contact us under sales@whitepaper-id.com.

 

WHINT Interface Documentation for SAP Process Orchestration / PI

Functionality

This solution creates an automatic documentation of your interface landscape into your SAP Solution Manager.

The result appears as the Business Blueprint in SOLAR01 / Configuration in SOLAR02 (SolMan 7.1).
The structure is as follows:

  • Business Scenario = <Configuration Scenario>
    • Business Process = <Sender> & <Sender Interface> (Integrated Configuration/Receiver Determination)
      • Process Step = <Receiver> & <Receiver Interface>

IFD02

The overall logic is using the structure of the Interface Catalog, providing a complete list of all end-to-end interfaces of your landscape: How to use the results of WHINT InterfaceCatalog.

The following information is provided in addition to the structure as Project Documents:

  • Business Process
    • Sender Channel
    • Interface Namespace
  • Process Step
    • Mapping
    • Routing Rule
    • Receiver Channel
    • Interface Namespace

Level Business Process (Integrated Configuration): <SenderParty> <SenderService> <Outbound Interface>

IFD03

Project Documentation: Communication Channel (incl. Status)

IFD04

Level Process Step (Receiver Interface of Integrated Configuration): <ReceiverParty> <ReceiverService> <Inbound Interface>

IFD05

Project Documentation: Routing Rule (Receiver & Interface Determination)

IFD06

Project Documentation: Mapping (incl. Parameters)

IFD08

Extended Receiver Determination (Mapping) as Routing Rule

IFD07

Change Mode:

Any update of your interface landscape is reflected automatically in the business blueprint structure.

Project Documents, which have been added manually (like e.g. Mapping Specifications/Mapping Implementation Guide/Service Implementation Guide/etc.) are not removed when the project is updated. This way you can even enhance your documentation on scenario/process/process step level adding your own documents.


Prerequisites

  • You need to have the WHINT Interface Catalog already, which reads all interfaces and routings from your Integration Directory. Instead creating an Excel document, the results are sent as an XML via Web Service (ABAP Proxy) to your SolMan.
  • Import the ABAP Transport provided by Whitepaper InterfaceDesign into your Solution Manager.
  • Import the ESR Content “WHINT_IFD, 2016.10 of whitepaper-id.com” (TPZ provided by Whitepaper InterfaceDesign) into your SAP PI/PO ESR.

The solution was tested with SAP Solution Manager 7.1 and will be released for 7.2 soon.


Configuration Guide

  • Change the existing iFlow/Integrated Configuration (PO) or Receiver Determination (PI) to add the Receiver Interface ProjectDocumentation_In [http://whint.de/xi/IFD] with the appropriate Operation Mapping InterfaceCatalogWithScenarios_To_ProjectDocumentation [http://whint.de/xi/IFD].
  • Set the URL and Solution Manager Project ID and Description via the mapping parameters

IFD01

Now the next time you generate your Interface Catalog, your Interface Documentation is sent to your Solution Manager and creates a maintenance project with all the information.

Integration Content Catalogs in SAP for Standard Interfaces

Looking for standard SAP interfaces?

There is a lot of interfaces and integration content available which you can use for your implementation project using SAP Process Orchestration (SAP PRO) / SAP Process Integration (SAP PI) or SAP HANA Cloud Platform, integration service (SAP HCI).

Here is an overview:



SAP API Hub

Catalog1


SAP Content Hub

Catalog2


SAP App Center

Catalog3


Enterprise Services Workplace

Catalog4


SAP Best Practices Explorer

Catalog5Catalog6


IDocs

  • You have to see in your WE60 or in SAP Help which IDoc might fit for your needs. Please consider that IDocs are not delivered by SAP anymore since 2003…

ABAP Inbound Proxy: Inside-Out vs. Outside-In

Implementing ABAP proxies is quite simple. You generate the ABAP classes via SPROXY (which connects to your ESR) and implement the code.

Regarding the Service Interface Design and Implementation you have a choice:

  • Outside-In: You define the Data Types and Message Types in the ESR and generate into ABAP
  • Inside-Out: You choose what you need in ABAP, import into ESR and use the ABAP perspective. Then you generate into ABAP.

Each approach has pros and cons. Here you see what makes the approaches so different and what fits your requirement best.

Development Process


Outside-In

IOOI1


Inside-Out

IOOI2


Summary


Outside-In

  • Clear choice if you are using Global Data Types (GDTs) and do not want to deal with the SAP-internal field names in the ESR or towards the Non-SAP applications

Inside-Out

  • Complex data structures (e.g. BAPIs) & no need to have an abstraction view in ESR
  • No usage of GDTs

 

 

How to expose a CSV file as a Service with SAP PI/PRO

Imagine you need to read a file content from a remote application and you do not want to set up a replication?
This scenario explains how to expose a CSV file (here a product list) as a WebService using the standard SOAP Adapter and the WHINT FileReader Adapter.

FRAS2

CSV File:
FRAS1


Design (ESR):

  • Create Service Interfaces (Sender and Receiver side)
  • For the receiver side you can also use the synchronous Inbound Service Interface FileReaderQueryResponse_In which is shipped with the Adapter
  • Optional: Create a Mapping if you do not want to expose the result of the CSV-to-XML conversion of the MessageTransformBean

FRAS9

FRAS10

FRAS11


Configuration (Eclipse/NWDS):

iFlow:
FRAS4

Receiver Channel (FileReader):
FRAS5

SFTP Connectivity (of course you can read the file from NFS or FTP as well):
FRAS6

Module Configuration (MessageTransformBean: CSV->XML):
FRAS7


Upload File to SFTP Server:
FRAS8

Test from SOAP UI:
FRAS3

Update: I think it is obvious that the Files can be also queried with other sender channels, not only from SOAP (like e.g. REST providing the response in JSON format):

FRAS12

How to use the results of WHINT InterfaceCatalog

The WHINT InterfaceCatalog for SAP PRO/PI generates two Excel Documents to bring more transparency to your interface landscape. Here is how you can better understand the results:


Channel Catalog

  • a list with all communication channels
  • contains all adapter attributes
  • contains all adapter modules
  • contains all parameters within each adapter module
  • You can filter and/or search for specific values as host names, user names, modules used, etc. across all channels

IFC14

 


Interface Catalog

  • a list of all sender/receiver interface combinations
  • contains operation mappings used
  • contains routing (receiver determination and interface determination)
  • contains an assignment to configuration scenario(s)

Example iFlow / Integrated Configuration

IFC9

IFC12IFC13

 

is shown in the InterfaceCatalog as follows:

IFC10IFC11

ABAP Outbound Proxy: Set EOIO Queue

In order to use EOIO (Exactly-Once-In-Order), you have to pass the Queue ID to the class before calling the Outbound Proxy in your ABAP Program. It can be a static value like shown below (parameter) or a dynamic object like a document number:

 

PARAMETERS: p_queue TYPE sxmsqidapp.

DATA: l_proxy TYPE REF TO z…,
async_messaging TYPE REF TO if_wsprotocol_async_messaging.

CREATE OBJECT l_proxy.

async_messaging ?= l_proxy->get_protocol( if_wsprotocol=>async_messaging ).
async_messaging->set_serialization_context( p_queue ).

WHINT WebDAV Adapter (On-Premise)

Functionality

This JCA Adapter for SAP PI / SAP Process Orchestration (PRO) supports the WebDAV protocol (Web-based Distributed Authoring and Versioning) which is based on http(s) and allows read/write access to any WebDAV provider that implements the protocol, such as e.g. Telekom MagentaCloud, Box.com, ownCloud, … 


Prerequisites

  • Works with SAP PI 7.1 and higher (incl. SAP Process Orchestration 7.50)
  • Deploy the SCA file provided by Whitepaper InterfaceDesign using NWDS/JSPM/SUM/Telnet
  • Import the TPZ file provided by Whitepaper InterfaceDesign into the ESR

Usage

  • Create a new sender or receiver communication channel and select the adapter “WebDAV” from namespace “http://whint.de/xi/WDA” (Software Component WHINT_WDA 2016.04 of whitepaper-id.com)
  • Transport Protocol: WebDav / HTTP
  • Message Protocol: File
  • Other Options
    • URL (http/https)
    • Proxy (optional)
    • Authentication (Username/Password)
    • Filename
    • Directory
    • Set/Use Adapter-specific Message Attributes (ASMA)
      • Namespace: http://whint.de/xi/WDA
      • FileName
      • Directory
      • Size (sender only)
      • Timestamp (sender only)
      • Host (sender only)
    • Sender Only
      • Poll-Interval in seconds (Sender)
      • Delete file (Yes/No)
      • Duplicate file checking (Yes/No)
      • Quality of Service (EO/EOIO)
      • Archiving on WebDAV Server (Directory/Filename)

Example

WDA1

WDA2