Design Principles for Interface Designers

This article is part of our Integration Cookbook.

When we build integration solutions, the following characteristics (principles) of application integration should be clear and defined as goals:

  • Enable Connectivity by integrating heterogeneous applications (protocol/format, security, routing, reliability)
  • Transparency through centralization of integration content
  • Control of interface operations through centralized monitoring
  • Efficiency through a standardized and pattern-oriented development process
  • Decoupling of application to minimize dependencies (separate sender from receiver)
  • Reuse of integration artefacts to reduce development effort and double maintenance
  • Simplification of complexity through a harmonized/unified modeling

Outside-In vs. Inside Out

The main question is: Do both applications use existing (import/export) interfaces/APIs or shall new ones be created?

  • Outside-In: Interfaces are modeled from scratch (typically in ESR) and both applications implement the interface structure (e.g. defined by a WSDL)  or
  • Inside-Out: Connecting existing interfaces via mappings. Most of the time we find existing structures like XSDs or WSDLs, IDoc/RFC, DTDs as well as File/CSV structures and database tables. The best is to import those structures as External Definitions. However, it is much more pragmatic to simply create a Data Type and Message Type in the ESR (although this is not an outside-in approach) based on an existing file as we do not have an equivalent XSD available.

Usually mixed versions exist, e.g. the sender side has an existing CSV export format and the receiver is an SAP sytem with an Enterprise Service (ABAP Proxy)

To consider:

  • Where does the business logic and integration logic take place?
  • Are enrichments necessary in the sender system/middleware?
  • Does the middleware have to calculate values in the mapping?
  • Are look-ups needed (e.g. to convert code lists)?

Integration Considerations

Which aspects are relevant for an interface?

  • Processing Time
  • Realtime (Event-based/manually) vs. Batch (Job)
  • Data-/Message-Volume
  • How business critical is the interface? (high availability)
  • Queuing/Serialization
  • Error-Handling
  • Usage of Acknowledgements
  • Retry & Correction methods
  • Archiving of messages

Business Logic in Interfaces 

We hear this sentence many times: “the business logic should not be in PI / in the middleware”. Why? Where does it come from? It is a 1990´s mentality where all the integration logic was programmed into the (file) export or import program. This time is over!
It makes absolutely sense to put intelligence and integration logic into a middleware.

Sender and receiver systems/applications are typically not harmonized, so it is key to have a smart middleware to covert whatever is necessary and make them speak to each other.

Examples:

  • Simple Currency/Language Code conversions
  • Status Code derivation (e.g. SAP knows 80, XYZ knows only 5)
  • Account determinations (G/L account, cost center, etc.) based on different criteria only the sender knows
  • Apply 20% discount on an amount
  • Summarize quantities of line items into a total value
  • Aggregate/Split positions of (bulk/compound) messages

Those are typical mapping tasks of a middleware, not only moving field values from one structure to another and make sure the data is transported.

Another aspect is to have a clearly defined responsibility on the maintenance of a mapping table. That´s why it makes sense to have the functional IT team controlling the value mapping table (and the middleware is accessing the table e.g. remotely via lookups).

WHINT Interface Monitoring: Concierge Service for SAP

Interface Monitoring

WHINT® Interface Monitoring is a consulting service to pro-actively monitor your SAP interface landscape.

We generate daily snapshots (if you wish more often) of your SAP middleware and backend systems and perform level-1/level-2 support via follow-up activities. Our pricing is super-competitive. Please contact us under sales@whitepaper-id.com.

 

WHINT Interface Documentation for SAP Process Orchestration / PI

Functionality

This solution creates an automatic documentation of your interface landscape into your SAP Solution Manager.

The result appears as the Business Blueprint in SOLAR01 / Configuration in SOLAR02 (SolMan 7.1).
The structure is as follows:

  • Business Scenario = <Configuration Scenario>
    • Business Process = <Sender> & <Sender Interface> (Integrated Configuration/Receiver Determination)
      • Process Step = <Receiver> & <Receiver Interface>

IFD02

The overall logic is using the structure of the Interface Catalog, providing a complete list of all end-to-end interfaces of your landscape: How to use the results of WHINT InterfaceCatalog.

The following information is provided in addition to the structure as Project Documents:

  • Business Process
    • Sender Channel
    • Interface Namespace
  • Process Step
    • Mapping
    • Routing Rule
    • Receiver Channel
    • Interface Namespace

Level Business Process (Integrated Configuration): <SenderParty> <SenderService> <Outbound Interface>

IFD03

Project Documentation: Communication Channel (incl. Status)

IFD04

Level Process Step (Receiver Interface of Integrated Configuration): <ReceiverParty> <ReceiverService> <Inbound Interface>

IFD05

Project Documentation: Routing Rule (Receiver & Interface Determination)

IFD06

Project Documentation: Mapping (incl. Parameters)

IFD08

Extended Receiver Determination (Mapping) as Routing Rule

IFD07

Change Mode:

Any update of your interface landscape is reflected automatically in the business blueprint structure.

Project Documents, which have been added manually (like e.g. Mapping Specifications/Mapping Implementation Guide/Service Implementation Guide/etc.) are not removed when the project is updated. This way you can even enhance your documentation on scenario/process/process step level adding your own documents.


Prerequisites

  • You need to have the WHINT Interface Catalog already, which reads all interfaces and routings from your Integration Directory. Instead creating an Excel document, the results are sent as an XML via Web Service (ABAP Proxy) to your SolMan.
  • Import the ABAP Transport provided by Whitepaper InterfaceDesign into your Solution Manager.
  • Import the ESR Content “WHINT_IFD, 2016.10 of whitepaper-id.com” (TPZ provided by Whitepaper InterfaceDesign) into your SAP PI/PO ESR.

The solution was tested with SAP Solution Manager 7.1 and will be released for 7.2 soon.


Configuration Guide

  • Change the existing iFlow/Integrated Configuration (PO) or Receiver Determination (PI) to add the Receiver Interface ProjectDocumentation_In [http://whint.de/xi/IFD] with the appropriate Operation Mapping InterfaceCatalogWithScenarios_To_ProjectDocumentation [http://whint.de/xi/IFD].
  • Set the URL and Solution Manager Project ID and Description via the mapping parameters

IFD01

Now the next time you generate your Interface Catalog, your Interface Documentation is sent to your Solution Manager and creates a maintenance project with all the information.

Integration Content Catalogs in SAP for Standard Interfaces

Looking for standard SAP interfaces?

There is a lot of interfaces and integration content available which you can use for your implementation project using SAP Process Orchestration (SAP PRO) / SAP Process Integration (SAP PI) or SAP HANA Cloud Platform, integration service (SAP HCI).

Here is an overview:



SAP API Hub

Catalog1


SAP Content Hub

Catalog2


SAP App Center

Catalog3


Enterprise Services Workplace

Catalog4


SAP Best Practices Explorer

Catalog5Catalog6


IDocs

  • You have to see in your WE60 or in SAP Help which IDoc might fit for your needs. Please consider that IDocs are not delivered by SAP anymore since 2003…

ABAP Inbound Proxy: Inside-Out vs. Outside-In

Implementing ABAP proxies is quite simple. You generate the ABAP classes via SPROXY (which connects to your ESR) and implement the code.

Regarding the Service Interface Design and Implementation you have a choice:

  • Outside-In: You define the Data Types and Message Types in the ESR and generate into ABAP
  • Inside-Out: You choose what you need in ABAP, import into ESR and use the ABAP perspective. Then you generate into ABAP.

Each approach has pros and cons. Here you see what makes the approaches so different and what fits your requirement best.

Development Process


Outside-In

IOOI1


Inside-Out

IOOI2


Summary


Outside-In

  • Clear choice if you are using Global Data Types (GDTs) and do not want to deal with the SAP-internal field names in the ESR or towards the Non-SAP applications

Inside-Out

  • Complex data structures (e.g. BAPIs) & no need to have an abstraction view in ESR
  • No usage of GDTs

 

 

How to expose a CSV file as a Service with SAP PI/PRO

Imagine you need to read a file content from a remote application and you do not want to set up a replication?
This scenario explains how to expose a CSV file (here a product list) as a WebService using the standard SOAP Adapter and the WHINT FileReader Adapter.

FRAS2

CSV File:
FRAS1


Design (ESR):

  • Create Service Interfaces (Sender and Receiver side)
  • For the receiver side you can also use the synchronous Inbound Service Interface FileReaderQueryResponse_In which is shipped with the Adapter
  • Optional: Create a Mapping if you do not want to expose the result of the CSV-to-XML conversion of the MessageTransformBean

FRAS9

FRAS10

FRAS11


Configuration (Eclipse/NWDS):

iFlow:
FRAS4

Receiver Channel (FileReader):
FRAS5

SFTP Connectivity (of course you can read the file from NFS or FTP as well):
FRAS6

Module Configuration (MessageTransformBean: CSV->XML):
FRAS7


Upload File to SFTP Server:
FRAS8

Test from SOAP UI:
FRAS3

Update: I think it is obvious that the Files can be also queried with other sender channels, not only from SOAP (like e.g. REST providing the response in JSON format):

FRAS12

How to use the results of WHINT InterfaceCatalog

The WHINT InterfaceCatalog for SAP PRO/PI generates two Excel Documents to bring more transparency to your interface landscape. Here is how you can better understand the results:


Channel Catalog

  • a list with all communication channels
  • contains all adapter attributes
  • contains all adapter modules
  • contains all parameters within each adapter module
  • You can filter and/or search for specific values as host names, user names, modules used, etc. across all channels

IFC14

 


Interface Catalog

  • a list of all sender/receiver interface combinations
  • contains operation mappings used
  • contains routing (receiver determination and interface determination)
  • contains an assignment to configuration scenario(s)

Example iFlow / Integrated Configuration

IFC9

IFC12IFC13

 

is shown in the InterfaceCatalog as follows:

IFC10IFC11

ABAP Outbound Proxy: Set EOIO Queue

In order to use EOIO (Exactly-Once-In-Order), you have to pass the Queue ID to the class before calling the Outbound Proxy in your ABAP Program. It can be a static value like shown below (parameter) or a dynamic object like a document number:

 

PARAMETERS: p_queue TYPE sxmsqidapp.

DATA: l_proxy TYPE REF TO z…,
async_messaging TYPE REF TO if_wsprotocol_async_messaging.

CREATE OBJECT l_proxy.

async_messaging ?= l_proxy->get_protocol( if_wsprotocol=>async_messaging ).
async_messaging->set_serialization_context( p_queue ).

WHINT WebDAV Adapter (On-Premise)

Functionality

This JCA Adapter for SAP PI / SAP Process Orchestration (PRO) supports the WebDAV protocol (Web-based Distributed Authoring and Versioning) which is based on http(s) and allows read/write access to any WebDAV provider that implements the protocol, such as e.g. Telekom MagentaCloud, Box.com, ownCloud, … 


Prerequisites

  • Works with SAP PI 7.1 and higher (incl. SAP Process Orchestration 7.50)
  • Deploy the SCA file provided by Whitepaper InterfaceDesign using NWDS/JSPM/SUM/Telnet
  • Import the TPZ file provided by Whitepaper InterfaceDesign into the ESR

Usage

  • Create a new sender or receiver communication channel and select the adapter “WebDAV” from namespace “http://whint.de/xi/WDA” (Software Component WHINT_WDA 2016.04 of whitepaper-id.com)
  • Transport Protocol: WebDav / HTTP
  • Message Protocol: File
  • Other Options
    • URL (http/https)
    • Proxy (optional)
    • Authentication (Username/Password)
    • Filename
    • Directory
    • Set/Use Adapter-specific Message Attributes (ASMA)
      • Namespace: http://whint.de/xi/WDA
      • FileName
      • Directory
      • Size (sender only)
      • Timestamp (sender only)
      • Host (sender only)
    • Sender Only
      • Poll-Interval in seconds (Sender)
      • Delete file (Yes/No)
      • Duplicate file checking (Yes/No)
      • Quality of Service (EO/EOIO)
      • Archiving on WebDAV Server (Directory/Filename)

Example

WDA1

WDA2

 

How to integrate with Microsoft Azure Service Bus using WHINT AMQP Adapter

This article describes the necessary steps you have to perform to connect with Microsoft Azure Service Bus using WHINT AMQP Adapter (On-Premise).

Azure Service Bus is a generic cloud-based messaging platform that enables you to send data asynchronously between decoupled systems – applications, services and devices – wherever they are.

For the message exchange component types are connected into processing chains within the service bus. The following components are actually supported by the WHINT AMQP Adapter.

  • Service Bus Namespace – A namespace provides a scoping container for addressing Service Bus resources within your application.
  • Service Bus Queue – A Service Bus queue is an entity in which messages are stored.
  • Service Bus Topics and Subscriptions – A topic can be visualized as a queue and when using multiple subscriptions, it becomes a richer messaging model, essentially a one-to-many communication tool.

(Source: https://azure.microsoft.com)


Publishing
Messages send to the SAP PI Adapter framework are published asynchronously to the message broker by an AMQP receiver communication channel.


Consuming
Asynchronously, a message consumer (receiving application) pulls the message from the queue and processes it. The producer does not have to wait for a reply from the consumer in order to continue to process and send further messages. Queues offer First In, First Out (FIFO) message delivery to one or more competing consumers. That is, messages are typically received and processed by the receivers in the order in which they were added to the queue. A sender communication channel is used to connect with the specified queue/topic and consume messages.


How-To: Create Service Bus

Create Namespace (New->Enterprise Integration->Service Bus)

AMQa1

AMQa2

The namespace is created.


How-To: Integration via Queues

The AMQP adapter does not support partitioned queues, so make sure that the option “Enable partitioning” is set to false when creating a queue on Azure Service Bus.

Create Queue:

AMQa3

Add Shared access policy (credentials):

AMQa4

AMQa5

Display Policy and copy Key:

AMQa6


How-To: Send to Queue

Configure Receiver Channel:

AMQa7

AMQa8

Create iFlow/Integrated Configuration (e.g. with Dummy Interfaces):

AMQa9

Test iFlow/ICO:

AMQa10

AMQa11

Monitoring (PI/PRO):

AMQa12

AMQa13

Monitor/check queue in Azure:

AMQa14


How-To: Integration via Topics

Create Topic:

AMQa15

Also create a Shared Access Policy (see above for queues).


How-To: Send to Topic (publish)

Configure Receiver Channel:

AMQa16

Create/Change iFlow/ICO and send message.

Monitor/check queue in Azure: Topics are using a Publish & Subscribe mechanism (Pub-Sub), so when there is no subscriber to a topic, the message is successfully transmitted, but not visible. So in order to see a successful transmission, we also consume the message topic by creating a AMQP Sender Channel that subscribes to the topic.


How-To: Receive Topic (consume a subscribed topic)

Azure: Create Subscription

AMQa17

AMQa18

Configure Sender Channel:

AMQa19

Send message (again).

Monitoring (PI/PRO):

AMQa20

AMQa21