Runtime Variables Overview for SAP Cloud Platform Integration (HCI)

This is an overview of useful and commonly used (dynamic) variables and runtime parameters of SAP Cloud Platform Integration (f.k.a. SAP HCI) based on Apache Camel´s expression language.

SAP Help: Headers and Exchange Properties

Those variables can be used in Integration Flows (e.g. in artifacts as Content Modifier, Router, Channels, …):

ParameterDescription
${in.body} or ${body}Message Payload
${header.var1}Message Header Variable (var1)
${property.prop1}Exchange Property (prop1)
${date:now:yyyy-MM-dd}Current Date/Time with format
${property. SAP_MessageProcessingLogID}Message Guid
${CamelFileName}File Name
${CamelHttpUri}HTTP URI (URL with QueryString)
${CamelDestinationOverrideUrl}SOAP Adapter URL
${exception.message}Error Message
${exception.stacktrace}Error Stacktrace

New SAP Training: WIFMS4 – Inbound Interface Management in SAP S/4HANA

We have developed a standard SAP Training about managing inbound interfaces (ABAP Proxies) in SAP S/4HANA and SAP Business Suite (ERP, CRM, …): WIFMS4.

training

You learn how to implement a robust error handling using already available (and free) SAP standard technology:

  • Forward Error Handling (FEH) and
  • Error and Conflict Handler (ECH)

without the need of implementing a heavy solution, such as SAP AIF.

SAP Blog/Announcement on WIFMS4

SAP Online Training Catalog (with dates)

Integration Cookbook for SAP Middleware

cookbook

Our Integration Cookbook is a collection of Good & Best Practices for SAP middleware products:

  • SAP Process Integration (PI)
  • SAP Process Orchestration (PO)
  • SAP Business Process Management (BPM)
  • SAP Cloud Platform Integration (HCI)

It provides principles, guidelines and naming conventions to build great integration solutions and interfaces.

Process Modeling Principles for Interface Designers

This article is part of our Integration Cookbook.

Modularization

  • Limit the process model to 25 – 50 steps to keep them manageable. This can be achieved by hiding complexity in sub-processes.

Interface Design

  • Use Service Interfaces based on Message Type and Data Type defined in ESR. Using External Definitions (XSD, WSDL) can lead to compatibility problems during import and runtime (also when anonymous types are used in the WSDL/XSD, the data structures might not be available for process context definition.
  • Service Interfaces used for communication between BPM and PI/AEX have to have the interface pattern Stateless (XI30-Compatible) in ESR.
  • The interfaces of a BPM process should be as lean as possible. Any mapping and conversion logic should be handled in PI already. Use a harmonized and lean data model for the process context and not to “pollute” the BPM with the specifics of the individual source/target systems. Instead use mappings in integration flows to perform conversions between the data model used for the process context and the specific requirements of the source/target interfaces.
  • XSD constraint violations, for example if a string field requires a minimum length of three characters but it is attempted to assign a value consisting of only two characters, can cause issues that are hard to track down. For example if starting a process instance fails because such a constraint is violated in the output mapping in the start event of a process the incoming message is considered to be successfully delivered to the BPM engine but despite that no corresponding process instance can be found in Manage Processes of NetWeaver Administrator, i.e. the only hints regarding that situation are exceptions in the Developer Trace. Due to that it is advisable to avoid using data types with XSD constraints when defining the data model for the process context.

Correlation Conditions

  • Use direct value comparisons (e.g. string-equals), i.e. avoid expensive operations

Connectivity BPM – PI

  • Using the XI protocol requires a service interfaces that is XI30 compliant as well as a specific BPM service group configuration and communication channels making use of the local proxy runtime.

Parallelism

  • BPM always locks whole context objects and not substructures of objects, so in case parallel process flows manipulate different substructures of the same context object, splitting up the object will reduce the number of access collisions.
  • If parallel process flows manipulate the same data structure, the access to the data structure is automatically serialized and might slow down the processing.
  • A data object can easily become a synchronization point, thus hamper the system performance, if it is modified from different artifacts of the process model concurrently. That is especially to be considered when using parallel split gateways or dynamic parallel loops (par-for-each) and trying to store the outcome of the parallel operation into a single/central data object.
  • The same issue can appear if you mark a single activity for dynamic parallel execution (par-for-each) and perform an output mapping to the same data object. Again, the execution would be serialized due to concurrent request for a write lock.
  • A real parallelization can be achieved by modeling an embedded sub-flow and execute this flow dynamically in parallel.
  • Additionally, in the sub-flow you have to separate the concrete activity from the mapping to the central DO. Use an intermediate DO to store the mapping result, i.e. to avoid a serialization by a write-lock on the central DO. Then map the intermediate DO content to the central DO. The system will serialize the execution at the mapping activity. But since mapping is typically a much less expensive operation than the ‘real’ activity, the lock won’t last long. This means the negative impact of the serialization at the DO is less than not using a sub-process.

Operation Mappings

  • In the process model Operation Mappings are consumed like any other service interface by assigning them to an automated activity. Unlike other calls to PI, a service reference of type WS is required here instead of XI as operation mappings are called via SOAP and not the XI protocol. Additionally, as the operation mapping is running on the local AEX of the Process Orchestration system, the service group has to point at the local system.
  • To ensure that calling the operation mapping works at runtime a BPM DC consuming an Operation Mapping needs a build-time dependency to the public part “def” of the framework DC “bie/sca/scdl/contributors/maas”. Usually that dependency is created automatically if an Operation Mapping is imported from ESR.
  • BPM does not support consuming Operation Mappings that contain the following characteristics:
    • Value Mappings
    • Parameters
    • Synchronous Interfaces Responses
    • Fault Messages
  • The combined length of application name and operation mapping service port name must not exceed 255 characters (SAP Note 1890617).

Context Size

  • Avoiding large process contexts (>1 MB) increases the number of concurrent processes that can be executed by the server. The count of mapped attributes (per activity) should be smaller than fifty.
  • Claim Check: Reducing the footprint of the process context can be achieved with the Claim Check Pattern.

Error Handling

  • The standard boundary event TechnicalError can be used to capture system errors during synchronous calls in BPM automated activities.
  • For the purpose of sending notifications or automated error handling activities details about the technical error are available as output fields and can be mapped to the process context.
  • Additionally, if interfaces consumed by BPM contain fault structures boundary events for catching those faults will be provided by BPM and can be added to the automated activity.
  • In case no boundary events are added to an automated activity the default behavior applies. In this case the process will be suspended into an error state. In NetWeaver Administrator such a process instance can be resumed manually and it will try to continue execution at the step where the error occurred. This can be automated by using the WHINT ProcessRestart Job.

BRM

  • Business rules which are meant to be reusable and not related to a specific process reside in a dedicated rules DC. Reusable business rules are called via web service by BPM and other consumers.
  • The web-service generation feature provided by BRM only works inside-out, i.e. it is not possible to directly use an ESR Service Interface as a web-service interface for a business rule.
  • Decision Table Maintenance
    • The content of BRM Decision Tables can be maintained in two basic ways, either by a developer using NetWeaver Developer or by authorized users in the web-based rules manager directly on the application server. A decision which method should be used for a specific scenario can be made based on the following aspects:
NWDS Rules Manager
Accessibility Developers only Authorized users, i.e. also business users if necessary
Software Logistics Content is transported through system landscape like all other development artifacts via NWDI, changes available to all developers Changes are applied directly on a single run time system, i.e. even changes directly on the production system are possible. No automatic transport of the content change to other systems of the landscape. No impact on the content defined by the developer in NWDS, developer would have to acquire and import the changed definition manually.

Further Recommendations

Naming Conventions for SAP Cloud Platform Integration

This article is part of our Integration Cookbook.

Name Pattern  Example
Integration
Package
<Area> <Scenario>

 

  • WHINT PaymentReporting Ingenico
  • Concur Integration
  • SuccessFactors Employee Central
Process
Integration
iFlow
<Area> <Interface>
  • WHINT PR Ingenico CollectionAppendixReport
  • Replicate Business Partner from SAP Business Suite
OData
Service
 tbd
System <Name>
  • SAP_ECC
  • SAP_S4HANA
  • SAP_C4C
  • Concur
  • SSFS
  • Sharepoint
  • ECB
  • Ingenico
Communication
Channel
<AdapterType>
_<Direction: SND/RCV>
_<opt:Technical Details>
_<opt:Objekt>
  • IDOC_RCV_EOIO
  • SOAP_SND_JournalEntry
  • MAIL_RCV_Attachment
Adapter (Camel)  tbd

 

Naming Conventions for SAP BPM

This article is part of our Integration Cookbook.

Name Pattern  Example
Software
Component 
Same as ESR SWC or
generic SWC “BPM” when
used in NWDI due to the
underlying components
  • BPM of whitepaper-id.com
Development
Component
bpm/<opt:SWC>
/<Application>/<Object>
  • bpm/otc/edi
  • bpm/le/shipment/tracking/ups
Process Name <Action>
_<opt:Application/Context>
_<opt:Object>
  • Download_ShipmentTracking
  • Query_FxRates
  • Process_EcomOrder

Naming Conventions for SAP PI

This article is part of our Integration Cookbook.

SLD

Name Pattern  Example
Software
Product
<Name>, <Version> of <vendor-url>
  • Microsoft Navision 3.5 of whitepaper-id.com
  • SAP ERP 6.0 of whitepaper-id.com
Software
Component

(same for Unit)
<Name>, <Version> of <vendor-url>
(use version 1.0 and increase only,
if you need to keep both)
  • MS_NAVISION_ERP 1.0 of whitepaper-id.com
Technical
System
SAP: <SID> on <host>
Non-SAP:
<NAME> on <host>

The combination of
name/SID and host makes
the Technical System unique!

  • WSM on winwsm
  • ORACLE on oradev01
Business
System
(only used if
several instances
exist)*
<Aplication-Name>_<opt:Role>
_<opt:Country/Region>
_<TransportLevel>
Transport Level examples:

  • D = Development
  • T = Test
  • Q = Quality
  • S = Sandbox
  • P = Production
  • SAP_CRM_EU_P
  • MS_SHAREPOINT_Q
  • CDB_FR_T

* An E-Mail or FTP Server can also be created as a Business Component in Integration Directory as it exists only once

Do not differentiate between On-Premise and Cloud Applications. Both have to be created in the SLD if you have multiple instances (Transport Levels).


ESR

Name Pattern  Example
Namespace http://<company-url>/xi/<shortname>/<area>
Interface
Action
  • Create
  • Change
  • Cancellation
  • By<Object>
  • …Create…
  • …Change… (not update)
  • …Cancellation… (not Cancel/Delete/Remove)
  • …ByID… / …ByCustomerName…
Message
Category
  • Request / Confirmation
  • Query / Response
  • Notification
  • Information
  • (for persistent operations)
  • (for reading operations: don’t use Read/Select)
  • (async message transfer for a special purpose) e.g. GoodsMovementNotification
  • (publish & subcribe mode)
Service
Interface
(Stateless XI 3.0)
<Document/Data>
<optional:_Version>
<optional: If-Action><MessageCategory(s)>
_<Direction: Out/In>
  • CustomerCreateRequestConfirmation_In
  • CustomerByIDQueryResponse_Out
  • PurchaseOrder_4010_Notification_Out
Service
Interface
with several
Operations
<Document/Data>
_<Direction: Out/In>
The Operation Names are
1:1 as the Stateless XI 3.0
Service Interfaces
  • Customer_Out
  • SalesOrder_In
Message Type <Document/Data>
<optional: Bulk>
<optional: If-Action>
<MessageCategory>

The Response Message of a
JDBC operation is always
<QueryMessage>_response

  • CustomerCreateRequest
  • CustomerCreateConfirmation
  • CustomerByIDQuery
  • CustomerByIDResponse
  • CustomerBulkChangeNotification
  • CostCenterBulkCreateInformation
Fault Message
Type
<Document/Data>
<optional: Action>
Fault
  • CustomerCreateFault
  • SalesOrderFault
Data Type = MessageType or by
adding Message as suffix
  • Inside-Out (XML/JSON/CSV/TXT/DB):
    Import what you can as External Definition!
  • Outside-In: See if you can copy a Global Data
    Type (GDT) from SAP GLOBAL 2.0
External
Definition
<Type>_<Document/Data>
  • XSD_ANSI_X12_4010_850
  • XSD_eLS
  • WSDL_DTElement
  • DTD_INVOICE
  • XSD_gasxml_v40
Operation
Mapping
<Sender-SWC>
_<SourceOperation>_To
_<TargetOperation>
without direction (_Out/_In)
  • SENDER_CustomerChangeNotification
    _To_BusinessPartnerChangeRequest

If Source and Target are equal,
they do not have to be repeated twice:

  • SENDER_CustomerByIDQueryResponse
Message
Mapping
<Sender-SWC>
_<SourceMessage>
_To_<TargetMessage><Sender-SWC>
_<TargetMessage>
_From_<SourceMessage>
for sync response
  • SENDER_CustomerByIDQuery
    _To_ClientDatabaseQuery
  • SENDER_CustomerByIDResponse
    _From_ClientDatabaseQuery_response

 

Imported
Archive
<Sender-SWC>_<Type>_<Content>
  • SAPCRM_XSLT_SalesOrder
Process
Integration Scenario
<Area>_<Function/Object>
_<opt:Sub-Area>
  • FIN_CostCenter
  • OTC_SalesOrder
  • PTP_PurchaseOrder
  • MDM_Product
Scenario
Action
<Activity>
_<Document/Data>
  • Send_CustomerChange
  • Check_CreditLimit
  • Create_SalesOrder
Channel
Template
<AdapterType>
_<Direction: SND/RCV>
_<opt:Object>
_<opt:Version>
  • IDOC_RCV_740
  • FILE_SND_AccountingDocument
  • REST_SND_Customer

 


DIR

Name Pattern  Example
Configuration
Scenario 
<Area>_<Function/Object>
_<opt:Sub-Area>
  • FIN_CostCenter
  • OTC_SalesOrder_Ecommerce
  • PTP_PurchaseOrder
Communication
Channel
<AdapterType>
_<Direction: SND/RCV>
_<opt:Technical Details>
_<opt:Objekt>
  • IDOC_RCV_EOIO
  • FILE_SND_EOIO_AccountingDocument
  • MAIL_RCV_Attachment
Business
Component (A2A)
<Application Name>
  • EMAIL
  • SFTP
Business
Component (B2B)
<PARTY>
_<Application/Technology>
  • DHL_SFTP
  • METRO_EDI
Business
Component (B2G)
<PARTY>
_<Application/Technology>
  • BZST_HTTP
Business
Component (BPM)
BPM_<Application/Context>
_<Data/Doc.>_<Action>
  • BPM_SAPS4_Orders_Aggregate
Party
(B2B/B2G)
<Company> /
<Government Organization>
  • DHL
  • Metro
  • BZSt
  • Clearingstelle
  • Google
  • EZB
Folder
(Eclipse)
Will be shown as Configuration
Scenario in Integration
Builder, however a different
naming should be used,
e.g. adding _ as prefix
  • _FIN_CostCenter
  • _OTC_SalesOrder
  • _PTP_PurchaseOrder
iFlow
(Eclipse)
<Sender-System
w/out TransportLevel>
__<Sender-Interface>
  • SAP_ERP__MATMAS_MATMAS05
  • MS_CRM_DYNAMICS__CustomerChange
  • WEBSHOP__EcomOrder

Content Organization in SAP PI for Interface Designers

This article is part of our Integration Cookbook.

ICB2

To design interfaces for SAP PI, you need to consider the following aspects and follow the principles regarding Software Components:

  • Separate Sender Application/System from Receiver
  • Mappings are always created in the Receiver (Target) Software Component
    • Generic Mapping functions can be created in a central, reusable Software Component
      • E.g. _MAPPING 1.0 of <vendor.com>
      • This Software component can be used as an underlying SWC in other components
  • BPM processes are ideally “speaking” the language of the sender OR receiver – sometimes they have their own set of interfaces. Based on that, the content is created on sender, receiver or a BPM-specific SWC.
  • Always use a /BPM namespace to indicate the usage

ICB3

  • Make Use of Process Integration Scenarios (swimlanes) to design your interfaces (and automatically document them)
    • They are located in a generic (central) software component: _INTEGRATION_SCENARIOS 1.0 of <vendor.com>
      Choose Software Components with internal Actions for Business Systems and
      Templates with local Actions for Business Component (also BPM Process)
  • The Vendor is always the creator of the Integration Content

ICB1


Categories (identify the usage area of your software component and be aware where it belongs)

A2A Internal/own systems communication: SAP, Databases,
File-Servers (NFS), JMS Queues, Inhouse Apps, Cloud Apps
B2B External Business Partners, e.g. EDI, Dun & Bradstreet,
Schufa, Public WebServices, general “External” Communication
B2G Business-To-Government: Elster, ECB, …

Areas (examples, use them in Process Integration Scenarios)

FIN Financials-orientied processes: Integration with
Payment Service Providers, Treasury Systems, Banks
LE Logistics Execution: Warehouses, Logistics Service Providers
OTC Order-To-Cash: Sales & Distribution
PTP Procure-To-Pay: Materials Management
MDM Master Data Management
HCM Human Capital Management

Messaging Principles for Interface Designers

This article is part of our Integration Cookbook.

Quality of Service

If you can influence the application design of the sender or receiver (doesn’t apply if an external partner/service provider with an existing service is involved):

QoS = Quality of Service

  • BE: Best Effort (Synchronous: the sender waits for immediate response. Basic integration which every beginner understands.)
  • EO: Exactly Once (Asynchronous: decoupled integration, a response might be answered by another message)
  • EOIO: Exactly Once in Order (Asynchronous, but serialized with specific queues)

Recommendations

  • Asynchronous Communication where possible, to reduce dependencies between sender and receiver applications and enable decoupling. Real-time Integration can also be achieved asynchronously, e.g. by using events to trigger an immediate message exchange. (Asynchronous does not mean batch mode!)
  • Synchronous Communication only where necessary: Quick Implementation (Error Handling is pushed to the sender/consumer), but strong dependency regarding development/maintenance as well as for the runtime behaviour. There should be a good reason to go for synchronous integration. Examples: Creditlimit-Check, ATP-Check.
  • BUT: Don´t be afraid, if the best fit is a synchronous setup – in the past there was a rule of thumb, to avoid synchronous interfaces in PI (for technical reasons). This does NOT apply anymore. PI serves this pattern very well by exposing SOAP/REST based services, e.g. to connect with databases (via JDBC) and decouple them from the front-end.

Usage of Queues (Async Only)

  • Dynamic Queue Assignment where possible using the object id  (Order, Material, Customer, …) for single messages
  • Static Queue Names for bulk messages (if the throughput allows it, see below)
  • No Queues, if high volume requirements exist (parallelization and mass processing with bulk messages). To ensure correct serialization (sequencing of objects), use timestamps inside the messages.

Message Throughput

For mass data scenarios, consider the following principles:

  • The ideal message size is within 500 KB and 10 MB in SAP PI
  • EO Messaging (Avoid EOIO with static queue names to allow  parallelization)
  • Consider direct (P2P) vs. mediated communication (EAI/HUB)

Design Principles for Interface Designers

This article is part of our Integration Cookbook.

When we build integration solutions, the following characteristics (principles) of application integration should be clear and defined as goals:

  • Enable Connectivity by integrating heterogeneous applications (protocol/format, security, routing, reliability)
  • Transparency through centralization of integration content
  • Control of interface operations through centralized monitoring
  • Efficiency through a standardized and pattern-oriented development process
  • Decoupling of application to minimize dependencies (separate sender from receiver)
  • Reuse of integration artefacts to reduce development effort and double maintenance
  • Simplification of complexity through a harmonized/unified modeling

Outside-In vs. Inside Out

The main question is: Do both applications use existing (import/export) interfaces/APIs or shall new ones be created?

  • Outside-In: Interfaces are modeled from scratch (typically in ESR) and both applications implement the interface structure (e.g. defined by a WSDL)  or
  • Inside-Out: Connecting existing interfaces via mappings. Most of the time we find existing structures like XSDs or WSDLs, IDoc/RFC, DTDs as well as File/CSV structures and database tables. The best is to import those structures as External Definitions. However, it is much more pragmatic to simply create a Data Type and Message Type in the ESR (although this is not an outside-in approach) based on an existing file as we do not have an equivalent XSD available.

Usually mixed versions exist, e.g. the sender side has an existing CSV export format and the receiver is an SAP sytem with an Enterprise Service (ABAP Proxy)

To consider:

  • Where does the business logic and integration logic take place?
  • Are enrichments necessary in the sender system/middleware?
  • Does the middleware have to calculate values in the mapping?
  • Are look-ups needed (e.g. to convert code lists)?

Integration Considerations

Which aspects are relevant for an interface?

  • Processing Time
  • Realtime (Event-based/manually) vs. Batch (Job)
  • Data-/Message-Volume
  • How business critical is the interface? (high availability)
  • Queuing/Serialization
  • Error-Handling
  • Usage of Acknowledgements
  • Retry & Correction methods
  • Archiving of messages

Business Logic in Interfaces 

We hear this sentence many times: “the business logic should not be in PI / in the middleware”. Why? Where does it come from? It is a 1990´s mentality where all the integration logic was programmed into the (file) export or import program. This time is over!
It makes absolutely sense to put intelligence and integration logic into a middleware.

Sender and receiver systems/applications are typically not harmonized, so it is key to have a smart middleware to covert whatever is necessary and make them speak to each other.

Examples:

  • Simple Currency/Language Code conversions
  • Status Code derivation (e.g. SAP knows 80, XYZ knows only 5)
  • Account determinations (G/L account, cost center, etc.) based on different criteria only the sender knows
  • Apply 20% discount on an amount
  • Summarize quantities of line items into a total value
  • Aggregate/Split positions of (bulk/compound) messages

Those are typical mapping tasks of a middleware, not only moving field values from one structure to another and make sure the data is transported.

Another aspect is to have a clearly defined responsibility on the maintenance of a mapping table. That´s why it makes sense to have the functional IT team controlling the value mapping table (and the middleware is accessing the table e.g. remotely via lookups).