Data integration for SAP systems with Apache Kafka
Connect multiple sources to your SAP system with pre-validation of semantic correctness, referential integrity, and full error handling and transparency of corrupt datasets.
Xeotek SAAPIX is our smart data integration solution for SAP systems with Apache Kafka. Xeotek SAAPIX meets the data quality requirements of enterprises and offers full transparency and control in case of corrupt or incomplete data sets.
- Semantic validations of incoming data
- Validation of referential integrity of correlating data sets
- Support for time-based dependencies of sources and data sets
- Easy handling and monitoring of corrupt data sets
- Support for data transformation and staging
- Customizable components
- Scalable microservice architecture
We have designed Xeotek SAAPIX to help our customers focus on their business aspects when integrating data instead of solving technical problems.
Click on the image to enlarge.
Errors do happen. Xeotek SAAPIX validates data before loading it into the target system to make sure that no corrupt data sets enter your SAP system. In case an erroneous data set is discovered, the data set with some additional hints is forwarded to a dedicated topic and can be further inspected. After correcting the data set, a new attempt to load the data can be made.
Xeotek SAAPIX integrates seamlessly with our Apache Kafka monitoring and development platform KaDeck. KaDeck offers an intuitive user interface, a visual representation of your connected sources and your data flow, and many more features to give your team full control over the data flow.
However, Xeotek SAAPIX also works without leveraging the capabilities of KaDeck.
Data integration for SAP with Apache Kafka is complicated when multiple sources are involved. With Xeotek SAAPIX we have designed a solution that can be highly customized, making it the ideal solution for any Apache Kafka to SAP data integration scenario.
Xeotek SAAPIX can be used to connect to almost any SAP system that supports remote function calls via SAP JCo (Java Connector).
Though writing custom semantic validation checks or data transformation components is easy, we also have some general components already available for SAP FPSL, SAP FSDP, and Smart AFI that can be customized to your needs.
Reach out to us for more information on our available integrations.
Need to connect to other systems as well?
We are working on further integrations. Don’t hesistate to contact us, if you need to integrate data from multiple sources via Apache Kafka to other systems as well.
To support all kinds of different integration scenarios, we have designed Xeotek SAAPIX to be easily extendable through plugins and configurations.
Xeotek SAAPIX consists of three parts:
- a microservice with semantic validators and data mapping components as plugins
- an RFC-enabled SAP function for integrity checks and the actual data integration
- a database for configurations
Custom semantic validators and data transformation components can be loaded into SAAPIX as plugins. The database stores information about the source topics, the order of data, and much more. Therefore a Xeotek SAAPIX instance can be highly customized.
You can configure the order of which data will be consumed. It is also possible to define a timer to skip a topic if no data has arrived after a certain amount of time. If a topic is skipped, a status message will be sent to the error topic.
The data validation and transformation
Data can be processed on a per-record basis or as batches. The latter can be referred to as a staging process. This enables one-to-many and many-to-many relations. We have developed an API to make it easy to write data validation and transformation components. If a validation or transformation was not successful, the data and the error message is sent to the error topic.
Referential integrity and data loading
Once the data consumption, semantic validation, and data transformation is finished, an SAP function is called using SAP JCo. The SAP function checks whether all required (referenced) data are present in the target system and passes on the data record to the SAP system’s entry point (an SAP function). This functionality is highly customizable and depends on the actual system you want to connect.
In case the required data is missing, the data is sent to the error topic for later inspection and reloading.