Integration Principles
As architects we often find ourselves designing solutions which bring together multiple, sometime disparate systems. These systems support different business functions, and bringing the data from these together can provide a much richer and more holistic view of the business.
When considering integrating systems, my experience has taught me the following design principles that one should take into account:
1) Timing Requirements
Does data need to be synched in real-time between systems, or can it be performed in a batch operation. When and how quickly do the end business users need access to information and what impact does not having this have on the business. Answering these questions help us decide whether information should be synched in batches or in real-time.
2) Synchronous vs Asynchronous Operations
From an integration point of view:
Synchronous processing involves initiating and completing a transaction in a single operation.
Asynchronous processing involves initiating a transaction, but completing it in the background.
If information needs to flow in real-time to a target system, and the confirming of successful processing is required immediately, synchronous is the preferred approach.
If we can submit a request to the target system, but we do not need to wait for the successful confirmation of completion, asynchronous is the preferred approach.
Synchronous Processing
Putting this in context of Salesforce lets take an example. Suppose we are submitting orders from Salesforce to an ERP system. If the order preparation and delivery aspects of the order process need to start immediately, it makes sense to transmit the order in real-time to the ERP. In addition to this, if we need to immediately know that the order has reached the ERP system, we would perform a synchronous transaction. This ensures the user know immediately if an issue has occurred when sending the record to the ERP system.
Asynchronous Processing
An example of asynchronous processing can be the creation of Accounts between systems. If the creation of an Account in the target system can be something which can continue to be processed in the background without the user having to wait for the operation to complete, we can take this approach. This ensures the user is not kept waiting for an operation to occur, and can be used in cases where a positive confirmation is not expected/required for a business process to continue.
3) Maintain good Data at Source
The topic of Data is a complete science on it's own, however it is important during integrations we promote good data. Data validation should occur in the source system, ensuring it meets defined business quality standards before it becomes in-scope for the integration. In Salesforce, the use of validation rules, page layouts (required fields) and trigger checks/error messages can ensure data meets quality standards before it is passed to an ERP system.
4) Perform transformation and integration processes in the middleware
The middleware layer should perform necessary data transformations before passing data to the target system. Keeping this functionality in the middleware ensures that source and target system can focus on the tasks they were designed for, with the middleware taking responsibility for ensuring data is in the right format for presentation to the target system.
In addition to this, an objective should be to ensure that future updates can be easily made without breaking the integrations. An example could be to send a lightweight message to the middleware layer to trigger the integration processes. This ensures that changes can be localised to the middleware layer, when updates are required, and not all the systems.
An approach I have adopted recently is sending an ERPMessage via ApexCallouts using REST to an endpoint in the middleware layer. The message contains the following content:
ERPMessage {SFRecordId:xxxxxx; Action:Create; Flow: Account}
This simple message provides the middleware with enough information to identify the Salesforce record in question, to know the operation to perform and to know the integration flow we would like to trigger. The middleware can then easily query Salesforce using the SOAP API to retrieve the Account. After performing necessary transformation, the details can be passed on to the ERP system.
5) Security
Keep information secure, encrypting the transformation layer using digitally signed certificates and using Named Principles in salesforce to store the authentication details, and passing these down in the header of the message(s) being transmitted to the middleware solution.
Above are a few points to get us started, stay tuned for part 2 with some more pointers on integrations.
When considering integrating systems, my experience has taught me the following design principles that one should take into account:
1) Timing Requirements
Does data need to be synched in real-time between systems, or can it be performed in a batch operation. When and how quickly do the end business users need access to information and what impact does not having this have on the business. Answering these questions help us decide whether information should be synched in batches or in real-time.
2) Synchronous vs Asynchronous Operations
From an integration point of view:
Synchronous processing involves initiating and completing a transaction in a single operation.
Asynchronous processing involves initiating a transaction, but completing it in the background.
If information needs to flow in real-time to a target system, and the confirming of successful processing is required immediately, synchronous is the preferred approach.
If we can submit a request to the target system, but we do not need to wait for the successful confirmation of completion, asynchronous is the preferred approach.
Synchronous Processing
Putting this in context of Salesforce lets take an example. Suppose we are submitting orders from Salesforce to an ERP system. If the order preparation and delivery aspects of the order process need to start immediately, it makes sense to transmit the order in real-time to the ERP. In addition to this, if we need to immediately know that the order has reached the ERP system, we would perform a synchronous transaction. This ensures the user know immediately if an issue has occurred when sending the record to the ERP system.
Asynchronous Processing
An example of asynchronous processing can be the creation of Accounts between systems. If the creation of an Account in the target system can be something which can continue to be processed in the background without the user having to wait for the operation to complete, we can take this approach. This ensures the user is not kept waiting for an operation to occur, and can be used in cases where a positive confirmation is not expected/required for a business process to continue.
3) Maintain good Data at Source
The topic of Data is a complete science on it's own, however it is important during integrations we promote good data. Data validation should occur in the source system, ensuring it meets defined business quality standards before it becomes in-scope for the integration. In Salesforce, the use of validation rules, page layouts (required fields) and trigger checks/error messages can ensure data meets quality standards before it is passed to an ERP system.
4) Perform transformation and integration processes in the middleware
The middleware layer should perform necessary data transformations before passing data to the target system. Keeping this functionality in the middleware ensures that source and target system can focus on the tasks they were designed for, with the middleware taking responsibility for ensuring data is in the right format for presentation to the target system.
In addition to this, an objective should be to ensure that future updates can be easily made without breaking the integrations. An example could be to send a lightweight message to the middleware layer to trigger the integration processes. This ensures that changes can be localised to the middleware layer, when updates are required, and not all the systems.
An approach I have adopted recently is sending an ERPMessage via ApexCallouts using REST to an endpoint in the middleware layer. The message contains the following content:
ERPMessage {SFRecordId:xxxxxx; Action:Create; Flow: Account}
This simple message provides the middleware with enough information to identify the Salesforce record in question, to know the operation to perform and to know the integration flow we would like to trigger. The middleware can then easily query Salesforce using the SOAP API to retrieve the Account. After performing necessary transformation, the details can be passed on to the ERP system.
5) Security
Keep information secure, encrypting the transformation layer using digitally signed certificates and using Named Principles in salesforce to store the authentication details, and passing these down in the header of the message(s) being transmitted to the middleware solution.
Above are a few points to get us started, stay tuned for part 2 with some more pointers on integrations.
Comments
Post a Comment