Custom adapters and runtime dynamic configuration with spring integration

Six month ago, We had develop a prototype to integrate differents ERP (enterprise resource planning) information across ftp system. One of the ERP used a rest endpoint to receive information. But the other ERP, could only use the ftp system to communicate information. it could not send information using rest endpoint.

Other issue of the project is that the client would want have the full control to configure ftp options. We had build a system where the client might configure the ftp options using a portal web. We had build a listener to detect the changes in the database configuration and create dynamicly the ftp adapters

We used spring integration to develop the system. The problem is that spring integration is not very prepared to create adapter in runtime and had not inbound jdbc adapters watcher.

How did we resolve that problems?. 

• We build custom inbound jdbc adapters watcher. JdbcInMemoryChangeWatcherAdapter. The adapter  detects changes of a table and sending the change information to the next channel. The updating is detected using a stamp field (watchField). This adapter only accept primary key of string type. The adapter works comparing all physic table information with a memory table, therefore this adapter is only prepared for small tables (normally configuration tables). 
• Once that the system receives the changes, we load inbound ftp configuration from xml file configuration. Look at DynamicFtpWatcher . The ftp dynamic options is loaded using environment properties. The only problem that we found is that the bean ids can't be populated using environment variables, and we needed identifier every ftp configuration with diferrents ids. To resolve this problem, we created  PreProcessorXmlApplicationContext. This class preprocess xml configuration, replacing bean ids, before of generating GenericXmlApplicationContext. We might have load the application context using diferrents context, In this way, we would have had conflict problems, but we wanted load the ftp context into the same parent to reuse channel, transforms, etc.

In the under figure, we can see the flow of the spring integration application



1) Configuration database watcher (IntegrationConfiguration)



2) Ftp dynamic configuration watcher (FtpDynamic.xml)



Is neccesary save configuration id into header enricher, so that after the system know how to route  the rest message, (shared by all configurations) to the correct channel.

3) Send ftp file to rest endpoint (IntegrationConfiguration)



4) Remove resources (FtpDynamic.xml)



Remove ftp file and sync folder file.

The channel is only executed if  the response of the rest service is "OK" and the id is equal to original configuration. Remind the rest channel is shared by all configurations

expression="headers.ftpId.equals('${id}') and headers.http_statusCode == T(org.springframework.http.HttpStatus).OK">

5) End (IntegrationConfiguration)


 
Source code prototype: https://github.com/davsuapas/SpringIntegrationDynamic