Martin Stanik
Rank: #4041
4Views
3Comments
Amazon SQS Connector and how to subscribe on queue and trigger background processing
Question
Reactive
Service
Application Type
Reactive, Service
Service Studio Version
11.10.5 (Build 37061)
Platform Version
11.10.2 (Build 25738)

By using SQS service (FIFO) from Amazon, is there any way (and recommended approach from Outsystems architecture committee) of how to trigger the background process after the message arrives to queue? Something similar to Lambda triggers or HTTP/s triggers (Fanout to HTTP/S endpoints), but something that is built already in Outsystems as a plug&play component? 

I am thinking of this use case scenario:

1. data source application after making a data change on DB fires the Event-carried event to SQS

2. subscriber (some type of service - not sure which one is capable to full fill this requirement) wakes up after getting push notification from SQS and pulls down the event content and starts the processing.  


What would be your recommendation to build a solution like that in Outsystems?

0
0
on 6th Jan
Copy link to comment
Daniël Kuhlmann
mvp_badge
MVP
Rank: #17

Hi Martin,

Not with purely OutSystems. I have only got it working by have a timer that checks the queue, then starting processing it untill the queue was empty after which the timer would restart itself again.

What you could try, is use what you described already, start via an lambda trigger an OutSystems rest service that then will read the SQS queue.

I could not use it due to the SQS queue setup we needed which then did not allow the use of Lambda functions. If I recall correct if it was FIFO the lambda could not be used.

Regards,

Daniel

0
0
on 7th Jan
Copy link to comment
Martin Stanik
Rank: #4041

Hi Daniel, 

cheers for that.

With timers tho, this solution introduces a message delivery lagging which I don't want to. Messaging systems suppose to work with near-real-time latency and that is my priority (sorry should have mentioned this in my initial question).

I have built a demo with Topics and Fanout by provisioning a Web Server with Rest API action, which is used by SNS configured on as HTTPS trigger. Partially, this proves my theory (it is no a "pure" queue but a combination of both of these two), but it seems to be quite clunky. Because it introduces another need like authentication, and new subscriber (manual) subscribing process and maintenance (overhead).  

Just want to make sure that Outsystems does not have something more like "out-of-the-box" I can use and save me some time.


Thank you.

0
0
on 7th Jan
Copy link to comment
Daniël Kuhlmann
mvp_badge
MVP
Rank: #17

I agree with all you write, it's a pity and not optimal solution to use timers. In the situation I had to use them it was excaptable to have a little lag after the queue was processed. Only when the queue was empty the timer would restart.

But the answer to your original question remains, there is nothing build into OutSystems that you can leverage to skip the lagging.

0
0
on 7th Jan
Copy link to comment