[Twitter Connector] Getting Tweets results in crash of deployment server

[Twitter Connector] Getting Tweets results in crash of deployment server

  
Forge Component
(3)
Published on 20 Jan by OutSystems R&D
3 votes
Published on 20 Jan by OutSystems R&D

Hi you,


Last week my personal environment crashed and with it every application I have on there.

I think I've localized the problem to fetching the timeline (of our company) and parsing it on my backend.


But everything seems to be in order, nothing that could cause the CPU usage to be in the red zone which leads to a crash of the deployment server.



Could anybody double check what I'm doing here?


backend has a button to retrieve the tweets, that, when pressed, crashes my personal environment:


This calls the Twitter Connector that I have modified for it to just return a timeline (no log in) according to someone here that helped me greatly:



where 'SetTwitterToken' is altered to just return a list of company tweets without the user having to log in to see the tweets:



This is used in my backend like so:


And this is how I configure the twitter connector in my back end module:


The timeline is retrieved, stored in a local list and looped over.

For every 'Twitter tweet' I:

1) parse the text so it filters out the html entities (in an ugly way):

2) get the main media url for the image in the Tweet. I download the picture and convert it to base64 for storage (takes 2 server actions per image, I'm thinking this could be it, but it has worked before)

3) assign the values to a simplified version of a Tweet 



So doing this has led to a crash of my deployment server but to me it does not seem to be prone to error, apart from the many server actions that can take up CPU usage I suppose

Lars De Pauw wrote:

Hi you,


Last week my personal environment crashed and with it every application I have on there.

I think I've localized the problem to fetching the timeline (of our company) and parsing it on my backend.


But everything seems to be in order, nothing that could cause the CPU usage to be in the red zone which leads to a crash of the deployment server.



Could anybody double check what I'm doing here?


backend has a button to retrieve the tweets, that, when pressed, crashes my personal environment:


This calls the Twitter Connector that I have modified for it to just return a timeline (no log in) according to someone here that helped me greatly:



where 'SetTwitterToken' is altered to just return a list of company tweets without the user having to log in to see the tweets:



This is used in my backend like so:


And this is how I configure the twitter connector in my back end module:


The timeline is retrieved, stored in a local list and looped over.

For every 'Twitter tweet' I:

1) parse the text so it filters out the html entities (in an ugly way):

2) get the main media url for the image in the Tweet. I download the picture and convert it to base64 for storage (takes 2 server actions per image, I'm thinking this could be it, but it has worked before)

3) assign the values to a simplified version of a Tweet 



So doing this has led to a crash of my deployment server but to me it does not seem to be prone to error, apart from the many server actions that can take up CPU usage I suppose

Hello Lars De Pauw,

Did you try to remove the image conversion action from the cycle and run it? I think it's a good way to despite if is that action that is giving the deployment server crash.


Another thing that I would recommend you to do is remove the image conversion action from the cycle and do the conversion asynchrony. 

Cheers,

Magda Pereira

Magda Pereira wrote:

Did you try to remove the image conversion action from the cycle and run it? I think it's a good way to despite if is that action that is giving the deployment server crash. 


Hi Magda,


I removed the image conversion completely. I only needed it because Android can't fetch images with non-http urls. But I overlooked the 'MediaUrlHttps'-attribute of the Tweet entity, so I didn't need the conversion after all! Works like a charm now :)