89
Views
10
Comments
Solved
How do I paginate data from a SOAP API webservice call
Application Type
Reactive

Hi All,

Very new to Outsystems, trying to evaluate the software with a proof of concept.

I am consuming data from a third party API through a SOAP Webservice call. The data is being returned ok, and I have managed to load it into a table, but I dont seem to be able to easily enable pagination. The API does not support pagination. I was hoping that I could load the whole data set, and have the table only display the records I want displayed.

I have created the startindex and maxrecords local variables, and created a client action that updates a Newstartindex value, but the table loads the full data.  

I suspect what I want to do is not possible, it would be good if someone could confirm this.

I read a post saying you could load the API retrieved data into a temporary db cache, does that sound plausible and easy(ish).

Whilst we are on the subject, I'm assuming that I might also struggle with sorting the data?

Thanks in advance

Matt

2023-10-21 19-42-11
Tousif Khan
Champion
Solution

Hello 

For the use case you can refer to this-
And there is a Article on it have a look at it .

https://itnext.io/paginating-sorting-and-filtering-data-from-a-data-action-f5cb997fdc60

Hope it will help

Thanks


2020-09-15 13-07-23
Kilian Hekhuis
 
MVP

Thanks for posting my article, I was about to do that :D.

2023-10-21 19-42-11
Tousif Khan
Champion

Hi @Kilian 

This was really infomative , It did clear my lots of doubts. Thank you for such a great contribution 

2020-09-15 13-07-23
Kilian Hekhuis
 
MVP

You're most welcome :). If you have any questions or doubts let me know.

2020-09-15 13-07-23
Kilian Hekhuis
 
MVP
Solution

Hi Matthew,

As you can read in my article that Tousif shared, paginating the output of a REST or SOAP API is not trivial, though doable. The alternative is indeed to put the data in a temporary table, but depending on the amount of data retrieved, storing the data in the table will take quite some time, so I would advise against it, unless it's fairly static data and you can easily build some synchronisation.

2023-10-21 19-42-11
Tousif Khan
Champion
Solution

Hello 

For the use case you can refer to this-
And there is a Article on it have a look at it .

https://itnext.io/paginating-sorting-and-filtering-data-from-a-data-action-f5cb997fdc60

Hope it will help

Thanks


2020-09-15 13-07-23
Kilian Hekhuis
 
MVP

Thanks for posting my article, I was about to do that :D.

2023-10-21 19-42-11
Tousif Khan
Champion

Hi @Kilian 

This was really infomative , It did clear my lots of doubts. Thank you for such a great contribution 

2020-09-15 13-07-23
Kilian Hekhuis
 
MVP

You're most welcome :). If you have any questions or doubts let me know.

2020-09-15 13-07-23
Kilian Hekhuis
 
MVP
Solution

Hi Matthew,

As you can read in my article that Tousif shared, paginating the output of a REST or SOAP API is not trivial, though doable. The alternative is indeed to put the data in a temporary table, but depending on the amount of data retrieved, storing the data in the table will take quite some time, so I would advise against it, unless it's fairly static data and you can easily build some synchronisation.

UserImage.jpg
Matthew Key

Thanks @touseef ahmed, I've yet to try it but it looks fairly straight forward. And thanks @Kilian Hekhuis  for the advice on a temporary table, if I did want to explore this as well, what woudl roughly be the best practice way to do this?


Thanks 

2020-09-15 13-07-23
Kilian Hekhuis
 
MVP

That depends a bit on what data is in the remote system, and how static it is. If it is highly dynamic data, constantly changing or being added to, for me it doesn't make sense to try to mirror it. If it's data that's only updated a few times a day, you could first bootstrap it in its entirety to an entity (by just using the SOAP call, then iterating with a For Each and perform an entity Create), and then do a periodic synchronization, calling the SOAP, updating changed records, adding new ones and deleting deleted ones. But that's all just basic data synchronization.

UserImage.jpg
Matthew Key

Thanks for that @Kilian Hekhuis it worked a charm, I now have the data being delivered through SOAP webservice call paginated.

 

Can I ask one more piece of advice please. 

 

The main app we are looking to build is a rebuild. The current app has data retrieved through a SOAP webservice, and it is displayed on screen using a jquery plugin called datatables, you can see them in action here at 

 

https://datatables.net/

 

This allows us to use JSON formatted data with the table, and it handle things like, pagination, sorting, searching, column filtering, mobile responsive, changing number of records to be displayed etc all handled within the client, therefore incredibly quick. Our data rarely exceeds 5,000 records and this preforms excellently.

 

I'm now looking to build this in outsystems, and it feels like there are a few options to do this

  1. Retrieve the data from     Webservice SOAP into temporary local fields and then use these as source     for Outsystems Table (as we have previously been working on), and then     build  the additional functionality     as stated above into this
  2. Replicate/sync the data into     Outsystems database data structures and use the tables linked to     Outsystems entities, then use all of the functionality that is already     included in these.
  3. Somehow add the Datatables     plugin into outsystems, and use this similarly to how it is being used     now, load all of the data through the initial SOAP webservice call. I     assume this comes at a cost in terms of functionality, i.e. You can't drag     and drop, or all editing is done in code etc.


In terms of our data, per user we generally load around 5000 records into these tables, in extreme cases up to 10,000. It is rare that the data would change in the back end whilst it was loaded, however changes could be made by the user online, and we would therefore have to reflect those changes back to them. It would seem quite easy to send all changes to the local DB cache, and also through the SOAP webservice too though? Loading the data isnt particularly intensive, so if we needed to refresh the entire local db cache on any update/addition that might work too.


Out of these three options, and the fairly limited information I have provided, which would you steer towards? And any gotchas in each?

2020-09-15 13-07-23
Kilian Hekhuis
 
MVP

Hi Matthew,

I'd advise you to start a new post, as it's basically a different topic. Thus more people will see your question, and we keep the posts as single-question ones. Also can you mark one or more answers as Solution? Thanks!

Community GuidelinesBe kind and respectful, give credit to the original source of content, and search for duplicates before posting.