Skip to Content (Press Enter)
OutSystems.com
Personal Edition
Community
Support
Training
Training
Online Training
Developer Schools
Boot Camps
Certifications
Tech Talks
Documentation
Documentation
Overview
ODC
O11
Forums
Forge
Get Involved
Get Involved
Jobs
Ideas
Members
Mentorship
User Groups
Platform
Platform
ODC
O11
Search in OutSystems
Log in
Get Started
Back to Forums
Ricardo Sousa
22
Views
14
Comments
Solved
ListDuplicate breaks Simple Queries Optimization
Question
Hi!
I'm writing this message to see if someone else experienced this issue.
We have a File Entity that has:
Filename (Text)
UpdateDate (DateTime)
Content (BinaryData)
We have a Web Screen that list File records and that allows the user to:
select all files and delete them
download files
As the web screen was taking a considerable time to load and had a considerable page size (1.5 Mb) we started to check what could be wrong.
First we noticed that Content was being used on the download screen action without first being fetched from the DB. That is problematic because that way the content need to be stored on the viewstate so we change it to fetch content just before the download action.
As Content was not being used anywhere else we were condidrent that we solved the problem, but we didn't, the page was still slow and the page size the same.
To try to figure out we did a test using an Advance Query that only gets Filename and UpdateDate and with this new query the page size was 71 Kb and the load time much better. This was what we expected but the result didn't make any sense, Simple queries only retreive what will be used so why it was retreiving the Content?
After some tests we figured out the culprit, the ListDuplicate inside the Delete All Selected.
So it seems that, even if the the ListRecords doesn't used the Content attribute, because of the use of ListDuplicate action all attributes are fetched.
After we changed the logic to not to used ListDuplicate while using a simple query the page load time improved (a lot) and the size was the same as when using the Advance Query (71 Kb)
OS version: Version 8.0.1.25
Kilian Hekhuis
Â
MVP
Solution
Hi Ricardo,
Long time no see :). Still at Logitech?
Anyway, with regards to the optimizations, as J. (where's Statler and Waldorf gone? :)) explained, the platform cannot really know what you're doing with the duplicated list once it's been duplicated. It can decide that the original list is only used in a way that allows the optimizer to limit the data retrieved, but it would be an optimizing nightmare to also trace what happens with the duplicated list. In these cases, it's a good thing to realize when and how the platform optimizes, and prevent programming structures that cause the optimizer to throw the towel in, so to speak.
See solution in context
João Heleno
Â
MVP
Hi Ricardo,
you should isolate binary data in another entity. Are you deleting your files one by one? Just use an advanced query and do a bulk delete.
Ricardo Sousa
Hi João,
Thank you for your reply, I agree with you on with the idea of having the BinaryData on another Entity but my question was more about the fact that the ListDuplicate "breaks" the optimization. Did you ever notice that?
J.
Â
MVP
Well, it's kinda obvious. listduplicate does what it has to do. Duplicates the list. It does it 100% otherwise you get nasty sideeffects... Imho its correct design
Ricardo Sousa
Hi ,
I don't agree with you, if I'm duplicating the ListRecords that only has a subset of values why it's duplicating everything?
J.
Â
MVP
Hi,
It's duplicating everything, because you are at design-time.
The structure has to be known, because if you assign attributes, alter attributes.
Since optimization is being done runtime, if you want what you want, is also optimzing every (sub)action
that *might* use the list-duplicate.
which can make life much more complicated than you actually want.
The keyword here is, optimization is being done "compiletime" and *NOT* design-time.
Ricardo Sousa
I think optimizations are done on compile time and not on runtime, and so the platform knows everything that is used by the webscreen and by its actions.
So, if the attributes are being used they need to be fetch, if they aren't then ListDuplicate is duplicating something that will not be used. Resuming the current way is, besides breaking the optimitations, copying to memory unnecessary data.
Again, maybe there is a good reason but I'm not seeing it.
J.
Â
MVP
I understand your reasoning, but the logic is getting too complex to gain actual advantages over it.
ScreenAction1: ListDuplicate, call action 1, calls action2, shows name, surname, lastname, accountnumber
Action1 : Loops through the list, duplicates odd number to a new list , calls action 3
Action2: updates the list and set the lastupdateby
Action3: loops through list, validates street
ScreenAction2: calls action 3, shows name, telephonenumber, birthdate
So, designtime you have everything.
before compiletime, outsystems needs to pass through everything recursively to actually know the complete subset of items which are actually used or not.
This means evaluate all if-conditions, switches and assigns in every action.
This is only in 1 espaqce, what if you have those actions and/or webblocks as public and you have multiple-espaces..
Hence the reason, don't overcomplicate things. just optimize a direct aggegrate from a webscreen.
Plain and simple.
Ricardo Sousa
What do you mean by "optimize a direct aggegrate from a webscreen" I'm still on version 8 so I don't have Aggregates but I would like to understand your suggestion.
PS - I had already found a way of doing what we need without using the ListDuplicate, when I created the post. This is just bothering me because I was promoting the use of simple queries because they are automatically optimized and also because scaffolding "automatically and silently" makes the pages slower, as it uses the ListDuplicate for the "Delete all selected" option (when applicable).
J.
Â
MVP
oops, aggregate is the new simple-query in 9 :)
1 reply
19 Jun 2015
Show thread
Hide thread
João Heleno
Â
MVP
I'm sorry Joost, but there is no comparison between aggregates and simple queries *wink*
Statler & Waldorf and the amazing flappy tweedles!
wrote:
oops, aggregate is the new simple-query in 9 :)
J.
Â
MVP
ok ok,
the simple query is dead, long live the aggregrate!
Kilian Hekhuis
Â
MVP
Solution
Hi Ricardo,
Long time no see :). Still at Logitech?
Anyway, with regards to the optimizations, as J. (where's Statler and Waldorf gone? :)) explained, the platform cannot really know what you're doing with the duplicated list once it's been duplicated. It can decide that the original list is only used in a way that allows the optimizer to limit the data retrieved, but it would be an optimizing nightmare to also trace what happens with the duplicated list. In these cases, it's a good thing to realize when and how the platform optimizes, and prevent programming structures that cause the optimizer to throw the towel in, so to speak.
Ricardo Sousa
Hi Kilian!
Yep, still working at Logitech :)
That's a very good example. Even if not impossible it will be a complicated trace of used attributes.
Anyway, at least Scaffolding should be optimizer friendly and not use the ListDuplicate on the "Remove Records" action. When I have more time I'll try to suggest something.
Regards!
Ricardo Sousa
Well, finding a way was stronger than me.
Here it goes:
Before:
After:
I'm using a Integer Structure List to store the "currentRowNumber"
Community Guidelines
Be kind and respectful, give credit to the original source of content, and search for duplicates before posting.
See the full guidelines
 Loading...