Logic Clarification (or modification)
8
Views
0
Comments
New
Training

I would like to suggest a logic clarification (or modification) to the training for the archiving.  It's possible there has been an oversight, or the training may lead to a dangerously wrong solution, or I'm just an idiot.  (More than one option may apply.)

https://learn.outsystems.com/training/journeys/web-developer-exam-2993/data-archiving-best-practice/odc/7017

Here is the current logic in question:

Text below it it, whole section pasted for clarity:

  1. Set Archive Threshold - Sets the number of records to archive by iteration. This threshold can be defined using a Setting, so it can be adjusted without redeploying the app.

  2. Set StartTime - Sets a local variable with the current time when the execution starts.

  3. CheckRecordsToArchive - Validates if there are records left to archive, according to the conditions defined using the control columns.

  4. Has Records to Archive? - Validates the output of the previous query and executes the archive branch if there are records to archive.

  5. MoveIntoArchiveStorage_InBulk - Executes a bulk INSERT INTO SELECT statement from the primary storage to the archive storage, combined with a clause to limit the amount of archived records (using the Archive Threshold set in 1). It implements the archive condition.

  6. MainStorage_SetArchivedFlag_InBulk - Executes a bulk UPDATE to set the IsArchived flag of the records archived in the previous step.

  7. Can Process More? - Validates if the current execution duration is approaching the Timer’s timeout (using an expression like DiffMinutes(StartTime, CurrTime()) < 15, for a default timeout of 20 minutes). If there’s still time, it continues checking for more records to archive. This approach helps prevent the Timer from timing out and having to repeat work.

  8. CommitTransaction - Commits the transaction to make all changes permanent in the database.

  9. WakeArchiveData - Re-wakes the Timer to check if there are still records to archive.


My suggestion it that  something should be added to step 6 "MainStorage_SetArchivedFlag_InBulk" to explicitly clarify that they should not use the exact same logic (where clause) used in step 5 to select items in step 6, because the data may have changed, leaving the data in query different.

This will result in the wrong records being flagged as archived, and subsequently getting deleted by the clean-up process.  They need to do some sort of check (inner join?) against the archive table.

I understand this is an edge case, but we all know one in a million edge cases tend to happen within a week of functionality going live.

Maybe there is an assumption that someone will know to do it this way, and I'm just being dumb, but the wording of the guidelines would tend to lead to someone reusing the same query logic, especially as an inner join query would be assumed to be much slower....?

Does anyone from OutSystems training, or in fact, their business processes team, have any thoughts on this.  It this a flaw in the training content, or in the archiving logic that was envisioned?

Or am I just barking up the wrong tree here? 

(I'm guessing this scenario is used in the exam as well.)