Bootstrap data with identifiers

Hi, I have problems importing data via bootstrap. The problems are similar to the previous post: https://www.outsystems.com/forums/discussion/33907/bootstrapping-with-identifiers/. The solution found by the poster works, although I do get warnings for data issues and it should simply not work like this. 

The correct way of handling was not mentioned in the post. Can some-one help me out?

The problem is as follows: The entity has two external references. Those references have the exact same name, and the datatype is refering to the key in the applicable entity. But while bootstrapping, the following message appears: "Not included in the bootstrap action". Clicking the message makes the following popup to appear: Excel column "DSOId" does not match the type of entity attribute "DSOId". Excel column type: Integer. Entity attribute type: 'DSO Identifier'. 

Can someone help me out?


Best...Coen

Coen,

Would it be possible for you to share your .oml file?

Just tried to create an example for boostrapping tables with foreign keys and now I probably see your point.

What you could do is to create that bootstrap action manually, it's not that complicated.

If you need an example of a bootstrap procedure you could create a simple entity without any foreign keys, use Service Studio to generate some bootstrap code and use it as a starting point for your code.

Make sure you define a correct structure for your Excel file, don't forget to use Long Integer data type in your structure on foreign key fields, you'd change that later, when calling your EntityCreateOrUpdate action, you would need to call LongIntegerToIdentifier() function to convert Long Integers to Identifiers for the platform to handle them correctly.

Hope this helps.

I could create an example for you if you could attach your .oml and Excel file here.

Processing Upload...

Vitaly Martynenko wrote:

Coen,

Would it be possible for you to share your .oml file?

Hereby the OML.


Vitaly Martynenko wrote:

Just tried to create an example for boostrapping tables with foreign keys and now I probably see your point.

What you could do is to create that bootstrap action manually, it's not that complicated.

If you need an example of a bootstrap procedure you could create a simple entity without any foreign keys, use Service Studio to generate some bootstrap code and use it as a starting point for your code.

Make sure you define a correct structure for your Excel file, don't forget to use Long Integer data type in your structure on foreign key fields, you'd change that later, when calling your EntityCreateOrUpdate action, you would need to call LongIntegerToIdentifier() function to convert Long Integers to Identifiers for the platform to handle them correctly.

Hope this helps.

I could create an example for you if you could attach your .oml and Excel file here.

Is this the procecure explained in this post? 

https://www.outsystems.com/forums/discussion/48480/bootstrap-foreign-keys/

Because I was able to follow all steps, and after that I did the bootstrap again, but still the foreign key field could not be linked. I finally managed to get some values in the foreign key while bootstrapping with some converter included (LongIntegerToIdentifier), but than the values were not correct....


Still struggling:-)



Coen Hendrikx wrote:

Vitaly Martynenko wrote:

Just tried to create an example for boostrapping tables with foreign keys and now I probably see your point.

What you could do is to create that bootstrap action manually, it's not that complicated.

If you need an example of a bootstrap procedure you could create a simple entity without any foreign keys, use Service Studio to generate some bootstrap code and use it as a starting point for your code.

Make sure you define a correct structure for your Excel file, don't forget to use Long Integer data type in your structure on foreign key fields, you'd change that later, when calling your EntityCreateOrUpdate action, you would need to call LongIntegerToIdentifier() function to convert Long Integers to Identifiers for the platform to handle them correctly.

Hope this helps.

I could create an example for you if you could attach your .oml and Excel file here.

Is this the procecure explained in this post? 

https://www.outsystems.com/forums/discussion/48480/bootstrap-foreign-keys/

Because I was able to follow all steps, and after that I did the bootstrap again, but still the foreign key field could not be linked. I finally managed to get some values in the foreign key while bootstrapping with some converter included (LongIntegerToIdentifier), but than the values were not correct....


Still struggling:-)



Well, yes, it's basically the same approach. Let me work with your oml to provide an example.


Oml takes quite some time to be processed for some reason

Hello Coen, 

When you bootstrap data that has references, it is hard to guarantee that the ids will match the destination entity. 

I see two approaches here. 

1. The id in the referenced entity is set to be NOT autonumber. This means you need to provide a value when creating records, including when bootstrap that entity. 

But them, those IDs will be fixed and you can use them without problems in your Entity with the FK, following the approach provided previously. 

Note that using the mentioned approach without setting the Id of the destination to be manually defined obliges you to bootstrap first the referenced entity, take the IDs, fix the Excel and only after that you can do the bootstrap of the entity with the FKs. Also, if you need to re-execute the bootstrap, you will have to update the excel and replace the one in the server, before doing the boot rap of the entitie with the FKs, because everytime you delete the data, new data will just have the IDs starting from the previous given value by the database. 

Other alternatives were to mess with the database (not worth the dangers). 

2. You find a column in the references entity that can be used as a lookup column. This way, you replace the FK column in the excel by the values of this lookup column. 

During bootrap, you need to add this attribute to the excel structure, and change the bootstrap. The change is simple. Inside the ForEach, use an aggregate to find the record, in the referenced entity, using the lookup column from the excel. The aggregate will return the actual ID of that like, and you can use it to create the new record, as the value to the FK. 

I usually follow the second approach. 

Cheers 

Thanks for your input. I have done some experimenting and might have find another way:

- Prepare the excel to have every column in it, but not the forein key column(s)

- Bootstrap the excel without the forein keys

- Go to Data tab; structures; applicable structure: Add for every foreign key an structure attribute

- Set mandatory to Yes

- Logic tab: Bootstrap server action: AssignSubjectRecord: Add the assignments for the foreign key-attributes

- In Excel: add the column(s) holding the foreign keys. Make sure to give them the exact same name.

- Data tab: Delete the entity

- Publish

- Data tab; Database; "Import New entities from Excel"; select the entity

- Publish (after publish, still no data is in the entity)

- Datatab; Database; Advanced; "Update Action to bootstrap from Excel"

- The entity should have data now

- Set the foreign-key attributes from Integer to the applicable Subject Identifier.


Seems to work:-). Do I harm the dataset in whatever way following those steps? Or is this a clean way of getting the foreign key columns in OS?

Solution

Hi,

For me is too cumbersome. ;)
In the end, It's just a variation of the method you saw previously.

And you end up with the exact same problems that I pointed. If you need to do the bootstrap again due to the fact you may have the need to wipe out the entities, something that we do with some regularity at the first stages of the development, if your "referenced" entity does not have a no-autonumber attribute, you have to do everything again.

A lookup in the Excel, if possible, is usually best, as it may need an extra effort on changing the bootstrap, you then are free to wipe out the entity without having to worry with this process again.

Cheers!



Solution

Hi,

Oml is still not available unfortunately but anyway let me describe a possible solution for bootstrapping data.

For each data you initially bootstrap add an UUID as identifier, create an extra entity to map those UUIDs to actual identifiers you'd have after inserting data to tables.

And than you could use those UUIDs to map entities between each other regardless of the time and environment you run those bootstraps. You could even use those mappings for future additional bootstraps.

We used this approach in a project recently and it works like a charm.

Cheers!

Thanks you all for your reactions. As I have to demonstrate the abilities of os for nu organization soon, i took the cumbersome solution:-), however I hope to build a production variant soon, respecting your suggestions!