Read and Process JSON file Objects into entity
Discussion

Hi,  

Previously, posted a questions for processing  and upload JSON content https://www.outsystems.com/forums/discussion/69732/json-deserialization-not-working-as-expected/

My objective of this discussion is to ultimately be able to read the JSON object read characters  > identify start character > count characters > identify  levels  / sub-levels of object > identify end of object :

1- need to split the file content with a character that will be identified every time reading starts

2- count how many specific characters was read "{" or  "}"

3- mark the end of each object with "," or something else

4- pass the data to an entity

since every JSON has its own depth of levels of objects, I am not sure how will I be able to run the above logic if at all doable!

I did the structure deserialize process and got the values of the JSON content which worked perfectly. However, I had to split the object one by one manually and save it into an entity. This is not a practical solution for big JSON file. 

any ideas?

mvp_badge
MVP

Hi Manah,

I'm not sure what you want to achieve here. JSON is basically a way to transport data, there's almost never a true need to try to manually parse it to extract parts of it.

Can you explain a bit more what kind of JSON you want to be able to parse, and what you want to achieve by splitting the JSON into smaller parts?

Hi Kilian, 

Appreciate your feedback. I have a JSON file content, its a public data that I extracted online from a public website in Arabic. I have this data now in JSON format and I need to use it in a way that will be available for public to search specific data rather than going thru several links / sub-links. The pages links / structure could only be saved in JSON format. 

I was hoping to get this data , which is publicly available, yet hard to navigate. I am repackaging this data and want to make it available in my PE. I watched the video about how to process JSON file using desearialize/serialize JSON object. I had to create an entity using excel bootstrap and for that I had to prepare a excel file. The preparation of this file was done manually by copying and pasting the each object into a cell. Thereafter I was able to complete the process JSON video by Outsystems: https://www.youtube.com/watch?v=4KfuGTxLQBo

Splitting the file make my big problem smaller, will be able to read objects { ... } of JSON file and add it to an entity in Outsystems. The object will hold the pair values { ... } separated by "," Assuming that I can read it all and save it with supporting data that describes the object parameters, if needed.

Object parameters will hold data about type of JSON object { ... },[ ...] in a way that I could find the query string, find the object , identify if there are sub-level in JSON file and retrieve the data.

Assuming that, above is doable, I will then make available to my Chatbot use case. This part is done and ready to listen and interact with my PE, however, I want to make JSON file content available to add more value to my Chatbot use case.

Hopefully I answered your question. If this is something that  is doable , I will be great  to get some ideas.

mvp_badge
MVP

Hi Manah,

So the basic problem is that it's difficult to deserialize the whole structure, since there's a lot of information you do not need? And the data you do need, and want to deserialize, is only a small subset of the total information?

Still, I would try to use the "Add Structure from JSON" functionality of Service Studio (context menu of the Structure folder) on the whole data set, and see if you can then deserialize the data. Once you've got everything in Structures, you can do with it whatever you want, like storing the data to a database Entity (which is the recommended approach for data you want to use in an app).

Hi Kilian,

I understand that create a structure into Outsystems will enable to me deserailize and access the data. I am not sure if my problem is clear!

Problem Background:

How to import the JSON structure file into Outsystems PE. In the video of https://www.youtube.com/watch?v=4KfuGTxLQBo , it demonstrated a process that starts with an entity which was added to the module as a dependency using the Outsystems integration. This entity was the first point where the JSON data was brought into Outsystems.


My Solution:

I had to create a entity manually "CustomerData", which is not a best solution, I believe. Thereafter I was able to play with the data. I do not thing is it a practical solution for a big JSON structure. Therefore is started this discussion.

My problem is: how to get the JSON structure content into Outsystems entity like the one highlighted. I could not find any where, based on my search / reading, that will help me work with big JSON structure file.

I hope this makes it clearer.



mvp_badge
MVP

Hi Manah,

It's still not completely clear to me. To store data in a database, you need to design a data model. Have you designed a data model before? Do you know, for example, what a relational database entails? What parent and child entities are and so forth? Because if you have never designed a data model, I can imagine you have trouble understanding how to "translate" JSON to one or several entities. And if that's the case, I think a simple forum thread is not enough to explain it all.

Perhaps you could show (a part of) the JSON structure, so we can advise you the type of Entities we'd create for it, and how to fill those.

I got the file attached. Yes, I did relational data base. But first I need to create the entities first.

runresults.json

Hello Kilian,

I really was hoping to get some guidance on how to import JSON resource list of objects in a proper entities. For the relationship between entities , I am ok with that, but not sure how to properly use outsystems based on the file attached as you suggested earlier.

mvp_badge
MVP

Hi Manah,

Unfortunately I can't read Arabic (we all have our flaws :)), but from the English labels I guess this is the JSON structure:

  • categories (list)
    • name
    • services (list)
      • name
      • selection1 (list)
        • name
        • selection2 (list)
          • name
          • Fields (list)
            • name

So if you were to put that in a rational database, for each of the lists you'd create an Entity, and to store everything in those Entities, you'd have a For Each for each list. See attachment for a naive example of just that.

Of course, this may be a bit over-rationalized, and you could go for a flatter approach, that will perform better, by having a Fields table that includes Ids for Service, Selection1 and Selection2, instead of Selection1 containing a ServiceId and Selection2 a Selection1Id. But that's more a data-modeling thing than an OutSystems thing.

StoreJSON.oml

Hello @Manah Hamadeh , maybe use excel to read the JSON, convert it to a table and select the properties you want and then you can do your magic and import to the OutSystems Database as an entity.

Reply to me, if you find this idea good enough.

Hi @Marcio Carvalho Thanks for your idea, I would really appreciate if your could share an example of how to do that?


Select the icon(1) then you choose the properties you want. In this case, I choose properties and then I click on the icon again and choose a name to bring the names of the properties from JSON


Now close and load

Then you have your selection list with all the things you wanted, you could have got all you wanted from the JSON, in this case, I just selected one column, one entity attribute.

Now I am ready to upload this to my database as an entity in my application.

The entity name, in this case, will be selection and the attribute will be name.

I did, it was fairly easy, but I need to get to know the how to use it to be able extract all those records and label them as well.

Community GuidelinesBe kind and respectful, give credit to the original source of content, and search for duplicates before posting.