[XML Records] Optional Number elements: a way to control if '0' should be left out or added.

[XML Records] Optional Number elements: a way to control if '0' should be left out or added.

  
Forge Component
(24)
Published on 15 Jun by Ricardo Silva
24 votes
Published on 15 Jun by Ricardo Silva

Looks like more and more is being added to this Forge Component and it's great!

I do however have a problem with number fields being added even though the data was empty.

Now i know that it's impossible to distinguish for a 0 value if it's really the number 0 or if it's NULL because of the outsystems engine, however would it be possible to add a configuration that would leave 0 values out if the field was optional in the xsd?


I really hope Outsystems will get a featurecomplete native implementation of xml and xsd's meaning they would have to start supporting null values as well (which would be a big step towards easier integration of Outsystems in a company application ecosystem meaning greater and faster adoption by more companies of the platform)


XML Records already has an option to "Exclude If Null"

Are you setting this to true and it's not working for you?

Ricardo Silva wrote:

XML Records already has an option to "Exclude If Null"

Are you setting this to true and it's not working for you?

Yes i set Exclude if Null.

It works for the Text Fields, but the output of number fields (Decimals in my case) seem to ignore that setting.


You can configure what is the "null value" for a specific parameter. The default is "empty". Since 0.0 or 0 is not empty, it doesn't "trigger".


Have you tried configuring your specific parameter to have a different "null value" according to your requirements ?

Ricardo Silva wrote:

You can configure what is the "null value" for a specific parameter. The default is "empty". Since 0.0 or 0 is not empty, it doesn't "trigger".


Have you tried configuring your specific parameter to have a different "null value" according to your requirements ?

Thanks, didn't know that was configurable on a per element basis! Going to try it this afternoon.


Ricardo Silva wrote:

You can configure what is the "null value" for a specific parameter. The default is "empty". Since 0.0 or 0 is not empty, it doesn't "trigger".


Have you tried configuring your specific parameter to have a different "null value" according to your requirements ?

Above works great, and i found out many ways to configure the xml output after diving deeper into the configuration possibilities of this component.


I do have another question about XmlRecords however:

In the RecordToXml function i want to map to the following structure

For this i encapsuled the xsd in a wsdl and got a structure generated by Outsystems like this:

This behaves a bit strange.. Because it is a sequence of Strings it creates a separate structure Text for this with an attribute Value of type Text, while i can rename Text to LineType i can not rename Value because that results in an error serverside (in the outsystems IDE it looks valid, bug reported just now)

So lets leave the names as they are and try to fix it with Attribute configs.

The thing is no matter what i try i cant seem to get rid of the Value element (that outsystems added, which will makes the resulting xml invalid and not according to the xsd)

      <Sections>
        <Sender>
          <LinesType>
            <Value>-----</Value>
          </LinesType>
          <LinesType>
            <Value>----</Value>
          </LinesType>
          <LinesType>
            <Value>----</Value>
          </LinesType>
          <LinesType>
            <Value>----</Value>
          </LinesType>
        </Sender>
        <Receiver>
          <LinesType>
            <Value>-----</Value>
          </LinesType>
          <LinesType>
            <Value>-----</Value>
          </LinesType>
          <LinesType>
            <Value>------</Value>
          </LinesType>
          <LinesType>
            <Value>-----</Value>
          </LinesType>
        </Receiver>

Any suggestions how to do this?

I'm expecting this structure:

      <Sections>
        <Sender>
          <Line>-----</Line>
          <Line>-----</Line>
          <Line>-----</Line>
        </Sender>
        <Receiver>
          <Line>-----</Line>
          <Line>-----</Line>
          <Line>-----</Line>
        </Receiver>


Hello Daniel,

I already compiled a step-by-step example of how to build the structures required.

Please check my reply on this post to get a general idea of how you should approach creating the required structures for use with XML Records. (Hint: the ones created by SOAP are not adequate).

Ricardo Silva wrote:

Hello Daniel,

I already compiled a step-by-step example of how to build the structures required.

Please check my reply on this post to get a general idea of how you should approach creating the required structures for use with XML Records. (Hint: the ones created by SOAP are not adequate).

Hi Ricardo,

Thanks for your reply!

We have "living" XSD's in our application landscape with which our Outsystems apps have to communicate using XML and our Enterprise Service Bus, and i want to be able to refresh the structures that generate these XML that validates against these schemas without having to do too much effort on the Outsystems side each time the XSD changes (low-code platform means something right?)

For this i create a dummy WSDL that wraps the XSD (programmatically) which generates the structures automatic. (because sadly outsystems does not support XSD's directly)

This means i am not able to generate the structures myself; actually i dont want to, i want them to be based on the xsd which is often changing it's structure. You say "the ones created by SOAP are not adequate" but can we not alter the forge component to make adequate xml that validates against the xsd in the wsdl?

Besides some strange behaviour, for example record and attribute config not working in some cases but working in other very similar cases, this approach -almost- works; i have to do a couple of post Record-To-XML regex-es to make the XML valid, 

Now I am wondering why your forge component does not work in some situations but does work in other situations, however i am convinced it would not take too much work to adapt/expand it so that we are able to convert xsd/wsdl generated structures directly back to xml which also validates against that same xsd.

The strange behaviour often occurs when the xsd's are strongly types, my motto is that the xsd should be as flat as possible as soon as there is much nesting going on -> create root types.

I would love for the forge component to play nice with this approach to make the XML integrations with other systems more manageble and less work-intensive.

Naturally it's more important that version 11 of the Outsystems platform finally decently supports the XML and JSON world including schemas and handy mapping tools, but as that is the most optimistic approach since 10 does not have it, it still very far away and we have to make do with this component untill then. (parsing XML yourself is ofcourse possible but in an Enterprise not desirable because the resulting code is very intensive to maintain)



I fully agree with Daniël.

From a customer perspective both SOAP and JSON are a basic requirement to make Outsystems interface properly.
These requirements should not only be ''adopted'' by the Outsystems Platform but also supported within the platform.
Since it hits the core of the platform from my perspective it should be either embedded IN the Outsystems platform as a "SUPPORTED EXTENSION" or find it's place at the same "level" as the SILK component has.
It's Essential.
Maybe that way you guys have the flexibility to amend / patch the extension when needed.

We all know Gonçalo Borrega created the XML extension and I think he did a pretty good job on it.
I question myself why -components created by Outsystems personnell- remain as something "External" and Outsystems doesn't take formal ownership of it as a company. Most of them have proven themselves as VITAL for productional use of Outsystems.

Saying the platform supports Webservices -Out of the Box- is too big a name for the current level of implementation if we need to consume a formally Unsupported Extension and things like X-Path isn't in that "Box".

It's like Volkswagen saying their car can carry a cow while they know a trailer would be needed in order to fullfill the task of actually transporting a cow without severly damaging the car. :)

Support on extensions is not guaranteed and als not part of the support contract we have and the level of Webservice support "built-in" is in short very limited.

I'd like to see some change in this and maybe even a "vison" presented to the Outsystems' customer base on how Outsystems will approach what I call "module mushrooming / wildgrow" ; especially for modules that are "expected to be part of" or even "presented as part of" the platform.

Daniel de Witte wrote:

Hi Ricardo,

Thanks for your reply!

We have "living" XSD's in our application landscape with which our Outsystems apps have to communicate using XML and our Enterprise Service Bus, and i want to be able to refresh the structures that generate these XML that validates against these schemas without having to do too much effort on the Outsystems side each time the XSD changes (low-code platform means something right?)

...

Eric Oud Ammerveld (PS9.1) wrote:

I fully agree with Daniël.

From a customer perspective both SOAP and JSON are a basic requirement to make Outsystems interface properly.
These requirements should not only be ''adopted'' by the Outsystems Platform but also supported within the platform.
Since it hits the core of the platform from my perspective it should be either embedded IN the Outsystems platform as a "SUPPORTED EXTENSION" or find it's place at the same "level" as the SILK component has.
It's Essential.
Maybe that way you guys have the flexibility to amend / patch the extension when needed.

We all know Gonçalo Borrega created the XML extension and I think he did a pretty good job on it.
I question myself why -components created by Outsystems personnell- remain as something "External" and Outsystems doesn't take formal ownership of it as a company. Most of them have proven themselves as VITAL for productional use of Outsystems.

Saying the platform supports Webservices -Out of the Box- is too big a name for the current level of implementation if we need to consume a formally Unsupported Extension and things like X-Path isn't in that "Box".

It's like Volkswagen saying their car can carry a cow while they know a trailer would be needed in order to fullfill the task of actually transporting a cow without severly damaging the car. :)

Support on extensions is not guaranteed and als not part of the support contract we have and the level of Webservice support "built-in" is in short very limited.

I'd like to see some change in this and maybe even a "vison" presented to the Outsystems' customer base on how Outsystems will approach what I call "module mushrooming / wildgrow" ; especially for modules that are "expected to be part of" or even "presented as part of" the platform.

Daniel de Witte wrote:

Hi Ricardo,

Thanks for your reply!

We have "living" XSD's in our application landscape with which our Outsystems apps have to communicate using XML and our Enterprise Service Bus, and i want to be able to refresh the structures that generate these XML that validates against these schemas without having to do too much effort on the Outsystems side each time the XSD changes (low-code platform means something right?)

...

Thanks for your support Eric!

The only thing i must add to your Volkswagen analogy is that in this case it must either be a Volkswagen without a towbar, a partially broken towbar, or a towbar with a really strange shape.


I think you guys are confusing some stuff. And before I continue, please note that the rest of this post is my personal opinion on the matter, it is not meant to be an official OutSystems positioning.

I fully agree with Eric when he says that SOAP and (maybe) REST with JSON are essential for the OutSystems Platform to be able to interface with other systems.

As such, having these capabilities built into the platform offers a great advantage for developers attempting to leverage the OutSystems Platform in their landscape (and also for OutSystems to be able to get into those customers).

However, even if XML is the basis of SOAP, this does not necessarily extend to XML. While I recognize it would remove a lot of pain from developing in the situations where it's needed, I do not believe that XML manipulation belongs as a built-in capability of the OutSystems Platform. This includes X-Path, Serialization, XSD, etc.

I think that enterprises should preferably use SOAP and / or REST to expose their services, instead of relying on weakly typed stuff like XML contracts.

As such I am perfectly ok with this being a community driven effort instead of an OutSystems driven effort. True that the current implementation was done by an OutSystems employee (a long long time ago), but the community should be able to produce this component (or another like it).

Both XML Records and ardoJSON heavily rely on how the code was generated, but (at least ardoJSON) there's nothing done here that someone with proper .NET coding skills couldn't come up with. What I did to generate the bindings for ardoJSON was to reverse engineer how the OutSystems Platform is generating structures and use reflection to access and fill up this data.

What I would really like to see is the OutSystems Platform provide mechanisms to make this kind of approach more ... approachable. If there was a proper API to introspect structures within extensions and a way for me to generate a structure in Service Studio, I (or anyone in the community) could make much more out of XML Records and ardoJSON, without a need for OutSystems to intervene.

So, instead of asking OutSystems to provide you with the tools to address your specific concerns, I think that the community should be asking OutSystems to provide the APIs for the community to be able to build upon the Platform and come up with the weird niche solutions for ourselves.

Ricardo Silva wrote:

I think you guys are confusing some stuff. And before I continue, please note that the rest of this post is my personal opinion on the matter, it is not meant to be an official OutSystems positioning.

I fully agree with Eric when he says that SOAP and (maybe) REST with JSON are essential for the OutSystems Platform to be able to interface with other systems.

As such, having these capabilities built into the platform offers a great advantage for developers attempting to leverage the OutSystems Platform in their landscape (and also for OutSystems to be able to get into those customers).

However, even if XML is the basis of SOAP, this does not necessarily extend to XML. While I recognize it would remove a lot of pain from developing in the situations where it's needed, I do not believe that XML manipulation belongs as a built-in capability of the OutSystems Platform. This includes X-Path, Serialization, XSD, etc.

I think that enterprises should preferably use SOAP and / or REST to expose their services, instead of relying on weakly typed stuff like XML contracts.

As such I am perfectly ok with this being a community driven effort instead of an OutSystems driven effort. True that the current implementation was done by an OutSystems employee (a long long time ago), but the community should be able to produce this component (or another like it).

Both XML Records and ardoJSON heavily rely on how the code was generated, but (at least ardoJSON) there's nothing done here that someone with proper .NET coding skills couldn't come up with. What I did to generate the bindings for ardoJSON was to reverse engineer how the OutSystems Platform is generating structures and use reflection to access and fill up this data.

What I would really like to see is the OutSystems Platform provide mechanisms to make this kind of approach more ... approachable. If there was a proper API to introspect structures within extensions and a way for me to generate a structure in Service Studio, I (or anyone in the community) could make much more out of XML Records and ardoJSON, without a need for OutSystems to intervene.

So, instead of asking OutSystems to provide you with the tools to address your specific concerns, I think that the community should be asking OutSystems to provide the APIs for the community to be able to build upon the Platform and come up with the weird niche solutions for ourselves.

I agree with you for the most part, however generating XML based on XSD schema (encapsulated in a WSDL or not) is not a niche solution.

In fact its what's holding back the Outsystems platform from being incorporated at a bigger rate in large and medium sized enterprise application landscapes.

Looking at the pricing of Outsystems, it aims at these large and medium enterprises as customers, who rely on Outsystems to be a low-code platform, and would be delighted not to have the need to have their developers build custom .net code.

Because large enterprises, without exception, have an application landscape with many applications that need to be integrated, Outsystems should consider it self-interest to have their platform integrate properly


Having an incomplete set of tooling for the interface part (agreeing with Daniel here) is definately NOT niche nor something someone would like to "re-invent"; we're not tampering with the W3C agreements either; this has all been worked out and proper agreements and standards exist.
Functionalities are partly in place but the full standard currently does not fully apply to the base of the platform.

This isn't bad; it just could be better and with that would definately contribute to the level of Enterprise adoptation.
Some time ago you implemented an SAP integration; this is even more specific than what we are asking for.

Enterprises have to deal with Legacy and will want to move away from that in a proper manner.
These days providing (micro)webservices is a good way.
Interfacing isn't the key selling point of Outsystems but it is for Enterprise tools like Tibco BW/BC.
Considering these kind of tools as "leading" within the market segment would be the "humble" vision a lot of major Enterprises would embrace.

We're not requesting you to mimic their methods; we want you be able to fully talk to their platform according to the rules and standards that apply to the existing technology.