BinaryToBase64(TextToBinaryData(MD5)) != CryptoApi.ComputeHash.Hash_base64

In my quest to discovering why a webservice is shooting down my authentications I've asked the following question and have gotten my answer. however i noticed something interesting I wanted to share.

At first I used 

GenerateSaltedMD5Hash(Content)

and then something i learned from https://www.outsystems.com/forums/discussion/40142/converting-text-to-base64-encoding/

BinaryToBase64(TextToBinaryData(MD5))

However I needed unsalted MD5 so i used 

https://www.outsystems.com/forums/discussion/44124/unsalted-md5-encryption-is-there-a-way-i-can-still-use-it/#Post159580

ComputeHash.Hash_Hex (Content , "MD5")

This Function Hash_hex has two text outputs one Hash_Base64 and Hash_hex.

The thing i expected was that when I would use Hash_hex with BinaryToBase64(TextToBinaryData(Hash_hex)) it would have the same outcome as Hash_base64. However this is not the case....Probably because to_binary changes the value.

So actually this is now a new question how to properly Encode text to base64?

Hi,


It should be sufficient to do 

BinaryToBase64(TextToBinaryData("your text","utf-8"))

please note, that default text will be converted to binarydata with ascii-encoding!

therefore, you should use utf-8 (utf-16 if you want)


Robert Hantink wrote:

So actually this is now a new question how to properly Encode text to base64?

Hi Robert Hantink ,

BinaryData Module contains BinaryToBase64,

Thus to convert Text to Base64 we need to convert the Text -> BinaryData  using TextToBinaryData

So you'll get something like:

BinaryToBase64(TextToBinaryData(MyText))


Hope this help,

Assif


Hi Robert,

I looked into CryptoAPI code and I don't think those 2 outputs will be the same even if you do all the conversions. But why this is done like this you will need to ask to CryptoAPI team.

Regards,

Marcelo

By doing so i still don't understand why ComputeHash comes with "CY9rzUYh03PK3k6DJie09g==" as base64 when I use "test" as value.

So I use computeHash( "test", "MD5") and get back 

Hash_base64 = "CY9rzUYh03PK3k6DJie09g=="
Hash_hex      = "098F6BCD4621D373CADE4E832627B4F6"

When I encode use 

BinaryToBase64(TextToBinaryData("098F6BCD4621D373CADE4E832627B4F6","utf-8"))

I get "MDk4RjZCQ0Q0NjIxRDM3M0NBREU0RTgzMjYyN0I0RjY="

Even if i use all the other encode types like Unicode, UTF16 and ASCII I get:

Unicode : MAA5ADgARgA2AEIAQwBEADQANgAyADEARAAzADcAMwBDAEEARABFADQARQA4ADMAMgA2ADIANwBCADQARgA2AA==

UTF 16: MAA5ADgARgA2AEIAQwBEADQANgAyADEARAAzADcAMwBDAEEARABFADQARQA4ADMAMgA2ADIANwBCADQARgA2AA==

ASCII: MDk4RjZCQ0Q0NjIxRDM3M0NBREU0RTgzMjYyN0I0RjY=


So the one from computeHash is probrably another format? I don't understand it anymore.

I've made a test page: https://pincvision-dev1.outsystemsenterprise.com/Payment_IS/Entry1.aspx?_ts=636831451469132701



Marcelo Ferreira wrote:

Hi Robert,

I looked into CryptoAPI code and I don't think those 2 outputs will be the same even if you do all the conversions. But why this is done like this you will need to ask to CryptoAPI team.

Regards,

Marcelo

Yes so I think the CryptoAPI Base64 is of a different kind and thus wrong. I just hope the Outystems one is 100% right.


Hi Robert,

Don't think any value is wrong they are only compute differently. For the calculation c# libraries are used so I don't think they are wrong.

Regards,

Marcelo

Well, if you check with an online decoder (I used base64decode.org), then MDk4RjZCQ0Q0NjIxRDM3M0NBREU0RTgzMjYyN0I0RjY= is converted back to 098F6BCD4621D373CADE4E832627B4F6, so the Outsystems one seem to be correct. 

Please note that you are comparing different things. In one case you are calculating a Base64 *hash* of the MD5 encrypted text "test". In the other case you are converting the *text* "098F6BCD4621D373CADE4E832627B4F6" to Base64. If you would like to compare it, you should convert "test": 

BinaryToBase64(TextToBinaryData("test","utf-8"))

Which gives dGVzdA==

Lennart Kraak wrote:

Well, if you check with an online decoder (I used base64decode.org), then MDk4RjZCQ0Q0NjIxRDM3M0NBREU0RTgzMjYyN0I0RjY= is converted back to 098F6BCD4621D373CADE4E832627B4F6, so the Outsystems one seem to be correct. 

Please note that you are comparing different things. In one case you are calculating a Base64 *hash* of the MD5 encrypted text "test". In the other case you are converting the *text* "098F6BCD4621D373CADE4E832627B4F6" to Base64. If you would like to compare it, you should convert "test": 

BinaryToBase64(TextToBinaryData("test","utf-8"))

Which gives dGVzdA==

Yes i know, that's why i made a test page :)


What I meant to say was that a hashcode is something else than a conversion. Nice test page though :-) 

Solution

Robert Hantink wrote:

Yes i know, that's why i made a test page :)

Though you claim you "know", I actually think, based on what I read above, that you're not quite sure what you're doing. "098F6BCD4621D373CADE4E832627B4F6" is the hexadecimal representation of a sequence of bytes, with values 09h, 8Fh, 6Bh etc. If you create a binary file with those byte values, and base64-encode it, you actually do get the "CY9rzUYh03PK3k6DJie09g==" value (I just tried with the attached file, which I created using the hex values).

However, you tried to encode the hexadecimal text representation, and of course that yields something completely different. So your initial expectation (in your first post that) "when I would use Hash_hex with BinaryToBase64(TextToBinaryData(Hash_hex)) it would have the same outcome as Hash_base64" is just plain wrong, based on false assumptions. It even says "hex", not binary!

So to sum up, the CryptoAPI is fine, you just have to understand what its output is.

Solution

Kilian Hekhuis wrote:

Robert Hantink wrote:

Yes i know, that's why i made a test page :)

Though you claim you "know", I actually think, based on what I read above, that you're not quite sure what you're doing. "098F6BCD4621D373CADE4E832627B4F6" is the hexadecimal representation of a sequence of bytes, with values 09h, 8Fh, 6Bh etc. If you create a binary file with those byte values, and base64-encode it, you actually do get the "CY9rzUYh03PK3k6DJie09g==" value (I just tried with the attached file, which I created using the hex values).

However, you tried to encode the hexadecimal text representation, and of course that yields something completely different. So your initial expectation (in your first post that) "when I would use Hash_hex with BinaryToBase64(TextToBinaryData(Hash_hex)) it would have the same outcome as Hash_base64" is just plain wrong, based on false assumptions. It even says "hex", not binary!

So to sum up, the CryptoAPI is fine, you just have to understand what its output is.

Aah ok. I am sorry stupid me. I get it now what is meant. So creating a Base64 from compute.hash_hex in this way 

BinaryToBase64(TextToBinaryData(ComputeHash.Hash_Hex, "utf-8"))

is wrong because you first have to create a binary from that ComputeHash.Hash_Hex. So it is save to use the hash_base64 directly.

I still would like to know how to convert Hex to binary in OutSystems? I would like to compare myself so I understand it better.

Thank you for clarifying that for me. I've been having a blind spot for hex or something. 

 

Hi Robert,

I don't think there's an easy way to convert hex to binary, but there are online converters like this one that allow you to do so. Could you mark my answer above as Solution, if your initial question has been answered? Thanks.