[TestFramework] Running UTF tests

Forge Component
(43)
Published on 18 Sep (8 days ago) by Indigo
43 votes
Published on 18 Sep (8 days ago) by Indigo

Hi,

I was able to import UTF tests to TestFramework, but when executing a test, it seems that the test SOAP service gets a 0 TestRunUnitTestId which breaks UTF functionality. 

For example, recording asserts and comments requires a non-zero TestRunUnitTestId. Here's an example error message:

System.Data.SqlClient.SqlException: The INSERT statement conflicted with the FOREIGN KEY constraint "OSFRK_OSUSR_POP_TESTRUN_UNITTEST_LOG_OSUSR_POP_TESTRUN_UNITTEST_TESTRUNUNITTESTID". The conflict occurred in database "TAX_OutSystems", table "dbo.OSUSR_POP_TESTRUN_UNITTEST", column 'ID'.

How is the integration with UTF suppose to work?

Br,

Toni

Hello Toni,


After importing the UTF Tests the TestFramework handle those tests as regular SOAPs. What I mean is that the UTF aren't more than regular SOAP calls, with a regular configuration, and what we do is to bootstrap those tests as SOAP API tests.


Can you please share some more detail? 

For example, the configurations on the UI (testcases, teststeps?), what variables are defined and so on.


On top of that, can you please try a SOAP API call, and check if everything runs smooth?


Thank you

Hi Davide,

Here's a screenshot of a unit test method template (see UTF_Template eSpace in the UTF application available in Forge). Notice how the method takes a TestRunUnitTestId in. Are you familiar with this pattern?


If there's a seamless integration between TestFramework and UTF, then I would expect an UTF test run to be created before calling the UTF test. Right?

Br,

Toni


Toni,


We are not creating a test run on UTF, that is the whole point there. TestFramework will execute and keep track of the executions, same as the results. If you want to keep that input variable you will be needing to configure it in TestFramework.


Nevertheless, we are planing on reviewing the integration with the UTF Framework in a near future.

Hi,

Ok if Test Framework is not creating a UTF test run (which makes sense, since it's UTF internal thing), how can Test Framework execute a UTF test? A UTF test requires TestRunUnitTestId (see screenshot above).

Toni 

Hi,


If you don't want to clean up that variable you need to configure it on the Test Framework with a Dummy value, that allows the execution of the test.

Hi,

That's a mandatory parameter for UTF, so I cannot clean it up. Could you give an example of a Dummy value that would allow the execution of the example test in UTF_Template?

Toni

Toni Juvani wrote:

Hi,

That's a mandatory parameter for UTF, so I cannot clean it up. Could you give an example of a Dummy value that would allow the execution of the example test in UTF_Template?

Toni

Hi Toni,

I got the same problem did you receive any solution on this?


Regards,


Romano

I'm also struggling to get UTF tests to run. 

  • I manage to get them to load in TestFramework.
  • Then I added a variable TestRunUnitTestId with value = 1... just to try it out on one test.
  • After running the test I get a message in the Log:

(System.Collections.Generic.KeyNotFoundException mscorlib)The given key was not present in the dictionary.

   at System.ThrowHelper.ThrowKeyNotFoundException()
   at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
   at OutSystems.NssTestFramework_SOAP_lib.CssTestFramework_SOAP_lib.MssSOAP_InvokeMethodManyResults(Object ssInvocationObject, String ssMethod, RLParameterRecordList ssInput, Boolean& ssSuccess, RLParameterRecordList& ssResult, String& ssErrorMessage)

ConstantsAreCorrect
     TestRunUnitTestId (Int): 1


Is that TestRunUnitTestId suppose to correctly correspond to the ID in UTF_Core.TestRun_UnitTest? Not sure how we'll hardcode that value in TestFramework. 

Hi,


I'm facing the same issue with the "seamless Unit Test Framework integration".

After some digging in the code, it seems that there is one Module missing that allows a proper integration with the Unit Test Framework. This missing Module goes by the name of UTF_API.

It's something that is really missing in the oap or is something expect for everyone to build their own?


Thanks.

Paulo Portela 

Paulo Portela wrote:

Hi,


I'm facing the same issue with the "seamless Unit Test Framework integration".

After some digging in the code, it seems that there is one Module missing that allows a proper integration with the Unit Test Framework. This missing Module goes by the name of UTF_API.

It's something that is really missing in the oap or is something expect for everyone to build their own?


Thanks.

Paulo Portela 


Hi Paulo,

That OAP is indeed missing and there's some other issues as well with that version.

For that reason, we're currently working full-time on a new version of the TestFramework that will be released in the very near future.

In the meantime I'll provide you the missing OAP.

If you need any further help, don't hesitate to contact us!

Best regards,
Rui Miguel Gamas

Hi Rui,


Thank you for the new OAP. However, the new UTF_Core integrates with a new Module (EspacesInformation_CS) that is missing.

Do you have an idea of when the new version of the TestFramework will be available?

Would it be possible to have a how-to integrate the Unit Testing Framework with the TestFramework?

Thanks.


Best Regards

Paulo Portela

Hey Paulo,

We're working to release it (early) this coming week.

With the new version you won't have to worry about integrating it with UTF. It'll be easier, but a how-to will be provided in the future with a troubleshooting 'manual' (currently ongoing).

Regarding the new version, I can PM you personally when it's ready, if you're okay with it.

Again, I'm really sorry about the inconvenience. We're aware of the problems with the current version in forge and that's why we're working hard to improve and release a new version.


Kind regards,

Rui Miguel Gamas

Hi Rui,


Yes, please send me a PM when it's ready. For some reason I'm not being notified when someone replies to my messages.

Thanks for your super fast replies. 


Thanks.

Paulo Portela

Rui Miguel Gamas wrote:

Hey Paulo,

We're working to release it (early) this coming week.

With the new version you won't have to worry about integrating it with UTF. It'll be easier, but a how-to will be provided in the future with a troubleshooting 'manual' (currently ongoing).

Regarding the new version, I can PM you personally when it's ready, if you're okay with it.

Again, I'm really sorry about the inconvenience. We're aware of the problems with the current version in forge and that's why we're working hard to improve and release a new version.


Kind regards,

Rui Miguel Gamas

 Can I also receive this PM when the new version is ready? I was also working to implement this today, and ran into the issue.

Thanks,

Brian Schwantner

 

Brian Schwantner wrote:

Rui Miguel Gamas wrote:

Hey Paulo,

We're working to release it (early) this coming week.

With the new version you won't have to worry about integrating it with UTF. It'll be easier, but a how-to will be provided in the future with a troubleshooting 'manual' (currently ongoing).

Regarding the new version, I can PM you personally when it's ready, if you're okay with it.

Again, I'm really sorry about the inconvenience. We're aware of the problems with the current version in forge and that's why we're working hard to improve and release a new version.


Kind regards,

Rui Miguel Gamas

 Can I also receive this PM when the new version is ready? I was also working to implement this today, and ran into the issue.

Thanks,

Brian Schwantner

 

 

 Hi Brian,

No worries, I'll ping you!


In case others want to be notified by PM, please comment here!

I'll gladly PM you too ;)


Thanks!

Rui Miguel Gamas