Log in to AnswerLog in to Follow
2021-09-09 10-04-43
Eemi Närhi
46Views
3Comments
[TestFramework] [TestFramework] Azure DevOps integration with TestFramework
Question
Forge
Testing
testframework
Web icon
TestFramework
Forge component by Indigo

Hi,

Has anyone been able to integrate TestFramework with Azure Pipelines in Azure DevOps? I'm building an OutSystems pipeline with Azure DevOps which is going to include running BDD Framework tests.

I recently started using TestFramework and I added some tests into TestFramework that I created with BDD Framework.

Now I'm wondering how to trigger those BDD tests in TestFramework as part of a pipeline?

It is mentioned on this guide, that TestFramework provides a REST API for triggering Test Framework tests as part of a Continuous Integration pipeline. The API allows tests to be started and the results to be collected. | https://success.outsystems.com/Documentation/How-to_Guides/DevOps/How_to_Automate_Unit_Testing_and_API_Testing#Test_Management

How can I access this specific API and use it to trigger my tests?

I'm aware that BDD Framework tests can be triggered in a pipeline with the BDDTestRunner API, but I'm interested to hear if anyone has experience integrating TestFramework with Azure pipeline?

0
0
21 Apr 2021

Hi @Eemi Närhi ,


By curiosity, why do you to trigger the Test runs from Test Framework and not from BDD API it self? 

What are the advantages you see on using Test Framework in a Azure DevOps pipeline?

And what are the disadvantages you see on using BDD API in a Azure DevOps pipeline?


Thanks in advance.

Urbano


0
0
21 Apr 2021

Hi @Urbano Freitas ,

thanks for the great questions. 

What caught my attention in the TestFramework was the dashboard-style Overview page and the graphs/charts for the history of test runs. It looks promising from test management perspective. If I recall correct the BDD Framework however doesn't provide such built-in visual overview of test run history. What I'm thinking is, if the Azure build pipeline could run those BDD tests in TestFramework and the development team could see the fresh test results and graphs on the TestFramework UI, that would be pretty cool.

Currently my plan is actually to trigger the test runs from BDD API in my Azure DevOps pipeline since I don't know how to use TestFramework properly in the Azure DevOps pipeline. So using the BDD API to trigger BDD tests seems like an optimal solution.

I'm still interested to hear if anyone has used Test Framework in an Azure DevOps pipeline.

Eemi

0
0
22 Apr 2021

@Eemi Närhi 

Thanks for the sharing these insights regarding why you want to use Test Framework.

I'm not aware of any customer using Test Framework in integration or Continuous Integration (CI) scenarios.


Finally, let me share that both Jenkins and Azure DevOps, provide ways of having of visualize test executions results.


See some examples of these charts on our public resources:

https://github.com/OutSystems/outsystems-pipeline/wiki/Building-an-OutSystems-pipeline-with-Azure-DevOps


I hope this helps.

Urbano

0
0
22 Apr 2021
Community GuidelinesBe kind and respectful, give credit to the original source of content, and search for duplicates before posting.