Built-in Unit testing capabilities
3516
Views
25
Comments
On our RadarOn our Radar
Backend

Service Studio should have new, separate "Tests" tab.

This would enable integrating tests to any eSpace and would minimize required overhead for testing any action, including private actions.

Why?

  1. Testing your code is more professional than not testing.
  2. Unit testing should be minimal extra effort for developer, because otherwise tests are usually not done.
  3. Publishing new code could be controlled - you cannot publish untested code, or code which has some failing tests, for example.
  4. Test code should never go to Production environment.
  5. Testing is good.

How?

  1. Tests are integrated to eSpace/module oml file.
  2. During code generation process, actions from "Tests" tab would create a separate .NET project and this project would be a friend assembly (internalsVisibleTo attributes configured) for main project to compile.
  3. 1-Click Publish deployments with tests could start from (currently underused?) personal area (or another IIS folder), where tests would be run.
  4. Depending on how tests go and how is decided in additional configuration, deployment would continue to Public area or show an error in Service studio.

This way, creating unit tests would be as effortless as when creating unit tests in any other modern programming language - just few clicks away. There could be also accelerators to generate unit test actions or "system events" to initialize/teardown test runs.

Currently, writing any tests to private actions is also either impossible or exposing some unwanted/test related code to production environments.

It's a big change, but from discussions with makers/experts @ ODC I've learned this should not be too far-fetched idea to implement. Maybe for P12, please?

Another, possibly the simplest way to enable testing / production code separation without massive modifications to existing compiling process could be a capability to define another espace as "friend" espace. This per-espace setting could reveal internal (=OS private) actions to the another espace marked as a friend and thus enable referencing these otherwise out-of-the scope actions for testing purposes. 

If done like this, feature could be achieved using small amounts of .NET reflection trickery or previously suggested InternalsVisibleTo attribute (that has been around since 2002 release of .NET 2.0). 

Security-wise, there should be no problems either, (albeit a bit more work during compilation) as friend assemblies can be defined as signed assemblies, too.

Personal opinion, but this topic is really heavily connected with OS small book #2, #5 and #7.

Mikko,

You have my vote, and thumbs up for the detailed description and high fidelity mock-up!

Regards,

Daniel

2014-02-07 17-02-43
Vasco Pessanha
Changed the category to
Backend
2014-02-07 17-02-43
Vasco Pessanha
Changed the status to
On our RadarOn our radar

Hey Mikko,

Thanks for your idea!

We are currently looking into unit testing capabilities.

Stay tuned!

This really would help us.

2023-02-10 19-42-59
João Melo
 
MVP

Really good idea and very well defined!

2018-06-02 00-30-13
Carina
Champion

Ofcourse we have a few testing forge components in Outsystems, but a full integration of some features of this into service studio would help greatly in making test-based development the norm - something that in my opinion has been lacking in Outsystems development.


Thumbs up for the idea, thanks

Hi Mikko,

You point at the best location for (unit) testing to include.

This way the OutSystems environment could grow to even have a percentage covered before going to production in the future. 

But it should always be optional, because if you want to do a quick demo you don't want to spoil time into the unit testing. (You should mention the possibility and the need to test to your audience...)

I vote for P12.

Congrats Mikko for this very detailed idea!

I will follow this up and hope it can be done at least in P12.

I would be very disappointed if OutSystems doesn't present some nice improvements regarding unit testing/test automation on Next Step. The community has made it clear that these are top priority featueres that we need.

This is very well needed features for enterprise applications. 

+1

Being able to write ServiceStudio native tests will motivate developers, ensure project success, and increase OutSystems sales.

It's important also to trigger those unit tests from a CI/CD pipeline, probably enhancing CI/CD Probe that is being used to discover BDD Framework tests (https://www.outsystems.com/forge/component-overview/6528/ci-cd-probe) ou using another approach. 

OutSystems market itself as fast and agile way to develop applications, but it seems by adding features to the product is really slow? Will OS 12 have this? or not?

Awesome Idea Mikko Nieminen!

I am really excited to see this feature in Service studio.

This was posted on 2018. I know there are already Unit testing forge component available, but this feature will be the great choice to implement the main testing component in service studio. 

Up for this.

+1

People stop add +1 as a comment, yes press the like button if you like it.

+1 comments get removed.


Nice idea Mikko Nieminen

Yes indeed we require unit testing to be developer friendly and it will encourage developers to perform unit test more often and thereby avoiding deployment of untested code.

But as Bert Leibbrand  said it should be optional

My suggestion would be rather than having it as a tab, how about having a separate type of module for testing. We can have a module which will have a type which is testing friendly and can be included as per need on application level.

Nice idea , any update about it ?

Very good idea!

For anyone interested, have a look at Testing Framework: https://www.outsystems.com/forge/component-overview/12754/testing-framework (full disclosure: I'm the author).

The forge component allows to easily test server-side actions, including private actions. It is more efficient than BDD Framework at writing unit tests and executing them.

Integration with CI/CD, and client-side tests, are both underway.

Merged this idea with 'Enhancing OutSystems Development with Built-In Unit Testing' (created on 15 Jan 2024 13:33:02 by Jonathan Bryden)

Currently there are forge components such as BDDFramework which allows unit tests to be setup and executed. As a developer this is great but requires jumping to the test module(s) each time a change is made to refresh dependencies, publish the test module and heading over to the page to test (Or using TestFramework to automate the execution of tests).


What would really be a game changer is having unit testing built into the OutSystems platform out of the box.

Being able to create unit test scenarios for server and service actions and having them executed as part of the 1 click publish would drastically improve the quality and immediately form part module itself allowing continuous testing of the module being worked on as well as the dependent modules. 

At the end of the publish, it would be great to view a report generated indicating test scenarios executed for the various server and service actions.

It would also be great to have a dashboards in Lifetime covering:

  • Tests executed by application and module.
  • Successful and failed unit tests.
  • Metrics such as code coverage.

Something like this, perhaps? Check out this forge component, would be nice base https://www.outsystems.com/forge/component-overview/15288/k-test-core-o11

Awesome Idea

Hi Mikko,

While there are a few testing components available on the OutSystems Forge, a deeper and more seamless integration of testing capabilities directly within Service Studio would significantly encourage a test-driven development (TDD) approach. In my view, OutSystems development has historically lacked strong support for test-based practices, and enhancing native tooling could play a key role in shifting this paradigm.