For OutSystems web (Reactive) applications, there is no OutSystems-native, fully license-free tool that covers end-to-end UI automation out of the box. Automation typically relies on external tools, and its success largely depends on how well these tools handle dynamic UIs.

Common and proven options include:

1. Playwright (open source)

• Strong support for modern, dynamic web applications

• More reliable than Selenium for async rendering and SPA behaviour

• Works well when stable selectors (e.g. data-test attributes) are used

2. Selenium (open source)

• Widely adopted and license-free

• Requires a careful selector strategy due to dynamic DOM updates in OutSystems

• Higher maintenance effort compared to Playwright

3. Cypress (free tier available)

• Developer-friendly and fast feedback

• Suitable for E2E and component testing

• Limitations in some cross-domain or advanced scenarios

4. JOSF – Java Open-Source Framework (OutSystems Community)

• Community-driven framework specifically designed for OutSystems UI automation

• Built on top of Selenium, with abstractions tailored for OutSystems patterns

• Helps reduce selector brittleness by encapsulating common UI interactions

• Requires Java knowledge and some setup effort, but can speed up test creation for teams familiar with OutSystems UI structures

From an OutSystems best-practice perspective:

• Use stable, explicit selectors instead of autogenerated IDs

• Combine UI automation with API-level tests for better coverage and lower maintenance

• Keep UI tests focused on critical user journeys rather than exhaustive validation

Abhijeet, let me sum up. Playwright and Selenium are the most common license-free choices, while JOSF (I am using this one now) can be a good option if you want a framework tailored to OutSystems UI patterns and are comfortable with Java-based tooling.

Also, for server-side testing, it often helps to create a separate test module where you centralize the actions you want to validate (business logic, validations, calculations, integrations wrappers, etc.).

In that module, you can define multiple test cases per action by:

• Preparing representative input sets (including edge cases)

• Executing the action with those inputs

• Asserting the expected outputs (or expected error/validation behavior)

This approach reduces reliance on UI automation, keeps tests more stable, and makes it easier to validate the core logic independently from the front-end. - For server-side testing, it will be necessary to have someone with some knowledge of the platform.

2 notes from my experience:

In my experience, when working on complex and long-term media projects, the creation of automated tests should begin right from the start of the project, unless you can get additional support (extra resources) from the team to leverage the testing of the code already developed.

I can tell you that having automated tests allowed us to deliver projects with a high level of quality, and the number of bugs reported by the client was very low.

Thank you for your insights !!