How to Trigger TOSCA Tests Using Jenkins or Azure DevOps

Introduction

Imagine this: you just merged a feature into your main branch. You want to ensure nothing broke UI flows still work, database interactions function correctly, and critical business processes behave as expected. Now imagine this entire validation happens automatically within minutes without any human intervention. That is the power of continuous integration (CI) and continuous testing. Using Tricentis Tosca (Tosca) for your automation, you can integrate it seamlessly into your CI pipeline.

In this post, we walk through how to trigger Tosca tests from two popular CI/CD systems: Jenkins and Azure DevOps. You will find practical examples, configuration steps, sample pipeline scripts, and tips for real-world setups. Whether you hold a Tosca Training License or are exploring Tosca automation for the first time, this guide is designed for hands-on learners.

Why Integrate Tosca into CI/CD

Before diving into the “how,” let’s explore the “why.” Integrating Tosca into CI/CD delivers multiple benefits:

• Automated continuous testing: Every build, merge, or deployment can automatically trigger regression or smoke tests.
  
  

• Faster feedback loop: Developers quickly know if their changes caused failures, reducing bug-fix cycles.
  
  

• Reliable regression management: Automation reduces human error and ensures high test coverage over time.
  
  

• Scalability and consistency: Using Tosca Distributed Execution (DEX), tests run consistently across multiple machines.
  
  

Integrating Tosca into CI/CD transforms it from a standalone test tool into a critical part of your delivery pipeline.

Overview: Tosca CI/CD Integration Options

Tosca supports multiple modes of CI/CD integration. Choosing the right method depends on your environment and needs:

Integration Method	Description
Tosca Execution Client	Uses a command-line client to trigger test execution, compatible with Tosca Distributed Execution (DEX) and Agent Orchestrator Server (AOS).
Tosca Server Execution API	Uses REST API calls to trigger test execution remotely; requires Tosca Server setup and proper authentication.
Remote Service Execution	Trigger tests via a remote execution service endpoint. Useful for legacy setups or simpler network environments.

For new projects, the recommended approach is using either the Tosca Execution Client or Execution API, as they are flexible, maintainable, and can be fully scripted.

Part 1: Triggering Tosca Tests with Jenkins

Jenkins is widely used, highly customizable, and integrates well with Tosca through plugins and scripting.

1.1 Setup: Install Plugin and Configure Agent

1. On your Jenkins master or agent, install the Tricentis Continuous Integration plugin.
   
   

2. Ensure Tosca is installed on the same machine or an accessible agent.
   
   

3. Identify the path to the Tosca CI client — either ToscaCIClient.exe for Windows or ToscaCIJavaClient.jar for cross-platform execution.
   
   

4. If using the Java client, ensure Java Runtime Environment 1.7 or later is installed and the JAVA_HOME variable is configured properly.
   
   

1.2 Configure Jenkins Job — Freestyle or Pipeline

Freestyle Job

• Add a Tricentis Continuous Integration build step.
  
  

• Configure the required fields:
  
  

• tricentisClientPath: Path to Tosca CI client.
  
  

• endpoint: Service URL for remote execution.
  
  

• Use either testEvents or configurationFilePath to specify which tests to run.
  
  

Pipeline (Jenkinsfile)

You can embed Tosca execution inside a Jenkins pipeline using the following syntax:

pipeline {

    agent any

    stages {

        stage('Run Tosca Tests') {

            steps {

                tricentisCI tricentisClientPath: 'C:\\ToscaCI\\Client\\ToscaCIClient.exe', \

                           configurationFilePath: 'path/to/configuration.xml', \

                           endpoint: 'http://<TOSCA_SERVER>:8732/TOSCARemoteExecutionService/'

            }

        }

    }

}

This triggers Tosca tests as part of the pipeline flow.

1.3 Execution via Remote Service or Distributed Execution

• Remote Service: Start the ToscaCIRemoteExecutionService.exe and point Jenkins to the endpoint.
  
  

• Distributed Execution (DEX + AOS): Use the Tosca CI Client configured with the DEX server endpoint along with a configuration file.
  
  

1.4 Advanced: Environment Setup, Test Filtering, Reports

• Use a configuration XML file for complex setups, including multiple execution lists or tag-based filtering.
  
  

• For DEX, the testEvents parameter specifies which TestEvents to run.
  
  

• After execution, Jenkins can fetch test results and logs for easy viewing in the Jenkins UI. Automatic publishing of Tosca reports is supported.
  
  

1.5 Real-World Example

Imagine you have a web application with Tosca test cases for login, checkout, and regression. Your Jenkins pipeline can include:

• Build Stage: Compile and build the latest code.
  
  

• Deploy Stage: Deploy to the test environment.
  
  

• Tosca Test Stage: Trigger Tosca tests using the Jenkinsfile approach.
  
  

• Report Stage: Publish Tosca results and send notifications if failures occur.
  
  

This ensures every code change triggers a full regression suite, catching issues early.

Part 2: Triggering Tosca Tests with Azure DevOps

Azure DevOps (ADO) provides repositories, pipelines, and release management. Tosca can integrate seamlessly.

2.1 Tosca + Azure DevOps Integration

• Tosca’s CI Client can be called from the command line, making it ideal for pipeline tasks.
  
  

• For multi-user workspaces, the CI Client triggers execution of an ExecutionList. Configuration parameters like ContinuousIntegrationBuildRootFolder and ContinuousIntegration help manage execution in CI pipelines.
  
  

• The Azure DevOps pipeline runs a Command Line or PowerShell task pointing to the Tosca CI Client and workspace.
  
  

2.2 Sample Azure DevOps Pipeline (Classic)

Add two tasks in the pipeline: a command-line task to execute Tosca tests and a task to publish test results.

Example Classic Pipeline Steps:

• Command Line Task:
  
  

• Tool: ToscaCIClient.exe
  
  

• Arguments: --workspace "C:\ToscaWorkspaces\MyProject.tws" --executionList "Regression" --run
  
  

• Publish Test Results Task:
  
  

• Point to the folder containing Tosca-generated XML or JUnit-style results.
  
  

2.3 Sample Azure DevOps Pipeline (YAML)

trigger:

  branches:

    include:

      - main

pool:

  vmImage: 'windows-latest'

steps:

- script: |

    & "C:\ToscaCI\Client\ToscaCIClient.exe" --workspace "C:\ToscaWorkspaces\ProjectA.tws" --executionList "SmokeTests" --run

  displayName: 'Run Tosca Smoke Tests'

- task: PublishTestResults@2

  inputs:

    testResultsFiles: '**/ToscaResults/*.xml'

    testRunTitle: 'Tosca UI Test Results'

This YAML executes Tosca tests and publishes results in Azure DevOps for visibility.

2.4 Triggering Tosca on Build Completion

• Run Tosca tests whenever a build pipeline succeeds or on PR merge.
  
  

• Maintain two pipelines:
  
  

1. Build Pipeline: Compiles code and runs unit tests.
   
   

2. Tosca Test Pipeline: Executes Tosca tests after build completion.
   
   

This approach keeps builds fast and only runs heavier UI or regression tests as needed.

2.5 Integration with Tosca Server + DEX + AOS

• Configure ToscaCIClient.exe.config with server endpoint, client ID/secret, and workspace root.
  
  

• Azure DevOps agents can trigger execution on remote Tosca agents, enabling scalability and centralized test management.
  
  

Best Practices & Real-World Considerations

When integrating Tosca with Jenkins or Azure DevOps:

Use Clear Execution Lists & Environments

• Maintain separate execution lists for smoke, regression, and nightly full regression.
  
  

• Use Tosca Test Configuration Parameters (TCP) to flag which lists should run in CI, avoiding unintended runs.
  
  

Use DEX + AOS for Scalability

• Large test suites or parallel execution benefit from Tosca Distributed Execution with AOS, which decouples execution from the build server.
  
  

Maintain Tosca Workspaces Consistently

• Ensure the Tosca workspace is version-controlled or accessible in a shared repository.
  
  

• Configure CI clients to reference the correct workspace path.
  
  

Handle Test Reports & Artifact Publishing

• Capture Tosca output (XML, HTML reports, logs) for dashboards, visibility, and traceability.
  
  

Manage Licensing

• Ensure Tosca Training License or Production License covers all execution agents.
  
  

• Parallel or distributed execution may require additional license seats.
  
  

Common Pitfalls & Solutions

Problem	Cause	Solution
Tests do not run	Incorrect client configuration or endpoint	Verify paths, endpoint URLs, permissions, and network access.
Execution hangs	Remote service not running or agent offline	Start remote execution service or ensure DEX agents are online.
Reports missing	Result files not found	Ensure Tosca outputs to known folders and configure the CI task accordingly.
License errors	Insufficient license seats	Confirm license availability for all agents.
Regression slows pipeline	All tests run on every build	Use separate execution lists and conditional triggers.

Real-World Example: Bringing It All Together

Scenario: Web application with frequent feature work, unit tests, and Tosca UI regression.

Setup: Azure DevOps with self-hosted Windows agents, Tosca Server + DEX + AOS.

Pipeline Design:

1. Build Pipeline: Triggered on merges, runs unit tests, builds artifact.
   
   

2. Tosca Test Pipeline: Triggered after build completion; runs smoke tests using Tosca CI Client and publishes results.
   
   

3. Nightly Full Regression: Scheduled nightly, executes full regression suite.
   
   

With this design, every commit triggers a smoke suite, and nightly builds run full regression, giving developers rapid feedback and QA teams comprehensive coverage.

Why This Matters for TOSCA Training & Automation Learners

• Bridges test automation and DevOps: Learn how to embed automation into delivery pipelines.
  
  

• Scalable automation architecture: Build repeatable and maintainable workflows.
  
  

• Career readiness: CI/CD integration is often expected of testers and automation engineers.
  
  

• Improved test management skills: Learn to structure execution lists, handle environments, and process results effectively.
  
  

Adding CI/CD integration skills multiplies the value of any Tosca Training License or Tricentis TOSCA Online Training program.

Conclusion 

Integrating Tosca tests into Jenkins or Azure DevOps automates testing, enforces quality gates, and provides rapid feedback on code changes. Using the Tosca CI Client and proper pipeline configuration allows automated execution of UI, API, or regression tests for every build or scheduled run.

Start small: set up a pipeline to trigger Tosca smoke tests on build success, then gradually expand to full regression and nightly runs. This approach unlocks the true potential of Tosca Automation in a modern DevOps environment.