TestComplete SmartBear: Desktop Application Testing Platform

Introduction to TestComplete

TestComplete by SmartBear is a commercial automated testing platform specializing in desktop application testing, particularly for Windows-based applications. While it supports web and mobile testing, TestComplete’s differentiating strength lies in GUI-based desktop automation—a niche underserved by modern open-source frameworks like Selenium or Playwright that focus primarily on web browsers.

The platform targets QA teams testing legacy Windows desktop applications (WPF, WinForms, Qt, Delphi), enterprise Java desktop apps, and hybrid desktop+web applications where traditional web automation tools fall short. TestComplete’s object recognition engine can identify UI elements even without accessible names or IDs, making it valuable for testing third-party applications or legacy software without modern accessibility features.

This guide explores TestComplete’s architecture, desktop automation capabilities, keyword/data-driven testing features, licensing model, and when specialized desktop testing tools justify their premium pricing over general-purpose alternatives.

Core Architecture

Object Recognition Engine

TestComplete’s proprietary Name Mapping technology identifies GUI objects using multiple recognition strategies:

Recognition Methods:

• Name: Object names assigned by developers
• Text Content: Visible text labels
• Class Name: Win32 control class
• Window Position: Relative coordinates
• Image Recognition: Visual pattern matching (OCR)
• Object Hierarchy: Parent-child relationships

Example mapped object:

Window: CalculatorApp.exe
├─ Panel: MainPanel
│   ├─ Button: btnAdd (Text="Add", Class="System.Windows.Forms.Button")
│   ├─ TextBox: txtInput1 (Name="InputField1")
│   └─ TextBox: txtInput2 (Name="InputField2")
└─ MenuBar: MainMenu
    └─ MenuItem: FileMenu → Exit

When applications update, TestComplete automatically attempts to re-identify objects using alternative properties, reducing test maintenance.

Keyword-Driven Testing

TestComplete supports keyword-driven testing without code:

Keyword Test Editor: Visual workflow designer for creating tests by dragging operations

Built-in Keywords:

• Click: Click on object
• SetText: Enter text into field
• SelectItem: Choose from dropdown/list
• CheckProperty: Verify object property
• RunScript: Execute custom code
• CallTest: Invoke another keyword test (reusability)

Example keyword test:

Test: CalculatorAddition
1. Run TestedApp "Calculator.exe"
2. Click txtInput1
3. SetText txtInput1 "5"
4. Click txtInput2
5. SetText txtInput2 "3"
6. Click btnAdd
7. CheckProperty lblResult "Text" "8"
8. CloseApplication Calculator

This enables non-programmers to create automated tests, lowering the technical bar for test automation.

Data-Driven Testing

TestComplete integrates data sources for parameterized testing:

Supported Data Sources:

• Excel spreadsheets
• CSV files
• Database tables (SQL Server, MySQL, Oracle)
• XML files
• TestComplete data tables

Example data-driven test:

Data Source: TestData.xlsx
Columns: username, password, expectedResult

Keyword Test: LoginTest
1. SetText txtUsername [Data.username]
2. SetText txtPassword [Data.password]
3. Click btnLogin
4. CheckProperty lblStatus "Text" [Data.expectedResult]

Iterations: 50 (one per data row)

Tests automatically iterate through data rows, generating 50 test executions from a single test definition.

Scripted Testing

For complex scenarios, TestComplete supports scripting in:

JavaScript: Modern syntax, popular among web developers Python: Familiar to data scientists, ML engineers VBScript: Legacy support for existing test assets JScript: Microsoft’s JavaScript implementation DelphiScript: For Delphi application testing C++Script: Performance-critical test operations C#Script: .NET developer-friendly

Example Python script:

def Test_CalculatorAddition():
    # Launch application
    calculator = Sys.Process("Calculator")

    # Perform operations
    calculator.Window("MainForm").txtInput1.SetText("5")
    calculator.Window("MainForm").txtInput2.SetText("3")
    calculator.Window("MainForm").btnAdd.Click()

    # Verify result
    result = calculator.Window("MainForm").lblResult.Text
    aqObject.CheckProperty(result, "Text", cmpEqual, "8", "Addition verification")

    # Cleanup
    calculator.Close()

Key Features

Cross-Platform Desktop Testing

TestComplete supports testing on:

Windows: WPF, WinForms, Win32, .NET, Qt, Delphi, Electron, UWP macOS: Cocoa applications, Electron apps Linux: Qt applications, Java Swing/JavaFX Java Desktop: Swing, JavaFX, SWT (cross-platform)

This breadth is unmatched by open-source tools focused on web testing.

Test Visualization

Test Visualizer: Captures screenshots at every test step, creates visual test documentation

Video Recording: Records full test execution with audio commentary

Test Log: Hierarchical log with expandable checkpoints, screenshots, and timing

Example test log:

✓ Test: LoginFlow (3.2s)
  ├─ ✓ Launch Application (0.8s)
  ├─ ✓ Enter Credentials (0.5s)
  │   ├─ Screenshot: login_form.png
  │   └─ Checkpoint: Username field populated
  ├─ ✗ Click Login Button (1.2s)
  │   ├─ Screenshot: error_state.png
  │   ├─ Error: "Login button not found"
  │   └─ Suggested Fix: Update Name Mapping
  └─ ⊘ Verify Dashboard (skipped due to previous failure)

Distributed Testing

TestComplete supports test execution across multiple machines:

TestComplete Execution Engine: Lightweight agent for remote test execution

TestExecute: Runtime-only license for CI/CD servers ($2,000 vs. $7,000 for full TestComplete)

Network Suite: Distribute tests across machines on local network

Example distributed run:

Master Machine: Test Controller
├─ Agent 1 (Windows 10): Suite A (browsers: Chrome, Edge)
├─ Agent 2 (Windows 11): Suite B (browsers: Firefox)
└─ Agent 3 (macOS): Suite C (Safari)

Parallel execution: 3x faster than sequential

CI/CD Integration

TestComplete integrates with major CI/CD tools:

Jenkins: TestComplete Plugin for parameterized builds Azure DevOps: Native integration with Azure Pipelines TeamCity: TestComplete runner plugin GitLab CI: Command-line execution via TestExecute Bamboo: TestComplete task add-on

Example Jenkins pipeline:

stage('TestComplete Tests') {
  steps {
    testcompletetest(
      suite: 'Regression',
      project: 'DesktopApp.pjs',
      timeout: 120,
      actionOnWarnings: 'MAKE_UNSTABLE',
      actionOnErrors: 'MAKE_FAILED'
    )
  }
}

Comparison with Alternatives

TestComplete vs. Ranorex: Similar capabilities, TestComplete has broader language support, Ranorex has cleaner UI

TestComplete vs. Open Source: TestComplete offers better desktop support but at 100x the cost of open-source alternatives

TestComplete vs. Tosca: Tosca is more expensive but better for model-based testing and SAP applications

Pricing and Licensing

TestComplete Licensing

Base Edition: $7,595/user/year (floating license)

• Desktop + Web + Mobile testing
• All script languages
• Keyword/data-driven testing
• Standard support

Pro Edition: $8,995/user/year

• Base features +
• Cross-browser web testing
• Mobile cloud integration (AWS Device Farm, Sauce Labs)
• Priority support

Enterprise Edition: Custom pricing (contact sales)

• Pro features +
• TestExecute runtime licenses included
• Enterprise support with SLA
• Dedicated account manager

TestExecute (Runtime Only)

TestExecute: $2,099/machine/year

• Execute tests created in TestComplete
• No test authoring capabilities
• For CI/CD servers, distributed testing

Perpetual Licenses

One-time purchase: $6,379 (Base) to $9,499 (Pro)

• Plus annual maintenance: 25% of license price
• Break-even point: ~3-4 years vs. subscription

Node-Locked vs. Floating

Node-Locked: Tied to specific machine (10% cheaper) Floating: Shared across team (recommended for teams 5+)

Total Cost Examples

Solo Tester:

• TestComplete Base: $7,595/year
• Or perpetual: $6,379 + $1,595/year maintenance

Team of 5 QA Engineers:

• 3 floating licenses: $7,595 × 3 = $22,785/year
• 5 TestExecute for CI: $2,099 × 5 = $10,495/year
• Total: $33,280/year

Enterprise (20 testers):

• 10 floating licenses: Negotiated ~$60,000/year
• TestExecute licenses: Included in Enterprise deal
• Total: $60,000-80,000/year

Best Practices

Name Mapping Strategy

Create baseline mappings early: Map all application objects before scripting

Use stable properties: Prioritize Name > Text > Position (Name is least likely to change)

Update mappings incrementally: When UI changes, TestComplete suggests updates

Version control mappings: Store Name Mapping files in Git alongside tests

Test Organization

Organize tests hierarchically:

Project: DesktopApp
├─ KeywordTests/
│   ├─ Smoke/
│   ├─ Regression/
│   └─ DataDriven/
├─ Scripts/
│   ├─ Utilities/
│   └─ Workflows/
└─ Stores/
    ├─ TestData.xlsx
    └─ Config.ini

Reusability

Create test libraries: Reusable keyword tests called from other tests

Script routines: Common functions in script units

Project Suite: Group related projects, share code/data

Example reusable routine:

def Login(username, password):
    app = Sys.Process("MyApp")
    app.Window("LoginForm").txtUser.SetText(username)
    app.Window("LoginForm").txtPass.SetText(password)
    app.Window("LoginForm").btnLogin.Click()
    aqUtils.Delay(2000)  # Wait for dashboard load

CI/CD Best Practices

Use TestExecute for agents: Don’t pay for full TestComplete on build servers

Parameterize tests: Pass environment/config via command-line arguments

Generate JUnit XML: For integration with CI reporting

TestExecute.exe "C:\Tests\Project.pjs" /run /project:MyProject
  /test:RegressionSuite /ExportLog:"results.xml" /ExportLogFormat:JUnit

Limitations

Cost: Prohibitively expensive for small teams or open-source projects

Learning Curve: Feature-rich but complex, 2-3 weeks ramp-up time

Windows-Centric: Despite cross-platform claims, strongest on Windows

Heavy Resource Usage: IDE can be slow on underpowered machines

Vendor Lock-In: Tests created in TestComplete are difficult to migrate to other tools

Limited Community: Smaller community than Selenium/Appium, fewer online resources

Conclusion

TestComplete excels in a specific niche: organizations with significant Windows desktop application testing needs and budget for commercial tools. The platform’s advanced object recognition, keyword-driven testing, and comprehensive desktop technology support justify the premium pricing for teams where open-source alternatives fall short.

Choose TestComplete if:

• Testing legacy Windows desktop applications (WPF, WinForms, Qt)
• Need keyword-driven tests for non-programmer testers
• Testing applications without accessibility support
• Budget allows $7,000-9,000/user/year

Choose alternatives if:

• Primarily web/mobile testing (Selenium, Appium, Playwright cheaper/better)
• Open-source requirement (WinAppDriver, Appium)
• Budget-constrained (Ranorex $2,000 cheaper, open-source free)
• Need model-based testing (Tricentis Tosca better fit)

For organizations with desktop-heavy test portfolios and enterprise budgets, TestComplete provides a mature, fully-featured platform that reduces desktop automation complexity. However, teams should evaluate whether modern applications built with web technologies might be better served by less expensive web automation frameworks.