TL;DR: Zebrunner is an enterprise test automation reporting platform that converts test execution noise into actionable quality signals through ML-powered failure analysis and real-time dashboards. Integrates with Selenium, Appium, Playwright, Cypress, Jenkins, and GitHub Actions. Best for teams with 1,000+ automated tests needing intelligent triage.

Zebrunner is a modern test automation reporting and analytics platform purpose-built for engineering teams managing large-scale automated test suites. As test suites grow to thousands of executions across multiple browsers, devices, and environments, traditional reporting tools create more noise than signal. Zebrunner addresses this through ML-powered failure analysis, real-time execution dashboards, and intelligent test stability tracking. According to a Capgemini World Quality Report 2023, 44% of QA teams report that inefficient test result analysis is a top obstacle to increasing test automation ROI. Test intelligence platforms like Zebrunner reduce triage time by 60–70% compared to raw log analysis, according to the platform’s published customer case studies. The platform supports Selenium, Appium, Playwright, Cypress, TestNG, JUnit, and pytest frameworks through official agents, and integrates with Jenkins, GitHub Actions, GitLab CI, and Jira for end-to-end quality visibility across the entire delivery pipeline.

Introduction to Zebrunner

Zebrunner is a modern test automation reporting and analytics platform designed specifically for engineering teams running large-scale automated test suites. Unlike traditional Test Case Management tools focused on manual testing, Zebrunner targets the “automated tests problem”: as test suites grow to thousands of tests executed across multiple browsers, devices, and environments, understanding failures becomes overwhelming.

“Test results without context are just noise. Zebrunner’s value is turning thousands of pass/fail states into quality signals — showing you not just what failed, but whether the failure is a flaky test, a regression, or a known issue that your team already tracked.” — Yuri Kan, Senior QA Lead

Zebrunner solves this through intelligent test result aggregation, ML-powered failure analysis, real-time dashboards, and deep integrations with CI/CD pipelines and test execution infrastructure. The platform’s core value proposition: transform test execution noise into actionable quality signals.

Founded by the team behind the open-source Zebrunner (formerly Zafira) reporting framework, the commercial Zebrunner platform builds on years of community feedback to deliver enterprise-grade test intelligence.

Core Architecture

Test Session Recording

Zebrunner captures every test execution as a Test Session containing:

Execution Metadata: Test name, parameters, tags, environment, build number

Artifacts: Screenshots, videos, logs, network traffic, device logs

Timeline: Step-by-step execution trace with timing

Known Issues: Links to filed bugs or known failure patterns

Example session view:

Test: checkout_with_paypal
Status: Failed
Duration: 42.3s
Browser: Chrome 120 (Linux)
Build: #3456

Timeline:
├─ 0.0s: Navigate to /cart
├─ 1.2s: Click "Checkout"
├─ 2.5s: Select PayPal payment
├─ 8.3s: [FAILED] PayPal iframe timeout
└─ 42.3s: Test terminated

Artifacts:

- screenshot_failure.png
- browser_console.log
- network_traffic.har

Real-Time Dashboard

Zebrunner provides live updates during test execution:

Test Run Progress: Tests queued, running, passed, failed in real-time

Failure Heat Map: Which test classes/features have highest failure rate

Execution Timeline: When tests started/completed, parallel execution visualization

Environment Status: Device farm availability, Selenium Grid capacity

This enables QA leads to monitor overnight regression runs and triage failures as they occur rather than waiting for CI pipeline completion.

ML-Powered Failure Classification

Zebrunner automatically categorizes test failures:

Product Defects: Reproducible application bugs requiring fix

Test Automation Issues: Flaky locators, race conditions, test data problems

Environment Problems: Infrastructure failures (Grid down, device offline)

Known Issues: Failures matching existing bug tickets

Machine learning models analyze failure patterns (error messages, stack traces, screenshots) to classify new failures, reducing manual triage time by 60-70%.

Test Stability Metrics

Zebrunner tracks test reliability over time:

Stability Score: Percentage of consistent pass/fail results (not flakiness)

Test: user_login_valid_credentials
Last 100 runs: 97 passed, 3 failed
Stability: 97%
Trend: Stable (was 96% last week)

Flakiness Detection: Tests with inconsistent results flagged automatically

Test: checkout_payment_processing
Last 100 runs: 78 passed, 22 failed (intermittent)
Flakiness: High
Recommended: Quarantine for investigation

Mean Time to Repair (MTTR): Average time from test failure to passing fix

Key Features

Multi-Framework Support

Zebrunner integrates with major test frameworks via agents:

Selenium/WebDriver: Java TestNG, JUnit, Python pytest, JavaScript WebdriverIO

Appium: Mobile iOS/Android test frameworks

Playwright: Native Playwright test reporting

Cypress: Via Cypress plugin

REST Assured: API test result import

Example integration (Java TestNG):

<dependency>
  <groupId>com.zebrunner</groupId>
  <artifactId>agent-testng</artifactId>
  <version>1.9.5</version>
</dependency>

@Test
@TestLabel(name = "priority", value = "critical")
@TestLabel(name = "feature", value = "checkout")
public void testGuestCheckout() {
  // Zebrunner automatically captures screenshots on failure
  // Links test to JIRA tickets via @TestCaseKey annotation
}

Smart Test Launcher

Zebrunner can trigger test execution (not just report results):

Scheduled Runs: Cron-based test execution

On-Demand: Manual trigger from UI with parameter selection

Smart Retry: Automatically rerun failed tests to distinguish flaky from real failures

Selective Execution: Run only tests affected by code changes

This eliminates need for custom CI/CD scripting for common test orchestration scenarios.

Integration Ecosystem

CI/CD: Jenkins, GitLab CI, GitHub Actions, CircleCI, Bamboo

Test Infrastructure: Selenium Grid, Browserstack, Sauce Labs, LambdaTest, device farms

Issue Trackers: JIRA, GitHub Issues, Azure DevOps

Notifications: Slack, Microsoft Teams, email with failure summaries

Test Management: TestRail, Zephyr, qTest (bidirectional sync)

Example Slack notification:

🔴 Regression Suite Failed (Build #3456)
━━━━━━━━━━━━━━━━━━━━━━━━
Passed: 487 | Failed: 13 | Skipped: 2
Pass Rate: 97.4% (was 98.1% yesterday)

Top Failures:

  - checkout_paypal (known issue: PAY-1234)
  - user_profile_edit (flaky: under investigation)
  - search_filtering (NEW: requires triage)

View Report: https://zebrunner.company.com/run/3456

Test Artifacts Management

Zebrunner provides centralized artifact storage:

Video Recording: Automatic video capture for web and mobile tests

Screenshots: On-demand and automatic on failure

Logs: Application logs, browser console, Appium server logs

Network Traffic: HAR files for API call analysis

Custom Artifacts: Test data files, generated reports

Artifacts are automatically associated with test sessions and retained per configured policy (30-90-180 days).

Comparison with Alternatives

FeatureZebrunnerReportPortalAllure TestOpsTestRailGrafana + Custom
Framework Support✅ 10+ frameworks✅ 15+ frameworks✅ 15+ frameworks⚠️ Via API⚠️ Custom
ML Failure Analysis✅ Built-in✅ Built-in⚠️ Basic❌ No❌ No
Real-Time Dashboard✅ Yes✅ Yes✅ Yes❌ Post-run only✅ Yes
Test Orchestration✅ Smart launcher❌ Reporting only✅ Full❌ No⚠️ External
Video/Screenshot✅ Automatic✅ Automatic⚠️ Via sidecar❌ No⚠️ Custom
SaaS + On-Prem✅ Both✅ Both✅ Both✅ Both⚠️ Self-hosted
Price$$ Medium$ Open-source$$$ High$$ Medium$ Infrastructure

Zebrunner vs. ReportPortal: Zebrunner offers commercial SaaS with support, ReportPortal is fully open-source but requires more setup

Zebrunner vs. Allure TestOps: Similar capabilities, Zebrunner focuses on ML-powered triage, Allure on test case documentation

Zebrunner vs. Grafana: Grafana requires custom dashboards and metric collection, Zebrunner is purpose-built for testing

Pricing and Licensing

Zebrunner Cloud

Startup: $99/month

  • Up to 10,000 test executions/month
  • 30-day data retention
  • Standard integrations
  • Community support

Business: $299/month

  • Up to 50,000 test executions/month
  • 90-day data retention
  • All integrations
  • Email support
  • Smart Test Launcher

Enterprise: Custom pricing

  • Unlimited test executions
  • Custom data retention
  • SSO, audit logs
  • Dedicated support
  • On-premise option

Zebrunner On-Premise

Self-Hosted License: Starting at $12,000/year

  • Perpetual license available
  • Deployment on your infrastructure
  • All Enterprise features

Open-Source Alternative: Zebrunner CE (Community Edition)

  • Free, limited features
  • Self-hosted only
  • Community support
  • Good for evaluation

Cost Example

Team running 100K tests/month:

  • Zebrunner Business: ~$500/month (volume discount)
  • Allure TestOps: ~$1,500-2,000/month (per-user pricing)
  • ReportPortal Open-Source: Free + $200-500/month infrastructure
  • Custom Grafana: $300-1,000/month (development + infrastructure)

Zebrunner offers competitive pricing for high-volume automation scenarios.

Best Practices

Test Labeling Strategy

Use consistent labels for powerful filtering:

@TestLabel(name = "priority", value = "P1")
@TestLabel(name = "feature", value = "payments")
@TestLabel(name = "platform", value = "web")
@TestLabel(name = "smoke", value = "true")

Enables queries: “Show all P1 payment tests that failed in last 7 days”

Failure Triage Workflow

  1. Daily Review: QA lead reviews new failures each morning
  2. Classification: Zebrunner ML suggests classification, human confirms
  3. Assignment: Link to existing JIRA or create new bug
  4. Quarantine: Move flaky tests to separate suite
  5. Weekly Cleanup: Review quarantined tests, fix or remove

This systematic approach prevents test suite decay.

Integration with Test Management

Link Zebrunner (execution) with TestRail (test design):

@TestCaseKey("TC-1234")  // TestRail test case ID
public void testCheckoutFlow() {
  // Zebrunner reports execution to TestRail
}

TestRail shows test case design, Zebrunner shows execution history—best of both worlds.

Custom Dashboard Creation

Build executive dashboards:

Quality Scorecard:

  • Pass rate trend (last 30 days)
  • Test stability score
  • MTTR (mean time to repair)
  • Top 10 flaky tests

Team Velocity:

  • Tests automated per sprint
  • Test execution count per day
  • Defect discovery rate

Zebrunner’s widget system enables no-code dashboard building.

Limitations

Not a Test Case Management Tool: Zebrunner doesn’t manage test case design, only execution results

Requires Automation: No value for pure manual testing teams (use TestRail instead)

Learning Curve: ML-powered features require training data (100+ executions) to be effective

Cost at Scale: High test execution volumes can become expensive on cloud tier

Limited Mobile-Specific Features: Works with Appium but lacks device farm management features

Implementation Recommendations

Phase 1: Pilot (Week 1-2)

  1. Select test suite: Choose 100-200 automated tests
  2. Instrument tests: Add Zebrunner agent
  3. Run executions: Generate 20-30 runs for ML training
  4. Configure integrations: Connect JIRA, Slack

Phase 2: Rollout (Week 3-4)

  1. Expand coverage: Add remaining test suites
  2. Train team: Failure triage workflow
  3. Set up dashboards: Create quality scorecards
  4. Establish metrics: Define stability/MTTR targets

Phase 3: Optimization (Month 2+)

  1. Tune ML models: Provide feedback on classifications
  2. Quarantine flaky tests: Achieve 95%+ stability
  3. Automate workflows: Slack notifications, auto-retry
  4. Optimize costs: Adjust retention policies

Conclusion

Zebrunner excels for engineering teams with substantial automated test investments who are drowning in test execution data. The platform’s ML-powered failure classification, real-time dashboards, and smart test orchestration transform chaotic test results into structured quality intelligence.

Choose Zebrunner if:

  • Running 10,000+ automated tests monthly
  • Struggling with failure triage overhead
  • Need real-time test execution visibility
  • Want intelligent test retry and orchestration

Choose alternatives if:

  • Primarily manual testing (TestRail better fit)
  • Want fully open-source (ReportPortal)
  • Need requirements traceability (Aqua ALM better fit)
  • Have <1,000 tests (overhead not justified)

For automation-heavy teams, Zebrunner delivers ROI through reduced triage time (60-70% savings), faster feedback loops (real-time vs. post-run), and improved test suite health (stability tracking). The platform represents the evolution from “test execution” to “test intelligence”—understanding not just what failed, but why, and what to do about it.

Official Resources

Test result analysis is a key bottleneck: the Capgemini World Quality Report 2023 found that 44% of QA teams cite inefficient test result analysis as a top obstacle to improving automation ROI.

According to Zebrunner customer case studies, teams adopting the platform reduce triage time by 60–70% compared to manual log analysis, with significant improvements in flaky test detection accuracy.

FAQ

What is Zebrunner and who is it for?

Zebrunner is a commercial test analytics platform for teams with 1,000+ automated tests — providing ML failure classification, real-time dashboards, and CI/CD integrations.

Zebrunner is a test automation reporting and analytics platform for engineering teams running 1,000+ automated tests. It transforms test execution noise into actionable signals through ML-powered failure analysis, real-time dashboards, video recording, and deep CI/CD integrations.

How does Zebrunner compare to ReportPortal?

Zebrunner: commercial SaaS, session recording, managed infra. ReportPortal: open-source, self-hosted, zero cost. Choose Zebrunner for enterprise support and mobile video capture.

Zebrunner is commercial SaaS with enterprise support and session recording; ReportPortal is open-source and self-hosted. Zebrunner excels at mobile testing with video capture; ReportPortal offers more customization and zero licensing cost. Choose Zebrunner for managed infrastructure and premium support.

What CI/CD and test frameworks does Zebrunner support?

Zebrunner supports Jenkins, GitHub Actions, GitLab CI, TeamCity, CircleCI plus Selenium, Appium, Playwright, Cypress, TestNG, JUnit, and pytest.

Zebrunner integrates natively with Jenkins, GitHub Actions, GitLab CI, TeamCity, and CircleCI. Framework support includes Selenium, Appium, Playwright, Cypress, TestNG, JUnit, and pytest through official agents and plugins.

How much does Zebrunner cost?

Subscription pricing based on parallel capacity and team size. Small teams: a few hundred dollars/month. Enterprise: custom pricing. Free trial available.

Zebrunner uses subscription pricing based on parallel execution capacity and team size. Small team plans start at a few hundred dollars per month. Enterprise pricing is custom with SSO, dedicated support, and advanced analytics. A free trial is available.

See Also