Test Summary vs Test Execution Report

Test Execution Report: A snapshot of a specific test run — how many tests passed/failed today.

Test Summary Report: A comprehensive document covering the entire testing phase — what was planned, what was done, what was found, and what it means for the release.

Report Structure

1. Executive Summary

One paragraph: Is the product ready? What is the biggest risk? What is your recommendation?

2. Scope and Objectives

What was planned to be tested, what was actually tested, and any deviations from the plan.

3. Test Results Summary

Aggregate results across all test cycles: total executed, passed, failed, blocked.

4. Defect Analysis

  • Defects by severity and status
  • Defect discovery trend (were bugs found early or late?)
  • Top defect categories (UI, API, performance, security)
  • Root cause analysis for critical bugs

5. Coverage Assessment

  • Requirements coverage: % of requirements with test cases
  • Code coverage: % from automated tests
  • Feature coverage: which features tested vs not tested

6. Risk Assessment

Remaining risks and their potential impact on the release.

RiskLikelihoodImpactMitigation
Untested edge cases in paymentMediumHighPost-release monitoring
Performance under peak load not verifiedHighHighSchedule load test before launch
3 deferred major bugsLowMediumFix in hotfix release if reported

7. Go/No-Go Recommendation

Clear recommendation with conditions:

  • Go: Release is ready. Remaining risks are acceptable.
  • Conditional Go: Release with specific conditions (e.g., hotfix plan for known issues)
  • No-Go: Critical issues prevent release. Specify what must change.

8. Lessons Learned

What went well and what should improve in the next testing cycle.

Exercise: Write a Test Summary Report

Write a test summary report for Release v4.0 of a banking application. Data:

  • 800 test cases planned, 780 executed, 720 passed, 45 failed, 15 blocked
  • 62 bugs found: 2 critical (both fixed), 12 major (10 fixed, 2 deferred), 30 minor, 18 trivial
  • One deferred major bug: international transfer limits not enforced for certain currencies
  • Code coverage: 82% overall, 93% in payment modules
  • Testing took 3 weeks (planned: 2 weeks) due to environment issues in week 1
Solution

Executive Summary: Release v4.0 is conditionally ready. 92.3% pass rate exceeds the 90% target. Both critical bugs are fixed and verified. However, one deferred major bug (international transfer limits) presents regulatory risk. Recommend release with immediate hotfix plan for BUG-445.

Go/No-Go: Conditional Go. Conditions: 1) BUG-445 hotfix branch ready before release. 2) Monitoring alert for international transfers above limits. 3) Customer support briefed on known issue.

Lessons Learned:

  • Environment issues caused 1-week delay. Action: Containerize test environment for v4.1.
  • Payment module coverage at 93% was achieved through pairing QA and dev — continue this practice.
  • 15 blocked tests were all caused by one flaky third-party API. Action: Implement mock service for testing.

Key Takeaways

  • Test summary reports are end-of-phase documents, not daily snapshots
  • Lead with the executive summary and go/no-go recommendation
  • Include risk assessment — what could go wrong after release?
  • Quantify everything: pass rates, coverage percentages, defect counts
  • End with actionable lessons learned to improve the next cycle