Module 1 Assessment

1. Most deficient STLC phase: Test Design and Test Execution

The variety of escaped defects (calculation logic, security, platform-specific display) suggests insufficient test coverage rather than a single process gap. This points to the Test Design phase — test cases are either not being created or do not cover critical scenarios. Additionally, the Test Execution phase may be deficient if tests exist but are not being run before deployment.

A secondary deficiency is in the Requirements Analysis/Planning phase — if risk-based testing were in place, payment calculations and login security would be the highest-priority test areas.

2. Top 3 metrics:

1. Defect Escape Rate — Track how many defects reach production per release. This directly measures the problem (bugs escaping). Target: reduce from current rate by 50% within 3 months.

2. Test Coverage (by risk level) — Measure what percentage of high-risk features (payments, security, core flows) have test cases. This will reveal where gaps exist. Target: 100% coverage for Critical and High-risk features.

3. Deployment Success Rate — Track what percentage of deployments go to production without rollback. This gives a single number that management can track. Target: 95%+ success rate.

Why these three: They directly address the current problem (production defects), are easy to start collecting immediately, and provide actionable data (coverage gaps tell you where to focus testing effort).

1. Requirements coverage:

• Total active requirements: 200 - 15 (deferred) = 185 active
• Requirements with test cases: 180 - (some may be deferred)
• If all 20 unlinked requirements include the 15 deferred: 5 active requirements with no tests
• Coverage = (185 - 5) / 185 × 100 = 97.3%
• Or conservatively: 180/200 × 100 = 90%

The exact percentage depends on whether the 15 deferred requirements are among the 20 without test cases. For the audit, present both the total and active coverage.

2. Three types of gaps:

3. Recommendations before the audit:

1. Immediately link or justify the 20 unlinked requirements. For the 5 non-deferred requirements without tests, either write test cases or document a risk-based justification for not testing them. This is the most critical action.

2. Review the 50 orphaned test cases. For each, either link to an existing requirement or document why the test exists without a formal requirement (e.g., security best practice, edge case coverage). Remove any truly unnecessary tests.

3. Document the 15 deferred requirements. Create a formal deferral record for each: why it was deferred, who approved the deferral, what the plan is for future implementation, and confirmation that the deferral does not affect patient safety.

4. Prepare an RTM summary report showing: total coverage percentage, coverage by risk level, list of gaps with justifications, and sign-off from the QA Lead and Compliance Officer.

1. Recommendation: TPI Next

Reasoning:

• Flexibility: With 5 teams at different maturity levels, TPI Next’s ability to improve individual key areas independently is more practical than TMMi’s requirement to satisfy all areas at a level.
• Budget: $50K/year does not support formal TMMi assessment and certification (which can cost $20K+ per assessment). TPI Next can be done with internal resources.
• Quick wins: TPI Next allows targeting specific weak areas for immediate improvement, delivering visible results that build momentum and executive support.
• ISO 27001 alignment: TPI Next’s security-related key areas (test environment, test data management) directly support ISO 27001 preparation.

However, if the company later needs formal maturity certification (e.g., for outsourcing contracts), TMMi can be pursued on top of TPI Next improvements.

2. First 3 improvement actions:

1. Create an Organizational Test Policy (TPI Next: Stakeholder Management, Controlled level)

   • Define minimum testing standards all 5 teams must follow
   • Include: mandatory test types, minimum coverage thresholds, defect management process, reporting requirements
   • Timeline: 1 month to draft, 1 month for review and rollout
   • Cost: Internal effort only (~$5K in time)
   • Why first: This establishes the foundation. Without a policy, each team will continue doing things differently.

2. Standardize Reporting across all teams (TPI Next: Reporting, Controlled level)

   • Define a common test status report template
   • Implement weekly reporting from all 5 teams
   • Create a consolidated quality dashboard for the CTO
   • Timeline: 2 months
   • Cost: ~$5K (tool configuration)
   • Why second: Visibility is essential. The CTO needs data to justify continued investment, and teams need to see how they compare.

3. Establish a common defect management process (TPI Next: Defect Management, Controlled level)

   • Define defect lifecycle, severity/priority definitions, SLAs by severity
   • Ensure all teams use the same tracking tool and workflow
   • Start tracking DRE and defect escape rate across all teams
   • Timeline: 2 months
   • Cost: ~$5K (tool configuration, training)
   • Why third: Consistent defect management enables meaningful cross-team metrics and identifies which teams need the most help.

Total Phase 1 cost: ~$15K, leaving $35K for tool investments, training, and Phase 2 improvements.

• Payment processing is the highest-risk feature (financial data, Stripe integration)
• Quiz taking must be accurate (affects certification)
• Certificate generation must be reliable (has legal/professional value)
• Consider GDPR for user data handling
• With 5 testers and bi-weekly releases, you cannot test everything manually every sprint
• The testing pyramid applies: automate regression, manually explore new features

Test Plan Outline: Online Learning Platform #

1. Scope

In scope:

• All student-facing features (registration, enrollment, lessons, quizzes, progress, certificates)
• Payment processing (Stripe) — all payment scenarios
• API testing — all public endpoints
• Cross-browser: Chrome, Firefox, Safari, Edge (latest 2 versions)
• Mobile responsive: iOS Safari, Android Chrome
• Performance: page load times, video streaming, quiz submission
• Security: authentication, payment data, user data protection
• GDPR: consent management, data export, data deletion, cookie policy
• Accessibility: WCAG 2.1 AA for all student-facing pages

Out of scope:

• Course content creation admin panel (low risk, internal users only, separate test cycle)
• Analytics and reporting dashboard (informational only, no data modification)
• Infrastructure testing beyond application layer (AWS responsibility)
• Load testing beyond 5x current traffic (planned separately)

2. Risk Assessment

3. Test Approach

Automation strategy: automate regression first (payment flow, quiz flow, enrollment flow). Target: 70% regression automation within 6 months.

4. Entry/Exit Criteria for System Testing

Entry:

• Integration testing complete with 95%+ pass rate
• Build deployed to staging environment
• Test data prepared (courses, users, payment sandbox)
• All critical/high defects from previous sprint resolved

Exit:

• 95% of planned test cases executed
• Zero open Critical defects, fewer than 3 High defects
• Payment flow: 100% test execution, 100% pass rate
• Regression suite passes with <2% failure rate
• Performance benchmarks met (pages load <3s, quiz submission <1s)

5. Key Metrics

1. DRE — Target >90%. Measures overall testing effectiveness.
2. Defect Escape Rate — Target <5%. Tracks bugs reaching production.
3. Regression Automation Coverage — Target 70% in 6 months. Measures automation progress.
4. Payment Defect Rate — Target 0 critical/high. Tracks our highest-risk area.
5. Sprint Velocity Impact — Track whether testing delays releases. Target: <5% of sprints delayed by testing.

6. Resource Allocation

Rotation: All testers participate in sprint planning and do at least one 30-minute exploratory session per sprint on any feature.