Feature Flags and Testing

What Are Feature Flags? #

Feature flags (also called feature toggles) are conditional statements in code that control whether a feature is active. They decouple code deployment from feature release — you can deploy code to production with new features hidden, then enable them gradually.

if (featureFlags.isEnabled('new-checkout-flow')) {
  renderNewCheckout();
} else {
  renderLegacyCheckout();
}

For QA engineers, feature flags add testing complexity but also provide powerful capabilities: you can test features in production safely, control A/B experiments, and roll back problematic features instantly without redeploying.

Types of Feature Flags #

Testing Strategy for Feature Flags #

The Combinatorial Challenge #

With N independent flags, there are 2^N possible combinations. With just 10 flags, that is 1,024 combinations — impractical to test exhaustively.

Pragmatic approach:

1. Test each flag independently in both on and off states (2N tests)
2. Identify dependent flags that interact and test those combinations
3. Test transitions — toggling a flag from off to on (and back) during a user session
4. Test the default state — what users see when the flag service is down

Test Matrix Example #

For a “new-checkout” feature flag:

Testing Progressive Rollouts #

Progressive rollouts expose features to increasing percentages of users:

Day 1: 1% of users  → Monitor error rates
Day 2: 5% of users  → Check key metrics
Day 3: 25% of users → Broader validation
Day 7: 100%         → Full release

QA responsibilities:

• Verify the percentage targeting works correctly
• Monitor error rates and user metrics at each stage
• Have a rollback plan ready
• Test that users in the rollout see the feature consistently

Feature Flag Tools #

Automation with Feature Flags #

Testing Both States in CI #

// playwright.config.ts
const projects = [
  {
    name: 'feature-off',
    use: {
      baseURL: 'https://staging.example.com',
      extraHTTPHeaders: { 'X-Feature-Flag': 'new-checkout=false' },
    },
  },
  {
    name: 'feature-on',
    use: {
      baseURL: 'https://staging.example.com',
      extraHTTPHeaders: { 'X-Feature-Flag': 'new-checkout=true' },
    },
  },
];

API-Based Flag Control #

// Before tests: enable flag
await fetch('https://api.launchdarkly.com/flags/new-checkout', {
  method: 'PATCH',
  headers: { Authorization: `Bearer ${LD_API_KEY}` },
  body: JSON.stringify({ on: true }),
});

// Run tests...

// After tests: restore original state
await fetch('https://api.launchdarkly.com/flags/new-checkout', {
  method: 'PATCH',
  body: JSON.stringify({ on: false }),
});

Exercise: Design a Flag Testing Strategy #

Your team is launching a new recommendation engine behind a feature flag. The flag has three variants: “off” (legacy), “basic” (simple recommendations), “advanced” (ML-powered recommendations). It is being rolled out progressively: 1% → 10% → 50% → 100%.

Design the testing strategy.

Best Practices #

1. Clean up old flags. Feature flags are technical debt. Once a feature is fully released, remove the flag and the old code path. Track flag lifecycle with a register.

2. Test the fallback behavior. If the flag service is down, what happens? The application should have sensible defaults.

3. Never nest feature flags deeply. Two levels of nesting maximum. More leads to untestable complexity.

4. Use flag overrides in test environments. Test tools should be able to force-enable or force-disable flags regardless of the targeting rules.

5. Monitor flag state changes. Log when flags are toggled and by whom. This helps correlate production issues with flag changes.