What Is Pairwise Testing?

Pairwise testing (also called all-pairs testing) is a combinatorial test design technique based on a key observation: most defects are caused by interactions between two parameters, not three or more simultaneously.

Instead of testing every possible combination (which grows exponentially), pairwise testing guarantees that every pair of parameter values appears in at least one test case.

The Combinatorial Explosion Problem

Consider testing a web application across:

  • OS: Windows, macOS, Linux (3)
  • Browser: Chrome, Firefox, Safari, Edge (4)
  • Language: EN, ES, RU (3)
  • Screen: Desktop, Tablet, Mobile (3)

Exhaustive testing: 3 x 4 x 3 x 3 = 108 combinations

Pairwise testing: approximately 12-15 test cases — covering every pair.

How Pairwise Works

flowchart TD A[List all parameters and values] --> B[Generate all possible pairs] B --> C[Create test cases that cover multiple pairs each] C --> D[Optimize: minimize total test cases] D --> E[Result: compact test suite]

For 3 parameters (A, B, C) with 2 values each (1, 2):

All pairs that need coverage:

  • A-B: (1,1), (1,2), (2,1), (2,2)
  • A-C: (1,1), (1,2), (2,1), (2,2)
  • B-C: (1,1), (1,2), (2,1), (2,2)

A pairwise suite with just 4 test cases:

TestABC
1111
2122
3212
4221

Every pair appears at least once. Exhaustive would need 2^3 = 8 tests.

Introduction to PICT

PICT (Pairwise Independent Combinatorial Testing) is Microsoft’s free, open-source command-line tool for generating pairwise test suites.

Installation:

# macOS
brew install pict

# Linux (build from source)
git clone https://github.com/microsoft/pict.git
cd pict && make

# Windows
# Download from GitHub releases

Basic usage:

Create a model file web-test.txt:

OS:       Windows, macOS, Linux
Browser:  Chrome, Firefox, Safari, Edge
Language: EN, ES, RU
Screen:   Desktop, Tablet, Mobile

Run PICT:

pict web-test.txt

Output (example — actual combinations may vary):

OS       Browser  Language  Screen
Windows  Chrome   EN        Desktop
macOS    Firefox  ES        Tablet
Linux    Safari   RU        Mobile
Windows  Edge     RU        Tablet
macOS    Chrome   RU        Desktop
Linux    Firefox  EN        Desktop
...

PICT generates approximately 12-16 test cases instead of 108.

Why Pairwise Is Effective

Research by Kuhn, Wallace, and Gallo (NIST) found:

  • 67% of defects are triggered by a single parameter value
  • 93% are triggered by pairs of values (2-way interactions)
  • 98% are triggered by triples (3-way)
  • Nearly 100% by 4-way interactions

Pairwise covers the 93% level with a fraction of exhaustive testing effort.

Advanced PICT Features

Adding Constraints

Not all combinations are valid. PICT supports constraints to exclude impossible cases:

OS:       Windows, macOS, Linux, iOS, Android
Browser:  Chrome, Firefox, Safari, Edge

# Constraints
IF [OS] = "iOS"    THEN [Browser] <> "Edge";
IF [OS] = "iOS"    THEN [Browser] <> "Firefox";
IF [OS] = "Android" THEN [Browser] <> "Safari";
IF [OS] = "Android" THEN [Browser] <> "Edge";
IF [OS] = "Linux"  THEN [Browser] <> "Safari";

PICT will never generate a test case with iOS + Edge or Android + Safari.

Sub-Models for Higher-Order Coverage

Sometimes certain parameter groups need stronger coverage. PICT supports sub-models:

OS:       Windows, macOS, Linux
Browser:  Chrome, Firefox, Safari
Language: EN, ES, RU
Theme:    Light, Dark

# Default: pairwise (2-way)
# But OS-Browser-Language needs 3-way coverage
{ OS, Browser, Language } @ 3

This ensures every triple of OS-Browser-Language values appears in at least one test, while other parameter interactions remain at pairwise level.

Seeding Specific Combinations

Force specific important combinations to appear:

Create a seed file seeds.txt:

OS        Browser  Language  Theme
Windows   Chrome   EN        Light
macOS     Safari   ES        Dark

Run with seeding:

pict model.txt /e:seeds.txt

PICT will include these exact combinations and optimize around them.

Weighted Parameters

Give priority to common configurations:

OS:       Windows(7), macOS(2), Linux(1)
Browser:  Chrome(6), Firefox(2), Safari(1), Edge(1)

Higher weights make those values appear in more test cases, increasing coverage for common configurations.

Negative Testing with PICT

Add invalid values prefixed with ~:

Age:    18, 30, 65, ~-1, ~200
Email:  valid@test.com, ~invalid-email, ~""

PICT ensures negative values appear in test cases but only one negative value per test case (single fault assumption).

Real-World Example: API Testing Matrix 

Method:      GET, POST, PUT, DELETE
Auth:        Bearer, API-Key, None
ContentType: application/json, multipart/form-data, text/plain
Status:      200, 400, 401, 404, 500

IF [Method] = "GET"  THEN [ContentType] <> "multipart/form-data";
IF [Auth] = "None"   THEN [Status] IN { 401 };
IF [Method] = "DELETE" THEN [ContentType] = "application/json";

Exercise: Generate a Pairwise Suite with PICT

Scenario: You’re testing a payment processing form with:

  • Card type: Visa, MasterCard, Amex
  • Currency: USD, EUR, GBP
  • Amount range: Small (<$10), Medium ($10-$100), Large (>$100)
  • 3D Secure: Enabled, Disabled
  • Recurring: Yes, No

Constraints:

  • Amex does not support recurring payments
  • GBP is not available with 3D Secure Disabled

Tasks:

  1. Write the PICT model file
  2. Calculate how many exhaustive tests would be needed
  3. Estimate the pairwise count
  4. Add constraints and verify the output
Hint

Exhaustive: 3 x 3 x 3 x 2 x 2 = 108 tests. PICT will generate roughly 12-18 tests with constraints.

The model file should define parameters on separate lines and constraints with IF/THEN syntax.

Solution

PICT model file (payment-test.txt):

CardType:  Visa, MasterCard, Amex
Currency:  USD, EUR, GBP
Amount:    Small, Medium, Large
3DSecure:  Enabled, Disabled
Recurring: Yes, No

IF [CardType] = "Amex" THEN [Recurring] = "No";
IF [Currency] = "GBP"  THEN [3DSecure] = "Enabled";

Exhaustive: 3 x 3 x 3 x 2 x 2 = 108 tests (minus impossible combinations)

Pairwise with PICT: approximately 12-15 test cases

Run: pict payment-test.txt

Example output:

CardType     Currency  Amount  3DSecure  Recurring
Visa         USD       Small   Enabled   Yes
MasterCard   EUR       Medium  Disabled  No
Amex         GBP       Large   Enabled   No
Visa         EUR       Large   Disabled  Yes
MasterCard   GBP       Small   Enabled   Yes
Amex         USD       Medium  Enabled   No
Visa         GBP       Medium  Enabled   No
MasterCard   USD       Large   Enabled   No
Amex         EUR       Small   Enabled   No
Visa         USD       Large   Enabled   No
MasterCard   EUR       Small   Enabled   Yes
Visa         EUR       Small   Disabled  No

~12 tests instead of 108 — an 89% reduction while covering all pairs.

Pro Tips

  • Start with pairwise, increase order if needed. 2-way (pairwise) catches ~93% of defects. If your domain is high-risk, bump critical parameter groups to 3-way.
  • Use constraints liberally. Every impossible combination removed is a wasted test avoided. Review your model with developers to catch all constraints.
  • Combine with other techniques. Use pairwise for the parameter combinations, then apply BVA to select specific values within each parameter.
  • Automate the pipeline. Integrate PICT into your CI: model file → PICT generation → parameterized test execution.
  • Version control your PICT models. Treat model files as test artifacts — review them, store them alongside tests, update them when parameters change.