Specification by Example: Living Documentation Through Collaborative Examples

The Problem with Traditional Requirements

Traditional requirements documents suffer from fundamental flaws:

Ambiguity: “The system should be user-friendly” means different things to different people
Obsolescence: Documentation becomes outdated the moment code changes
Communication gaps: Business stakeholders and developers interpret requirements differently
Lack of validation: No way to verify if implementation matches intent

Specification by Example (SbE) addresses these issues by using concrete examples as the primary artifact for specifying, implementing, and testing software.

What Is Specification by Example?

Specification by Example is a collaborative approach to defining requirements and tests. Rather than writing abstract specifications, teams create living documentation consisting of executable examples that illustrate system behavior.

Core Principles

1. Derive scope from goals: Start with business goals, not features 2. Specify collaboratively: Business, development, and testing work together 3. Illustrate using examples: Concrete scenarios, not abstract descriptions 4. Refine the specification: Iterate on examples until they’re clear and testable 5. Automate validation: Turn examples into executable tests 6. Validate frequently: Examples double as regression tests 7. Evolve the documentation: Update examples as system evolves

The SbE Process

1. Example Workshops (Three Amigos)

Representatives from three perspectives collaborate:

Business (Product Owner): What value are we delivering? Development (Engineer): How will we build it? Testing (QA): What could go wrong?

Workshop Agenda:

Present user story/feature
Brainstorm example scenarios
Identify happy paths, edge cases, error conditions
Refine examples until everyone agrees
Document in structured format

2. Example Format

Examples typically follow Given-When-Then format (Gherkin syntax):

Scenario: Premium customer receives loyalty discount
  Given a customer with premium membership
  And they have been a member for 3 years
  When they purchase an item worth $100
  Then they receive a 15% discount
  And the total is $85

3. Automation

Examples are automated using tools like:

Cucumber: BDD framework for Gherkin specifications
FitNesse: Wiki-based acceptance testing framework
Concordion: Turns plain-language specs into executable tests
SpecFlow: .NET implementation of Cucumber

4. Validation

Automated examples run continuously:

During development (fast feedback)
In CI/CD pipeline (regression safety)
As living documentation (always up-to-date)

Concrete Example: E-Commerce Discounts

Traditional Requirements Document

The system shall apply discounts based on customer loyalty tier.
Premium customers receive higher discounts than standard customers.
Discounts vary by membership duration and purchase amount.

Problems:

What exactly are the discount percentages?
What defines a “premium” customer?
How does membership duration affect discounts?
What happens at boundary conditions?

Specification by Example Version

Feature: Loyalty-Based Discounts
  As a business owner
  I want to reward loyal customers with discounts
  So that they continue shopping with us

Background:
  Given the following discount tiers:
    | Membership Type | Years | Discount |
    | Standard       | 0-1   | 0%       |
    | Standard       | 2-4   | 5%       |
    | Standard       | 5+    | 10%      |
    | Premium        | 0-1   | 10%      |
    | Premium        | 2-4   | 15%      |
    | Premium        | 5+    | 20%      |

Scenario: New standard customer receives no discount
  Given a standard customer who joined 6 months ago
  When they purchase items totaling $100
  Then the discount applied is $0
  And the final price is $100

Scenario: Long-term premium customer receives maximum discount
  Given a premium customer who joined 6 years ago
  When they purchase items totaling $100
  Then the discount applied is $20
  And the final price is $80

Scenario: Customer upgraded to premium mid-tenure
  Given a customer who was standard for 3 years
  And then upgraded to premium 2 years ago
  When they purchase items totaling $100
  Then their total tenure is 5 years
  And they receive the premium 5+ year discount of 20%
  And the final price is $80

Scenario: Discount does not exceed purchase amount
  Given a premium customer with 6 years membership
  When they purchase items totaling $10
  Then the discount applied is $2
  And the final price is $8
  But the discount never exceeds the purchase total

Benefits:

Unambiguous: Exact percentages and conditions specified
Comprehensive: Edge cases (membership changes, boundary values) covered
Testable: Can be automated immediately
Validated: Examples verified by all three amigos

FitNesse: Wiki-Based Specifications

FitNesse turns wiki pages into executable tests, making specifications accessible to non-technical stakeholders.

FitNesse Example: Shopping Cart

Wiki Page (Readable by Business Stakeholders):

!3 Shopping Cart Calculation

Given these items in a cart:

|script                              |
|start|shopping cart                 |
|add item|Widget       |price|10.00 |
|add item|Gadget       |price|25.00 |
|add item|Doohickey    |price|15.00 |
|check   |subtotal            |50.00 |
|apply tax|8%                        |
|check   |tax amount          | 4.00 |
|check   |total               |54.00 |

What if we apply a coupon?

|script                              |
|start   |shopping cart              |
|add item|Widget       |price|10.00 |
|add item|Gadget       |price|25.00 |
|apply coupon|SAVE10                 |
|check   |subtotal            |50.00 |
|check   |discount            | 5.00 |
|check   |subtotal after discount|45.00|
|apply tax|8%                        |
|check   |total               |48.60 |

Java Implementation (Fixture):

public class ShoppingCart {
    private BigDecimal subtotal = BigDecimal.ZERO;
    private BigDecimal taxRate = BigDecimal.ZERO;
    private BigDecimal discount = BigDecimal.ZERO;

    public void start() {
        subtotal = BigDecimal.ZERO;
        discount = BigDecimal.ZERO;
    }

    public void addItem(String name, BigDecimal price) {
        subtotal = subtotal.add(price);
    }

    public BigDecimal subtotal() {
        return subtotal;
    }

    public void applyTaxPercent(BigDecimal percent) {
        taxRate = percent.divide(new BigDecimal("100"));
    }

    public BigDecimal taxAmount() {
        return subtotalAfterDiscount().multiply(taxRate)
            .setScale(2, RoundingMode.HALF_UP);
    }

    public void applyCoupon(String code) {
        if ("SAVE10".equals(code)) {
            discount = subtotal.multiply(new BigDecimal("0.10"));
        }
    }

    public BigDecimal discountAmount() {
        return discount;
    }

    public BigDecimal subtotalAfterDiscount() {
        return subtotal.subtract(discount);
    }

    public BigDecimal total() {
        return subtotalAfterDiscount().add(taxAmount());
    }
}

Execution:

Business stakeholders write examples in wiki
Developers implement fixtures
FitNesse runs examples against code
Green = spec matches implementation
Red = mismatch between spec and code

Concordion: HTML-Based Specifications

Concordion allows writing specifications in plain HTML or Markdown with special markup for executable assertions.

Concordion Example: User Registration

Markdown Specification:

# User Registration

When a new user registers, we need to validate their information.

## Valid Registration

When [Alice](- "#name") registers with email [alice@example.com](- "#email")
and password [SecurePass123!](- "#password"),
the registration [succeeds](- "?=register(#name, #email, #password)").

Her account is created with:
- Username: [alice](- "?=getUsername()")
- Email: [alice@example.com](- "?=getEmail()")
- Status: [ACTIVE](- "?=getStatus()")

## Invalid Email

When [Bob](- "#name") registers with email [not-an-email](- "#email")
and password [SecurePass123!](- "#password"),
the registration [fails](- "?=register(#name, #email, #password)").

The error message is: [Invalid email format](- "?=getErrorMessage()")

## Weak Password

When [Carol](- "#name") registers with email [carol@example.com](- "#email")
and password [123](- "#password"),
the registration [fails](- "?=register(#name, #email, #password)").

The error message contains: [Password must be at least 8 characters](- "?=getErrorMessage()")

Java Fixture:

@RunWith(ConcordionRunner.class)
public class UserRegistrationTest {
    private RegistrationService service = new RegistrationService();
    private RegistrationResult result;

    public boolean register(String name, String email, String password) {
        result = service.register(name, email, password);
        return result.isSuccess();
    }

    public String getUsername() {
        return result.getUser().getUsername();
    }

    public String getEmail() {
        return result.getUser().getEmail();
    }

    public String getStatus() {
        return result.getUser().getStatus().toString();
    }

    public String getErrorMessage() {
        return result.getErrorMessage();
    }
}

Output: Beautiful HTML report showing which examples passed/failed, readable by stakeholders.

Living Documentation Benefits

Traditional documentation becomes outdated. Living documentation evolves with the codebase:

Traditional Docs:

Last updated: March 2024
Status: Unknown if still accurate
Tests: Separate from documentation

Living Documentation:

Last run: 2 minutes ago (CI build #4523)
Status: All 127 examples passing
Tests: Documentation IS the tests

Documentation as a Safety Net

When developers change code:

Without Living Docs:

Change code
Hope tests catch issues
Manually check docs
Maybe update docs (probably not)
Deploy and pray

With Living Docs:

Change code
Examples automatically fail if behavior changed
Update examples to match new behavior
Documentation automatically current
Deploy confidently

Specification by Example vs. BDD

SbE and Behavior-Driven Development (BDD) are closely related:

BDD is a software development methodology that emphasizes collaboration and uses examples. It’s the “how” of building software.

SbE is a technique for capturing requirements using examples. It’s the “what” we’re building.

Relationship:

BDD uses SbE as its core practice
You can practice SbE without full BDD adoption
BDD adds additional practices (ubiquitous language, outside-in development)

Example:

Feature: Password Reset                    # SbE: What we're building

Scenario: User requests password reset     # SbE: Example specification
  Given a user with email "alice@example.com"
  When they request a password reset
  Then they receive a reset email
  And the email contains a valid reset link

The Gherkin syntax serves both as SbE documentation and BDD test automation.

Real-World Implementation: Banking Transfer Feature

Traditional Approach

Requirements Doc:

The system shall allow users to transfer money between accounts.
Transfers must validate sufficient funds.
Daily transfer limits apply.

Separate Test Cases:

TC-001: Test successful transfer
TC-002: Test insufficient funds
TC-003: Test daily limit exceeded

Result: Disconnect between requirements, tests, and implementation.

Specification by Example Approach

Collaborative Example Workshop:

Product Owner: “We want to allow fund transfers up to $10,000 per day.” Developer: “What happens if someone tries to transfer $9,000 twice in one day?” Tester: “What about transfers across different currencies?” PO: “Second transfer should fail. For now, only USD.” Developer: “Do pending transfers count toward the limit?” Tester: “What if someone transfers exactly $10,000.00 versus $10,000.01?”

Resulting Specification:

Feature: Account Transfers with Daily Limits

Background:
  Given the daily transfer limit is $10,000 USD
  And limits reset at midnight Pacific Time

Scenario: Successful transfer within limit
  Given Alice has $5,000 in her checking account
  And she has made no transfers today
  When she transfers $1,000 to Bob's account
  Then the transfer succeeds
  And Alice's balance is $4,000
  And her remaining daily limit is $9,000

Scenario: Transfer rejected for insufficient funds
  Given Alice has $500 in her checking account
  When she attempts to transfer $1,000 to Bob
  Then the transfer fails
  And the error is "Insufficient funds"
  And Alice's balance remains $500

Scenario: Multiple transfers approaching daily limit
  Given Alice has $20,000 in her account
  And she has made no transfers today
  When she transfers $6,000 to Bob
  And she transfers $3,000 to Carol
  Then both transfers succeed
  And her remaining daily limit is $1,000

Scenario: Transfer rejected when exceeding daily limit
  Given Alice has $20,000 in her account
  And she has already transferred $9,500 today
  When she attempts to transfer $1,000 to Bob
  Then the transfer fails
  And the error is "Daily transfer limit exceeded"
  And her remaining daily limit is $500

Scenario: Boundary test at exact limit
  Given Alice has $15,000 in her account
  And she has already transferred $9,999.99 today
  When she transfers $0.01 to Bob
  Then the transfer succeeds
  And her remaining daily limit is $0.00

Scenario: Pending transfers count toward limit
  Given Alice has $15,000 in her account
  And she has a pending transfer of $8,000
  When she attempts to transfer $3,000 to Bob
  Then the transfer fails
  And the error is "Daily transfer limit exceeded (including pending)"

Outcomes:

Clarity: Everyone knows exactly how limits work
Edge cases: Boundary values (e.g., $9,999.99 + $0.01) identified
Validation: Business logic codified in executable tests
Documentation: Always reflects current system behavior

Common Pitfalls and How to Avoid Them

1. Writing Implementation Details in Examples

Bad:

Given the database table "users" contains a row with id=123
And the "premium_member" column is true
And the "membership_date" is "2019-01-01"

Good:

Given a premium customer who joined on January 1, 2019

Fix: Use business language, hide implementation details.

2. Too Many Examples

Problem: 50 scenarios for one feature, mostly redundant.

Fix:

Focus on key examples illustrating different behaviors
Use data tables for combinatorial testing
Extract common patterns into Background section

3. Examples Without Assertions

Bad:

When the user clicks the submit button
Then the form is submitted

Good:

When the user clicks the submit button
Then they see a confirmation message "Thank you for your submission"
And they receive a confirmation email
And the submission appears in the admin panel

Fix: Include observable outcomes, not just actions.

4. Technical Stakeholders Only

Problem: Developers write all examples in isolation.

Fix: Always include three amigos (business, dev, QA) in workshops.

5. Treating Examples as Just Tests

Problem: Examples written after code is done, just for testing.

Fix: Examples drive development—write them BEFORE coding (like TDD/BDD).

Tools Comparison

Tool	Language	Format	Learning Curve	Best For
Cucumber	Many	Gherkin	Medium	Teams familiar with BDD
FitNesse	Java, .NET	Wiki tables	Low	Non-technical stakeholders
Concordion	Java, .NET	HTML/Markdown	Medium	Documentation-focused teams
SpecFlow	.NET/C#	Gherkin	Medium	.NET shops doing BDD
Gauge	Many	Markdown	Low	Teams wanting readable specs
Robot Framework	Python	Keyword-driven	Medium	Test automation teams

Getting Started with Specification by Example

Week 1: Pilot Feature

Choose one small feature
Run three amigos workshop
Write 3-5 key examples
Automate with chosen tool
Reflect on what worked

Week 2-4: Expand

Add SbE to new features
Refine workshop process
Build fixture library
Train team members

Month 2-3: Mature Practice

Retrospect existing features with examples
Integrate into CI/CD
Measure documentation coverage
Celebrate living docs success

Conclusion: Examples as the Source of Truth

Specification by Example transforms requirements from static documents into living, validated artifacts. By making examples:

Concrete: No ambiguity about expected behavior
Collaborative: Business, dev, and QA align early
Executable: Examples verify implementation
Living: Documentation evolves with code

The result is software that provably does what stakeholders expect, documented in a format everyone understands, validated continuously, and always up-to-date.

Traditional requirements ask “Did we build it right?” Specification by Example ensures we build the right thing, the right way, with proof that it works.

Start small, collaborate actively, and let examples drive your development. The clarity and confidence they bring will transform how your team builds software.