Test Definitions

Test definitions are the core of HttpProbe. They define your API tests in a declarative way, allowing you to focus on what you want to test rather than how to test it. Test definitions support hooks to run other tests before and after test execution, creating powerful, reusable test workflows.

Table of contents

  1. Basic Structure
  2. File Formats
  3. Top-Level Elements
    1. Name and Description
    2. Hooks
    3. Variables
    4. Suites
      1. Suite Configuration
  4. Test Cases
    1. Request
      1. Request Body
    2. Assertions
  5. Complete Example

Basic Structure

A test definition file contains the following main sections:

name: "Test Definition Name"
description: "Description of the test definition"
variables:
  # Variables to use in the test cases
  variable_name:
    type: string
    value: "variable value"

# Hooks to run before and after test execution
before_all:
  - "path/to/setup.test.yaml"
after_all:
  - "path/to/cleanup.test.yaml"
before_each:
  - "path/to/auth.test.yaml"
after_each:
  - "path/to/logging.test.yaml"

suites:
  # Test suites
  - name: "Suite Name"
    cases:
      # Test cases
      - title: "Test Case Title"
        request:
          # Request details
          method: GET
          url: "https://api.example.com/endpoint"
          headers:
            - key: "Header-Name"
              value: "Header Value"
          body:
            type: json
            data: null  # or JSON object/array
          assertions:
            # Assertions to validate the response
            status: 200
            # Other assertions...
          export:
            # Export values from the response for use in subsequent tests
            body:
              - path: "$.token"
                as: "auth_token"

File Formats

HttpProbe supports test definitions in two formats:

  • YAML (.yaml, .yml) - Recommended for readability
  • JSON (.json) - Useful for programmatic generation

Top-Level Elements

Name and Description

name: "User API Tests"
description: "Tests for the user management API endpoints"

Hooks

Hooks allow you to run other test definitions at specific points in the test lifecycle:

# Run before any test suites in this definition
before_all:
  - "setup/initialize.test.yaml"

# Run after all test suites in this definition have completed
after_all:
  - "cleanup/teardown.test.yaml"

# Run before each test suite in this definition
before_each:
  - "common/auth.test.yaml"

# Run after each test suite in this definition
after_each:
  - "common/logging.test.yaml"

Hooks are useful for:

  • Setting up test data before tests run
  • Cleaning up resources after tests complete
  • Obtaining authentication tokens before each test suite
  • Logging test results after each test suite

Variables exported by hooks are available to subsequent tests. For example, if a before_each hook exports an authentication token, that token is available in the test suite that follows.

The name is used in test reports and logs to identify the test definition. The description provides additional information about the purpose of the tests.

Variables

Variables allow you to define reusable values that can be referenced in your test cases.

variables:
  base_url:
    type: string
    value: "https://api.example.com/v1"
  admin_token:
    type: string
    value: "${env:ADMIN_TOKEN}"  # From environment variable
  request_id:
    type: string
    value: "${random(16)}"  # Generated random string

Variables can be referenced in URLs, headers, and request bodies using the syntax ${variable_name}.

Suites

Test suites group related test cases together.

suites:
  - name: "User Management"
    config:
      concurrent: true  # Run test cases in this suite concurrently
    cases:
      # Test cases for user management
  - name: "Authentication"
    cases:
      # Test cases for authentication

Suite Configuration

The optional config section allows you to configure suite-specific behavior:

config:
  concurrent: true  # Run test cases in this suite concurrently

Available configuration options:

  • concurrent: When set to true, test cases in the suite will run concurrently instead of sequentially. This can significantly improve performance when test cases are independent, but should be used carefully if test cases depend on each other or export variables that other test cases need. See the Concurrency documentation for more details.

Test Cases

Each test case represents a single API request with its assertions.

- title: "Get User Profile"
  request:
    method: GET
    url: "${base_url}/users/123"
    headers:
      - key: Authorization
        value: Bearer ${admin_token}
      - key: Accept
        value: application/json
    body:
      type: json
      data: null
    assertions:
      status: 200
      headers:
        content-type: "application/json; charset=utf-8"
      body:
        "$.id": 123
        "$.email": "user@example.com"

Request

The request section defines the HTTP request to be made:

  • method: HTTP method (GET, POST, PUT, DELETE, etc.)
  • url: The endpoint URL (can include variables)
  • headers: List of HTTP headers to include
  • body: Request body (if applicable)

Request Body

For request bodies, you must specify:

  • type: The content type (currently supported: json, text)
  • data: The actual body content

For JSON bodies, you can specify the data in several ways:

# Inline JSON object
body:
  type: json
  data:
    name: "John Doe"
    email: "john@example.com"

# JSON string with variables
body:
  type: json
  data: |
    {
      "name": "John Doe",
      "email": "john@example.com",
      "token": "${api_token}"
    }

# No body
body:
  type: json
  data: null

Assertions

The assertions section defines the expected response:

assertions:
  # Status code assertion
  status: 200
  
  # Header assertions
  headers:
    content-type: "application/json; charset=utf-8"
    cache-control: "no-cache"
  
  # Body assertions using JSONPath
  body:
    "$.id": 123
    "$.name": "John Doe"
    "$.roles[0]": "admin"

  # Schema assertion
  schema: |
    {
      "type": "object",
      "required": ["id", "name", "email"],
      "properties": {
        "id": { "type": "integer" },
        "name": { "type": "string" },
        "email": { "type": "string", "format": "email" }
      }
    }

See the Assertions page for detailed information on all available assertion types.

Complete Example

Here’s a complete example of a test definition file:

name: "User API Tests"
description: "Tests for user management API endpoints"
variables:
  base_url:
    type: string
    value: "https://api.example.com/v1"
  admin_token:
    type: string
    value: "${env:ADMIN_TOKEN}"

suites:
  - name: "User Management"
    cases:
      - title: "List Users"
        request:
          method: GET
          url: "${base_url}/users"
          headers:
            - key: Authorization
              value: Bearer ${admin_token}
            - key: Accept
              value: application/json
          body:
            type: json
            data: null
          assertions:
            status: 200
            headers:
              content-type: "application/json; charset=utf-8"
            body:
              "$.users": "length > 0"

      - title: "Get Specific User"
        request:
          method: GET
          url: "${base_url}/users/123"
          headers:
            - key: Authorization
              value: Bearer ${admin_token}
            - key: Accept
              value: application/json
          body:
            type: json
            data: null
          assertions:
            status: 200
            body:
              "$.id": 123
              "$.name": "John Doe"
              "$.email": "john@example.com"