Assertions

Assertions are the heart of API testing with HttpProbe. They allow you to validate that responses meet your expectations by checking status codes, headers, response bodies, and schemas.

Table of contents

1. Introduction
2. Types of Assertions
   1. Status Code Assertions
   2. Header Assertions
   3. Body Assertions
      1. JSONPath Syntax
      2. Value Comparisons
      3. Type Validation
   4. Schema Assertions
   5. GraphQL Assertions
3. Handling Assertion Failures
4. Combining Assertion Types
5. Best Practices
6. Examples
   1. Testing a User API
   2. Testing Authentication
   3. Testing Pagination

---

Introduction

Assertions in HttpProbe are defined in the assertions section of a test case. They specify the expected properties of the HTTP response. When a test is executed, the actual response is compared against these assertions to determine if the test passes or fails.

assertions:
  status: 200
  headers:
    content-type: "application/json"
  body:
    "$.status": "success"
    "$.user.id": 123

Types of Assertions

HttpProbe supports five main types of assertions:

1. Status code assertions
2. Header assertions
3. Body assertions
4. Schema assertions
5. GraphQL assertions

Status Code Assertions

Status code assertions verify that the HTTP response has the expected status code:

assertions:
  status: 200  # Expects HTTP 200 OK

You can assert any standard HTTP status code:

status: 201  # Created
status: 400  # Bad Request
status: 404  # Not Found
status: 500  # Internal Server Error

Header Assertions

Header assertions validate that the response contains specific HTTP headers with expected values:

assertions:
  headers:
    content-type: "application/json; charset=utf-8"
    cache-control: "no-cache"

Header names are case-insensitive, matching HTTP standards.

Body Assertions

Body assertions verify the content of the response body using JSONPath expressions:

assertions:
  body:
    "$.id": 123
    "$.name": "John Doe"
    "$.enabled": true

JSONPath Syntax

JSONPath is a query language for JSON, similar to how XPath is for XML. HttpProbe uses JSONPath to extract values from JSON responses for validation.

Common JSONPath expressions:

Expression	Description	Example
$	Root object	$ (the entire document)
$.property	Child property	$.name
$.property.nested	Nested property	$.user.email
$[index]	Array index	$[0] (first item)
$.array[index]	Array property with index	$.users[0] (first user)
$.array[*]	All items in array	$.users[*].name (all user names)
$.*.property	Property in all objects	$.*.id (all IDs)

Value Comparisons

Body assertions can use various comparison operators:

body:
  # Equality (default)
  "$.status": "success"
  
  # Numeric comparisons
  "$.count": "> 5"       # Greater than
  "$.price": ">= 10.5"   # Greater than or equal
  "$.stock": "< 100"     # Less than
  "$.rating": "<= 5"     # Less than or equal
  
  # String contains
  "$.message": "contains error"  # String contains 'error'
  
  # Array length
  "$.items": "length 10"      # Exactly 10 items
  "$.users": "length > 0"     # At least 1 item
  "$.roles": "length <= 5"    # At most 5 items

Type Validation

HttpProbe automatically checks that the value type matches the expected type:

body:
  "$.id": 123              # Expects a number
  "$.name": "John"         # Expects a string
  "$.active": true         # Expects a boolean
  "$.settings": null       # Expects null
  "$.tags": ["tag1", "tag2"] # Expects an array

Schema Assertions

Schema assertions validate the entire response structure using JSON Schema:

assertions:
  schema: |
    {
      "type": "object",
      "required": ["id", "name", "email"],
      "properties": {
        "id": { "type": "integer" },
        "name": { "type": "string" },
        "email": { "type": "string", "format": "email" },
        "age": { "type": "integer", "minimum": 18 },
        "tags": {
          "type": "array",
          "items": { "type": "string" }
        }
      }
    }

JSON Schema allows you to validate:

• Required properties
• Property types
• String formats (email, date-time, etc.)
• Numeric ranges
• Array contents
• And much more

GraphQL Assertions

GraphQL assertions are designed for APIs that return HTTP 200 for both success and error responses. They validate the structure of GraphQL responses, which contain data and errors fields.

assertions:
  status: 200
  graphql:
    no_errors: true
    data:
      "$.user.id": "123"
      "$.user.email": "/.+@.+/"
    data_schema: |
      { "type": "object", "required": ["user"] }

Available GraphQL assertion options:

Option	Description
no_errors	true to assert no errors, false to assert errors are present
data	JSONPath assertions relative to response.data
errors	List of expected errors matched by message and extensions.*
partial_data	false to fail if both data and errors coexist
data_schema	JSON Schema to validate response.data against

Data paths use JSONPath expressions relative to response.data, so $.user.id refers to response.data.user.id. Data assertions support:

• Variable interpolation — "${user_id}" resolves exported and defined variables
• Comparison operators — "> 5", ">= 10", "contains error" (same as body assertions)
• Regex patterns — "/.+@.+/" (wrapped in forward slashes)
• Length checks — "length > 0", "length 10", "length <= 5"

For full details, see the GraphQL Testing page.

Handling Assertion Failures

When assertions fail, HttpProbe provides detailed error messages to help you understand what went wrong:

Test case "Get User" FAILED:
  - Expected status code 200, got 404
  - Header 'content-type' not found in response
  - JSONPath '$.user.email' not found in response body
  - Schema validation error: required property 'email' not present

The failure messages include:

• The expected and actual values
• The path to the failed property
• Specific details about schema validation errors

Combining Assertion Types

You can combine multiple assertion types in a single test case:

assertions:
  status: 200
  headers:
    content-type: "application/json"
  body:
    "$.success": true
    "$.data.count": "> 0"
  schema: |
    {
      "type": "object",
      "required": ["success", "data"],
      "properties": {
        "success": { "type": "boolean" },
        "data": {
          "type": "object",
          "required": ["count", "items"],
          "properties": {
            "count": { "type": "integer" },
            "items": { "type": "array" }
          }
        }
      }
    }

Best Practices

1. Start with status code assertions as a baseline check
2. Validate critical headers like content-type and authentication tokens
3. Check key properties in the response body that are essential to your test case
4. Use schema validation for comprehensive structure validation
5. Focus on business-critical assertions rather than checking every property
6. Use specific JSONPath expressions to target exactly what you want to validate
7. Include detailed error messages to make debugging easier when tests fail

Examples

Testing a User API

cases:
  - title: "Get User Profile"
    request:
      method: GET
      url: "${base_url}/users/123"
      headers:
        - key: Authorization
          value: Bearer ${token}
      body:
        type: json
        data: null
    assertions:
      status: 200
      headers:
        content-type: "application/json; charset=utf-8"
      body:
        "$.id": 123
        "$.name": "John Doe"
        "$.email": "john@example.com"
        "$.subscription.active": true
        "$.roles": "length > 0"

Testing Authentication

cases:
  - title: "Login with Valid Credentials"
    request:
      method: POST
      url: "${base_url}/auth/login"
      headers:
        - key: Content-Type
          value: application/json
      body:
        type: json
        data: |
          {
            "username": "validuser",
            "password": "validpassword"
          }
    assertions:
      status: 200
      body:
        "$.success": true
        "$.token": "contains eyJ"  # JWT tokens start with eyJ
      schema: |
        {
          "type": "object",
          "required": ["success", "token", "user"],
          "properties": {
            "success": { "type": "boolean" },
            "token": { "type": "string" },
            "user": {
              "type": "object",
              "required": ["id", "username"],
              "properties": {
                "id": { "type": "integer" },
                "username": { "type": "string" }
              }
            }
          }
        }
        
  - title: "Login with Invalid Credentials"
    request:
      method: POST
      url: "${base_url}/auth/login"
      headers:
        - key: Content-Type
          value: application/json
      body:
        type: json
        data: |
          {
            "username": "validuser",
            "password": "wrongpassword"
          }
    assertions:
      status: 401
      body:
        "$.success": false
        "$.message": "contains Invalid credentials"

Testing Pagination

cases:
  - title: "List Items with Pagination"
    request:
      method: GET
      url: "${base_url}/items?page=1&limit=10"
      headers:
        - key: Accept
          value: application/json
      body:
        type: json
        data: null
    assertions:
      status: 200
      body:
        "$.data": "length <= 10"
        "$.meta.page": 1
        "$.meta.limit": 10
        "$.meta.total": "> 0"