Http Api Test Runner

SKILL.md

---
name: http-api-test-runner
description: Build reusable HTTP API test artifacts from user-provided endpoints, authentication, request data, expected results, and validation rules. Use this skill when the user wants to create .http files, run HTTP/REST API checks, replay browser or curl requests, validate JSON fields or response markers, compare expected vs actual responses, or generate formatted PASS/FAIL API test reports.
---

# HTTP API Test Runner

Use this skill to turn ad hoc HTTP endpoint checks into repeatable `.http` cases and a runnable formatter script.

The user provides endpoints, authentication, request data, scenarios, and expected results. Produce artifacts that can be rerun and that print a readable PASS/FAIL report instead of raw response dumps.

## When To Use

Use this skill when the user asks to:

- Write `.http` tests for an HTTP/REST API.
- Replay browser Network requests or cURL commands.
- Test endpoints with cookies, bearer tokens, custom headers, request bodies, or no authentication.
- Validate status codes, JSON paths, response markers, list membership, error codes, or login behavior.
- Run several API cases and summarize which cases passed or failed.
- Create a formatted script for API response inspection.

## Collect Inputs

Ask only for missing information. Do not ask for values already provided.

Collect:

- **Base URL / host**: for example `https://api.example.test`.
- **Endpoints**: full URLs or path templates.
- **HTTP method**: default to `GET` when unspecified.
- **Authentication**: cookie, bearer token, custom headers, or unauthenticated.
- **Request data**: query params, path params, JSON body, form body, content type.
- **Cases**: positive, negative, auth failure, validation failure, boundary cases.
- **Expected results**: exact status code, JSON field, marker, list entry, or error behavior.
- **Output needs**: short summary, detailed per-case report, key fields, raw response save path.

For cookie-based tests, tell the user to copy the complete request cookie from a successful browser Network request:

```text
Please copy the full `Cookie:` request header from the browser Network panel for a successful target request. Do not reconstruct cookies only from the Application/Storage panel, because HttpOnly, path, domain, and host-bound cookies may be missed.
```

Use the user's language for generated file comments and final instructions.

## Artifact Pattern

Create two files by default, near the user's existing test or technical docs unless they specify another location:

```text
<feature>.api-tests.http
<feature>.api-verify.sh
```

If the project already uses another naming style, follow it.

The `.http` file is the source of truth. It should contain:

- Variables such as `@host`, `@resourceId`, `@cookie`, `@token`.
- One request per case.
- Human-readable case titles using `###`.
- Explicit `expect.*` comments for assertions.
- No real secrets by default.

The formatter script should:

- Read the `.http` file.
- Resolve variables such as `{{host}}`, `{{resourceId}}`, `{{cookie}}`.
- Accept secrets from environment variables.
- Execute requests with timeout handling.
- Parse JSON responses when possible.
- Print formatted PASS/FAIL/SKIP output.
- Exit non-zero if any non-skipped case fails.

Prefer `bash` + `curl` for portability. For JSON parsing, prefer `python3` embedded snippets over mandatory `jq`, unless the repo already depends on `jq`.

## Secret Handling

Default behavior:

- Do not write real cookies, service tokens, bearer tokens, session IDs, user IDs, or internal credentials into skill files or generated committed artifacts.
- Use placeholders in `.http`, such as `@cookie = <set via COOKIE>` or `@token = <set via AUTH_TOKEN>`.
- Read secrets from environment variables in scripts, such as `COOKIE`, `AUTH_TOKEN`, `ADMIN_COOKIE`.
- If secrets are missing, skip authenticated cases with a clear `[SKIP]` message instead of failing parsing.

If the user explicitly asks to embed cookies or tokens for local convenience:

- Do it only in a local test script, never in publishable docs.
- Add a warning near the top of the file:

```text
This script temporarily embeds real credentials for local testing only. Do not commit, share, or publish it.
```

Before publishing or committing, check for secrets and internal identifiers with searches such as:

```bash
rg -n "password|secret|session_id|auth_token|access_token|refresh_token" <artifact-dir>
rg -n "Authorization: Bearer [A-Za-z0-9._-]+|C[o]okie: [A-Za-z0-9_%-]+=" <artifact-dir>
```

## Assertion Contract

Prefer explicit assertions in `.http` comments. Title-based inference is allowed only as a fallback.

Supported assertion comments:

```http
# expect.status = 200
# expect.errno = 0
# expect.errno_not = 0
# expect.contains = TARGET_MARKER
# expect.not_contains = ERROR_MARKER
# expect.json_path = data.status
# expect.equals = enabled
# expect.exists = data.items[0].id
# expect.absent = data.deprecatedField
# expect.list_contains = data.items[].code:TARGET_CODE
# expect.list_not_contains = data.items[].code:TARGET_CODE
# expect.save = tmp/case-1.response.json
# output.fields = code,message,data.id,data.status
```

Rules:

- `expect.status` validates HTTP status.
- `expect.errno` and `expect.errno_not` apply to APIs that expose an `errno`-style field.
- `expect.contains` and `expect.not_contains` search the response text.
- `expect.json_path` + `expect.equals` validates one JSON field.
- `expect.exists` and `expect.absent` validate JSON path presence.
- `expect.list_contains` validates a projected list path contains a value.
- `expect.save` optionally saves raw response for debugging.
- `output.fields` controls key fields printed in the formatted report.

If an API uses `code`, `status`, `error`, or another convention instead of `errno`, adapt the generated assertions and output names to the API.

## .http Template

Use this structure for common GET and POST cases:

```http
### <feature> API tests
#
# Usage:
# 1. Confirm the target environment has the code/config under test.
# 2. Export secrets when needed:
#    COOKIE='full Cookie header' AUTH_TOKEN='token value' bash './<feature>.api-verify.sh'
# 3. Run the formatter script to see PASS/FAIL output.

@host = https://api.example.test
@resourceId = sample-123
@cookie = <set via COOKIE>
@token = <set via AUTH_TOKEN>

### Case-1 GET positive: resource should be enabled
# auth = cookie
# expect.status = 200
# expect.json_path = data.status
# expect.equals = enabled
# output.fields = code,message,data.id,data.status
GET {{host}}/v1/resources/{{resourceId}}
Cookie: {{cookie}}

### Case-2 GET negative: missing resource should return error
# expect.status = 404
# expect.contains = not_found
GET {{host}}/v1/resources/missing-id

### Case-3 POST positive: create should return success
# auth = bearer
# expect.status = 200
# expect.json_path = code
# expect.equals = 0
POST {{host}}/v1/resources
Content-Type: application/json
Authorization: Bearer {{token}}

{
  "name": "sample"
}
```

## Script Output

The formatter script should print enough information to diagnose failures without opening raw JSON:

```text
[PASS] Case-1 GET positive: resource should be enabled
  URL: https://api.example.test/v1/resources/sample-123
  method: GET
  expectation: json_path data.status == enabled
  HTTP=200 cost=123ms
  key fields: code=0,message='',data.id='sample-123',data.status='enabled'

[FAIL] Case-2 GET negative: missing resource should return error
  URL: https://api.example.test/v1/resources/missing-id
  method: GET
  expectation: contains not_found
  HTTP=200 cost=98ms
  reason: expected HTTP status 404 but got 200

Summary: PASS=1 FAIL=1 SKIP=0
```

For non-JSON responses, print status, headers or content type when useful, response size, and text marker results.

## Implementation Guidance

When generating the formatter script:

- Keep case definitions synchronized with the `.http` file.
- Support `GET`, `POST`, `PUT`, `PATCH`, and `DELETE` when present.
- Preserve request headers from the `.http` case.
- Preserve JSON or form bodies exactly enough for repeatable testing.
- Use `curl --silent --show-error --location --max-time 20`.
- Separate transport failures, HTTP status failures, JSON parse failures, and assertion failures.
- Print the case containing a matching marker or failed list element when validating list responses.
- Save raw responses only when requested or helpful for debugging.

## Running Tests

After creating artifacts:

1. Syntax-check the script:

```bash
bash -n './<feature>.api-verify.sh'
```

2. Run without secrets to verify parsing and expected SKIP behavior:

```bash
bash './<feature>.api-verify.sh'
```

3. Run with user-provided secrets:

```bash
COOKIE='full Cookie header' AUTH_TOKEN='token value' bash './<feature>.api-verify.sh'
```

4. Summarize:

- Which cases passed, failed, or skipped.
- Whether failures look like auth, request shape, environment config, data setup, or code behavior.
- Which fields or raw response files are most relevant.

## Debugging Rules

If authenticated cases fail or return login errors:

- Ask for the full `Cookie:` or `Authorization:` request header from a successful Network request.
- Check host, path, and environment mismatch.
- Check missing app-specific auth headers or CSRF headers.

If a positive case does not contain the expected marker:

- Print the exact assertion and the inspected JSON path or response section.
- Re-check feature switch/config, user/account conditions, region, state, type, and time-window requirements.

If list validation appears to pass but the displayed item looks unrelated:

- Display the exact list item that matched.
- Prefer explicit JSON path assertions over broad text search when the response structure is stable.

## Completion Checklist

Before finishing:

- `.http` and script contain the same cases.
- Authenticated cases skip cleanly when secrets are missing.
- Script runs with provided secrets and produces formatted PASS/FAIL output when the network is reachable.
- Raw secrets are not embedded unless the user explicitly requested local-only embedding.
- Publishable skill files contain no internal hostnames, real identifiers, company-specific markers, or credentials.
- Final response tells the user how to run the script and how to interpret the output.