Web-Ingestion Policy Quickstart

June 12, 2026

4 min read

Developer Guide

Overview

Agent Permission gives crawlers and AI data pipelines a checkpoint before external web content is fetched or stored. Use the CLI or API to get an allow, deny, or escalate decision with source evidence and a signed receipt.

CLI Quickstart

Use the public CLI when your workflow starts in a terminal, CI job, or AI coding-agent setup.

Give This To Your Agent

For one personal Codex setup, install once at user scope. Codex installs default to that scope so new repositories can reuse the same runner instead of looking for a project-local preflight file.

npx -y agent-permission@latest install --adapter codex

For repository-owned instructions or exact Claude, ChatGPT, and OpenClaw install commands, see the agent install guide.

Configure your API key

node "$HOME/.config/agent-permission/agent-permission.mjs" config set api-key <YOUR_API_KEY>
node "$HOME/.config/agent-permission/agent-permission.mjs" config doctor

Run a permission check

node "$HOME/.config/agent-permission/agent-permission.mjs" check https://example.com/blog-post --action summarize --exit-code

Check URL Permission

Evaluate a web-ingestion purpose—crawl, summarize, extract, or train—against a target URL before the pipeline fetches or stores its content.

POST /v1/checkapplication/json

POST https://agent-permission-api-788656703262.us-central1.run.app/v1/check
Headers:
  Content-Type: application/json
  Authorization: Bearer <YOUR_API_KEY>

Request Payload Parameters

url (string, required): The target URL path to inspect.
action (string, required): The action intended by the agent. Accepted values are crawl, summarize, extract, train, transact, post.
mode (string, optional): Enforcement mode. In strict mode, underspecified summarize and extract actions escalate instead of defaulting to allow.

Compatibility boundary: post and transact remain accepted values, but their results reflect only publisher-declared URL signals. They are not user, account, spending, payload, or organizational authorization.

Request JSON Sample

{
  "url": "https://example.com/blog-post",
  "action": "summarize"
}

Response Payload

A successful response contains a consolidated decision, supporting reasons, the policy sources used, and a service-signed receipt that records the check.

Response JSON Sample

{
  "id": "chk_1234abcd",
  "decision": "allow",
  "confidence": "high",
  "risk": "low",
  "reasons": [
    "robots.txt explicitly allows this path"
  ],
  "sources": [
    {
      "type": "robots_txt",
      "url": "https://example.com/robots.txt",
      "fetched_at": "2026-07-18T12:00:00.000Z",
      "evidence": "User-agent: *\nAllow: /"
    }
  ],
  "receipt": {
    "receipt_id": "rcpt_1234abcd",
    "check_id": "chk_1234abcd",
    "url_hash": "sha256-of-the-requested-url",
    "action": "summarize",
    "decision": "allow",
    "source_hashes": ["sha256-of-robots-txt"],
    "issued_at": "2026-07-18T12:00:00.000Z",
    "account_id": "account-reference",
    "signature_alg": "HMAC-SHA256",
    "signature": "service-signature"
  }
}

Decision Types

allow: The available publisher signals support continuing under the selected policy mode.
deny: The resource explicitly blocks the request (e.g. user-agent disallow).
escalate: Underspecified metadata terms; human-in-the-loop validation recommended before acting.

Receipt Validation

Receipts use an HMAC service signature. Validate a complete receipt with POST /v1/verify-receipt or the CLI verify-receipt command. Validation is performed by Agent Permission with the service signing key; it is not independent offline verification. See Trust & Data Handling for the exact boundary.

Integration Examples

Python

import requests

def verify_agent_preflight(target_url, intent):
    res = requests.post(
        "https://agent-permission-api-788656703262.us-central1.run.app/v1/check",
        headers={
            "Content-Type": "application/json",
            "Authorization": "Bearer YOUR_API_KEY"
        },
        json={
            "url": target_url,
            "action": intent
        }
    )
    data = res.json()
    return data.get("decision") == "allow"

Node.js / TypeScript

async function canAgentExecute(url: string, action: string): Promise<boolean> {
  const response = await fetch('https://agent-permission-api-788656703262.us-central1.run.app/v1/check', {
    method: 'POST',
    headers: {
      'Content-Type': 'application/json',
      'Authorization': 'Bearer YOUR_API_KEY'
    },
    body: JSON.stringify({ url, action })
  });
  
  const result = await response.json();
  return result.decision === 'allow';
}

Add a Policy Check Before Ingestion

Check publisher signals before a crawler, RAG loader, extraction job, or training pipeline stores web content.

Install the Ingestion Gate Create Free Ingestion Key