Testing MCP Tool Annotations: Validate readOnlyHint, destructiveHint, and openWorldHint for ChatGPT and Claude (July 2026)

Tool annotations are small fields with a large review footprint. They tell MCP hosts whether a tool reads data, changes state, can be retried, or reaches outside a closed system. That affects user confirmation prompts in ChatGPT and Claude, and it affects whether a submitted app gets through review.

This guide explains the current annotation model, how ChatGPT and Claude use it, how to choose the right values for common tool patterns, and how to test those values so a teammate cannot accidentally ship a risky tool with safe-looking metadata.

TL;DR: Set readOnlyHint, destructiveHint, idempotentHint, and openWorldHint explicitly on every MCP tool. ChatGPT plugin submission requires readOnlyHint, destructiveHint, and openWorldHint; the MCP spec also defines idempotentHint and title. Treat annotations as reviewed metadata, not comments. Test them with tools/list in CI, then rescan and publish a new ChatGPT plugin version when annotations change.

What Changed Since the Earlier Version

The rules around tool annotations have become clearer since the first version of this post:

• The current MCP schema defines title, readOnlyHint, destructiveHint, idempotentHint, and openWorldHint under ToolAnnotations. The boolean hints are still optional in the base protocol, but their defaults are intentionally conservative.
• OpenAI’s Apps SDK docs now mark readOnlyHint, destructiveHint, and openWorldHint as required for ChatGPT app tool descriptors. The same docs say the hints shape how ChatGPT frames the tool call to the user, while the server still needs to enforce auth and policy.
• OpenAI now submits and publishes Apps as Plugins. A plugin can contain skills, an MCP-backed app, or both. The plugin portal scans your MCP endpoint, stores the discovered metadata snapshot, and requires a new reviewed version when you change tool annotations.
• Anthropic’s current MCP connector docs let Claude connect to remote MCP servers from the Messages API with allowlists, denylists, and per-tool configuration. That makes clean tool metadata useful beyond a directory submission.
• The MCP project has also explained annotations as a “risk vocabulary.” That framing is useful: annotations help a host describe the risk of a tool call, but hosts should not treat them as proof when the server is untrusted.

The practical guidance is simple: write the annotations as if a reviewer, a host, and a future teammate will all depend on them. Then test that your server actually exposes what you think it exposes.

The Annotation Fields

The current MCP spec treats tool annotations as hints. They describe behavior, but they do not replace authorization, confirmation, rate limits, input validation, or server-side policy checks.

Field	Default in MCP	Meaning
title	none	Human-readable tool name
readOnlyHint	false	The tool does not modify its environment
destructiveHint	true	A write tool may delete, overwrite, remove, revoke, or otherwise destroy data
idempotentHint	false	Repeating the same call with the same arguments has no additional effect
openWorldHint	true	The tool may interact with external entities outside a closed domain

Those defaults are cautious. If you omit annotations, a host should assume the tool writes data, might be destructive, is not safe to retry, and reaches an open world. That is good for safety, but bad for product quality. A read-only search tool with no annotations can look like a dangerous write tool, which creates extra approvals and review friction.

Set every boolean explicitly.

How to Choose the Right Values

Start with side effects, not the tool name. A function called get_invoice may still mutate state if it records a billable audit event or marks the invoice as viewed.

readOnlyHint

Set readOnlyHint: true only when the tool changes nothing.

Good read-only examples:

• get_order
• list_saved_reports
• search_docs
• preview_invoice
• calculate_tax_estimate

Write examples that should set readOnlyHint: false:

• create_ticket
• update_profile
• delete_file
• send_email
• publish_post
• run_export when it queues a job or writes a file

Read-only is strict. If the tool changes product state, external state, permissions, billing, notifications, or durable user-visible history, do not mark it read-only.

destructiveHint

destructiveHint only matters when readOnlyHint is false.

Set it to true when the tool can delete, overwrite, revoke, remove, archive, cancel, replace, transfer, spend, or trigger another hard-to-reverse side effect.

annotations: {
  readOnlyHint: false,
  destructiveHint: true,
  idempotentHint: true,
  openWorldHint: false,
}

Set it to false when the write is additive or reversible, such as creating a draft, adding an internal comment, or saving a preference.

annotations: {
  readOnlyHint: false,
  destructiveHint: false,
  idempotentHint: false,
  openWorldHint: false,
}

Creating data is still a write. It is usually not destructive, but it is not read-only.

idempotentHint

Set idempotentHint: true when repeating the same call with the same arguments has no extra effect.

Usually idempotent:

• Setting a profile field to a specific value
• Updating a status to a specific enum value
• Deleting a record that is already deleted, if the server handles the second call as a no-op
• Replacing a draft with the same content

Usually not idempotent:

• Sending a message
• Charging a card
• Creating a new ticket without a dedupe key
• Appending a comment
• Incrementing a counter

This field matters for retries and failure handling. If a network timeout happens after the server processed the call, an idempotent tool is much safer to retry than a non-idempotent tool.

openWorldHint

openWorldHint is about reach. In MCP, a web search tool is open-world because it interacts with external entities. A private memory tool is closed-world because its domain is bounded.

Set openWorldHint: true for tools that:

• Search or fetch the public web
• Call third-party APIs
• Send email, Slack, SMS, or other external messages
• Publish public content
• Push code, open public issues, or update hosted repositories
• Submit forms outside your own app
• Trigger browser automation against arbitrary sites

Set openWorldHint: false only when the tool stays inside a closed system you control, such as your own private database or a local deterministic computation.

For ChatGPT app planning, OpenAI describes this as tools that publish content or reach outside the user’s account. If a tool can affect someone outside the current user’s private workspace, mark it open-world.

Common Annotation Patterns

Use this table as a starting policy. Then adjust for your app’s actual side effects.

Tool pattern	readOnlyHint	destructiveHint	idempotentHint	openWorldHint
Internal search, list, get	true	false	true	false
Public web search or URL fetch	true	false	true	true
Preview or dry run	true	false	true	depends
Create internal draft	false	false	false	false
Set or update internal field	false	false	true	false
Append comment	false	false	false	depends
Delete, revoke, overwrite, cancel	false	true	depends	depends
Send external message	false	false	false	true
Publish public content	false	false or true	false	true
Charge, transfer, submit order	false	true	usually false	true

“Depends” should make you write a short policy note. For example, appending an internal comment to a private ticket can be closed-world, while appending a GitHub comment on a public issue is open-world.

ChatGPT Plugin Submission Impact

For ChatGPT, annotations are not just local hints. They are part of the app metadata that the plugin portal scans from your production MCP endpoint.

OpenAI’s current submission docs say Apps are submitted and published as Plugins. When the portal scans your MCP endpoint, it stores discovered metadata with the draft version. If you change tool names, descriptions, schemas, annotations, security schemes, tool _meta, UI resource references, visibility, or MCP server instructions, you need to deploy the change, scan the endpoint again, submit a new version for review, and publish it after approval.

That changes how you should handle annotations:

1. Treat annotations as a versioned API contract.
2. Test annotations before the production scan.
3. Include annotation changes in release notes when you submit a new version.
4. Keep old tool contracts compatible while the reviewed version is still live.
5. Do not “fix it on the server” and assume ChatGPT’s published metadata changes immediately.

ChatGPT also requires readOnlyHint, destructiveHint, and openWorldHint on tool descriptors. idempotentHint is optional in the OpenAI table, but you should still set it. It gives future hosts and teammates useful retry information, and it removes ambiguity from your own policy.

Claude Connector Impact

Claude has more than one surface that can see MCP tools. A user can connect remote MCP servers through Claude’s MCP connector in the Messages API, and teams can also submit connectors for broader distribution.

The host-specific review details can change, so do not overfit your tool metadata to one portal form. For Claude Connectors, the durable rule is to keep every tool easy to review:

• Use a clear title.
• Use a narrow description.
• Separate read tools from write tools.
• Mark read-only tools with readOnlyHint: true.
• Mark destructive writes with destructiveHint: true.
• Mark external or public reach with openWorldHint: true.
• Keep tool schemas specific enough that Claude can choose the right tool.

If the same MCP server supports ChatGPT and Claude, use one strict annotation policy across both. Cross-host consistency is easier to test and easier to explain during review.

Setting Annotations in a sunpeak Project

In a sunpeak project, keep annotations next to the tool config so the side-effect policy lives beside the handler contract:

import type { AppToolConfig } from 'sunpeak/mcp';

export const tool: AppToolConfig = {
  resource: 'order-status',
  title: 'Get Order Status',
  description: 'Look up the current status of an order by order ID.',
  annotations: {
    readOnlyHint: true,
    destructiveHint: false,
    idempotentHint: true,
    openWorldHint: false,
  },
};

A write tool should be just as explicit:

import type { AppToolConfig } from 'sunpeak/mcp';

export const tool: AppToolConfig = {
  resource: 'send-notification',
  title: 'Send Notification',
  description: 'Send a push notification to a user by user ID.',
  annotations: {
    readOnlyHint: false,
    destructiveHint: false,
    idempotentHint: false,
    openWorldHint: true,
  },
};

For an existing MCP server that is not built with sunpeak, the same fields belong in the tool descriptor your framework returns from tools/list. The test strategy below works either way because it checks the protocol output, not your source file format.

Test the Protocol Output, Not Just Source Files

Source-level tests are useful, but the host only sees tools/list. Your CI should verify the MCP descriptor that ChatGPT, Claude, or another host will actually read.

With sunpeak’s testing framework, use the mcp fixture:

import { expect, test } from 'sunpeak/test';

test('every tool exposes explicit annotations', async ({ mcp }) => {
  const { tools } = await mcp.listTools();

  for (const tool of tools) {
    expect(tool.title, `${tool.name} is missing a title`).toEqual(expect.any(String));
    expect(tool.annotations, `${tool.name} is missing annotations`).toBeDefined();

    for (const key of [
      'readOnlyHint',
      'destructiveHint',
      'idempotentHint',
      'openWorldHint',
    ] as const) {
      expect(
        typeof tool.annotations?.[key],
        `${tool.name} must set ${key} to true or false`
      ).toBe('boolean');
    }
  }
});

This catches missing annotations before the plugin portal or connector review does.

Test for Contradictions

The first test only checks presence. Add consistency checks so a tool cannot claim incompatible behavior.

test('tool annotations do not contradict each other', async ({ mcp }) => {
  const { tools } = await mcp.listTools();

  for (const tool of tools) {
    const annotations = tool.annotations ?? {};

    if (annotations.readOnlyHint === true) {
      expect(
        annotations.destructiveHint,
        `${tool.name} cannot be read-only and destructive`
      ).toBe(false);

      expect(
        annotations.idempotentHint,
        `${tool.name} is read-only, so repeated calls should be safe`
      ).toBe(true);
    }
  }
});

You can also check open-world policy:

test('external tools are marked open-world', async ({ mcp }) => {
  const { tools } = await mcp.listTools();
  const externalVerbs = /^(fetch|browse|send|email|post|publish|push|submit)_/;

  for (const tool of tools.filter((candidate) => externalVerbs.test(candidate.name))) {
    expect(
      tool.annotations?.openWorldHint,
      `${tool.name} appears to reach outside the app`
    ).toBe(true);
  }
});

Naming tests are intentionally blunt. They catch common drift, but they should not be your only policy.

Add a Tool Policy Table

For serious apps, write the expected annotations in one place and assert the live MCP server matches it. That is better than guessing from prefixes.

const expectedAnnotations = {
  get_order: {
    readOnlyHint: true,
    destructiveHint: false,
    idempotentHint: true,
    openWorldHint: false,
  },
  send_order_email: {
    readOnlyHint: false,
    destructiveHint: false,
    idempotentHint: false,
    openWorldHint: true,
  },
  delete_order_draft: {
    readOnlyHint: false,
    destructiveHint: true,
    idempotentHint: true,
    openWorldHint: false,
  },
} as const;

test('tools match the reviewed side-effect policy', async ({ mcp }) => {
  const { tools } = await mcp.listTools();
  const byName = new Map(tools.map((tool) => [tool.name, tool]));

  for (const [name, annotations] of Object.entries(expectedAnnotations)) {
    const tool = byName.get(name);

    expect(tool, `${name} is missing from tools/list`).toBeDefined();
    expect(tool?.annotations).toMatchObject(annotations);
  }
});

This test turns annotation review into normal code review. If someone changes send_order_email to also post to a public channel, the expected annotation policy has to change in the same pull request.

Test Submission Readiness

Before submitting to ChatGPT or Claude, run a release-oriented annotation check against the same production URL you plan to submit.

npx sunpeak test init --server https://your-app.example.com/mcp
pnpm test

For ChatGPT plugin submission, add a checklist item after deployment and before portal scan:

• Production MCP endpoint is reachable from the public internet.
• tools/list includes every expected tool.
• Every tool has title, readOnlyHint, destructiveHint, idempotentHint, and openWorldHint.
• OpenAI-required fields, including readOnlyHint, destructiveHint, and openWorldHint, are booleans.
• Tool descriptions match the real behavior.
• Tool annotations match a checked-in side-effect policy table.
• Write tools have reviewer test cases.
• Destructive or open-world tools have user-visible confirmation copy that matches the actual side effect.

Then scan the production endpoint in the plugin portal. If the portal reports different metadata than CI, treat that as a release blocker and inspect the deployed server, cache, and environment.

Common Mistakes

Marking a cached write as read-only

A tool that reads from an external API and writes the result to your database is not read-only, even if the user only sees retrieved data.

Fix: separate fetch_preview from save_fetch_result, or mark the combined tool as a write.

Treating additive writes as destructive

Creating a draft or adding a private note is usually not destructive. Marking every write as destructive creates unnecessary approval friction and makes review harder because reviewers have to infer your real policy from vague metadata.

Fix: reserve destructiveHint: true for deletes, overwrites, revocations, irreversible sends, financial actions, or writes with comparable risk.

Treating public reach as closed-world

If a tool sends a Slack message, posts to GitHub, publishes a page, submits a third-party form, or searches the open web, it should usually be open-world.

Fix: ask whether the tool reaches outside the user’s closed private domain. If yes, set openWorldHint: true.

Forgetting metadata versioning

Changing annotations on the server does not automatically update an already published ChatGPT plugin metadata snapshot.

Fix: deploy the change, scan the MCP endpoint again, submit the new app metadata version for review, and publish after approval.

Depending on host prompts instead of server policy

Annotations help hosts frame calls for the user, but they are hints. They are not an auth system.

Fix: enforce authorization, input validation, rate limits, idempotency keys, and business rules in the server. Assume a hostile or buggy client can call tools directly.

Where Annotation Tests Fit

Annotation tests belong in the integration layer of your MCP App testing strategy. They are fast because they call tools/list and sometimes tools/call; they do not need a browser, a live host account, or model credits.

Use the layers this way:

1. Unit tests check handler utilities and side-effect classifiers.
2. Integration tests check tools/list, annotations, schemas, structuredContent, _meta, and error results.
3. Inspector E2E tests check how the host presents read, write, destructive, and open-world flows in the rendered app.
4. Live tests check the final ChatGPT or Claude behavior only when a real host account is needed.
5. Evals check whether models choose the right tool given realistic prompts.

sunpeak gives new MCP App projects a local inspector and testing stack through npx sunpeak new. For an existing MCP server, npx sunpeak inspect --server URL lets you inspect tools locally, and npx sunpeak test init --server URL adds protocol and inspector tests around the server you already have.

The value is not just avoiding rejection. Correct annotations make the product calmer. Read tools run with less friction, risky tools get the right confirmation, and reviewers can see that your tool contract matches your app’s real behavior.