Pre-Submission Testing for MCP Apps: Validate Before Publishing to ChatGPT and Claude (October 2026)

You built an MCP App. It works in your local inspector. Now you want to submit it to ChatGPT as a plugin or list it in the Claude Connectors Directory so users can install it without a custom setup.

This guide is the pre-submission pass I would run before sending an app to review. It covers protocol checks, UI checks, privacy checks, reviewer credentials, and the current requirements that changed since the first version of this article.

TL;DR: Test the deployed server, tool contracts, rendered UI, privacy boundaries, and reviewer access before opening either portal. For ChatGPT, package the MCP connection in a plugin ZIP, scan the production server, run five positive and three negative cases, and provide a video walkthrough. For Claude, test the remote connector in Claude and prepare app-response screenshots if it renders UI. Use sunpeak for repeatable local checks, then verify the submitted build in each live host.

What the Current Review Flows Require

The two submission paths share the MCP server but ask for different review materials:

• OpenAI’s plugin submission flow begins with a ZIP package that declares the MCP connection. The portal connects to and scans the live server, verifies its domain, and asks for review materials. Initial MCP app review needs five positive cases, three negative cases, a video walkthrough, release notes, and reviewer access where sign-in is required.
• Claude’s connector submission flow accepts remote HTTPS MCP servers and MCP Apps through the developer portal. For an MCP App listing, it requires three to five PNG carousel images, at least 1000 pixels wide, cropped to the app response. Local MCP bundles are no longer accepted as connector listings.
• The MCP Apps extension defines the shared UI resource, host bridge, tool visibility, and iframe behavior. Test that contract first, then verify the submission build in each host.

The practical result: reviewers are checking more than “does it load?” They are checking whether the model can choose tools safely, whether users see relevant output, whether the UI works on the surfaces you support, and whether the data you collect matches what you disclose.

Build the Submission Gate First

Before you touch the submission portal, create a release gate that a teammate can run:

pnpm test
pnpm test:e2e
pnpm test:visual
pnpm test:live

Your exact commands may differ, but the gate should cover:

• Protocol-level tool checks with tools/list and tools/call
• Rendered UI checks in every supported display mode
• Light and dark theme screenshots
• Mobile viewport checks
• Authentication and reviewer-account checks
• Privacy and data-minimization checks
• CSP and external-domain checks
• Live-host checks for ChatGPT and Claude when possible

sunpeak gives you this shape out of the box. The testing framework runs protocol tests with the mcp fixture, UI tests with the inspector fixture, and host-specific E2E projects for ChatGPT and Claude replicas. That matters because host bugs often hide in UI details, such as iframe height, safe areas, theme tokens, and how a host presents write confirmations.

Tool Annotation Testing

Tool annotations affect model behavior, user trust, and reviewer approval. OpenAI’s current plugin guidelines require all three hints as explicit booleans. Classify the tool’s actual effects, including indirect effects, before writing an assertion.

For ChatGPT, set these explicitly:

Annotation	Set to true when	Example
readOnlyHint	The tool reads, computes, or previews without changing external state	get_order_status
destructiveHint	The tool can delete, overwrite, revoke, or cause another potentially destructive or irreversible effect	delete_file, send_invoice
openWorldHint	The tool reaches public or open-ended entities, including read-only web search or arbitrary destinations	search_public_web, send_to_any_address

An additive write such as creating a private note can be readOnlyHint: false and destructiveHint: false. A read-only public web search can be readOnlyHint: true and openWorldHint: true. A private account API can have openWorldHint: false even though it uses the network. These are behavior checks, not guesses from the verb or transport. The hints do not replace server authorization or confirmation for irreversible actions.

For Claude Directory submissions, every tool needs a title and the applicable readOnlyHint or destructiveHint. I still recommend setting all relevant annotations consistently across hosts because the same MCP server often serves multiple clients.

In a sunpeak project, annotations live beside the tool config:

import type { AppToolConfig } from 'sunpeak/mcp';

export const tool: AppToolConfig = {
  resource: 'order-status',
  title: 'Get Order Status',
  description: 'Look up the current status of an order by order ID',
  annotations: {
    readOnlyHint: true,
    destructiveHint: false,
    openWorldHint: false,
  },
};

Then enforce them with integration tests:

import { test, expect } from 'sunpeak/test';

test('every tool has explicit safety annotations', async ({ mcp }) => {
  const { tools } = await mcp.listTools();

  for (const tool of tools) {
    expect(tool.title, `${tool.name} missing title`).toBeTruthy();
    expect(tool.annotations?.readOnlyHint, `${tool.name} missing readOnlyHint`).toEqual(
      expect.any(Boolean)
    );
    expect(tool.annotations?.destructiveHint, `${tool.name} missing destructiveHint`).toEqual(
      expect.any(Boolean)
    );
    expect(tool.annotations?.openWorldHint, `${tool.name} missing openWorldHint`).toEqual(
      expect.any(Boolean)
    );
  }
});

test('known tool effects match annotations', async ({ mcp }) => {
  const { tools } = await mcp.listTools();
  const expected = {
    get_order_status: { readOnlyHint: true, destructiveHint: false, openWorldHint: false },
    create_private_note: { readOnlyHint: false, destructiveHint: false, openWorldHint: false },
    delete_file: { readOnlyHint: false, destructiveHint: true, openWorldHint: false },
  };

  for (const [name, annotations] of Object.entries(expected)) {
    const tool = tools.find((candidate) => candidate.name === name);
    expect(tool, `${name} is missing`).toBeDefined();
    expect(tool?.annotations, `${name} annotations changed`).toMatchObject(annotations);
  }
});

Replace the names and expected values with your own tool inventory. Review each handler and its downstream calls because run_report might only compute a result in one app and enqueue a job in another. The first test catches missing booleans; the second catches a value that no longer matches the implementation.

Tool Name, Description, Input, and Output Tests

Reviewers and models use the tool contract as the manual for your app. A tool with a broad description, vague input schema, or noisy output is harder to review and harder for the model to call correctly.

Test these rules:

• Tool names are unique and action-specific, such as get_invoice or create_ticket
• Descriptions say what the tool does, not why your app is better
• Inputs are specific to the task and avoid raw chat transcripts, broad context blobs, and unnecessary location fields
• Outputs return the data the user needs, without debug fields, logs, trace IDs, or unrelated identifiers
• Tools that return structured data include structuredContent, and if you define an outputSchema, responses conform to it

The structured output part is easy to miss. The current MCP spec allows tools to return structuredContent and an optional outputSchema. If you use them, validate them before submission:

test('order lookup returns declared structured content', async ({ mcp }) => {
  const result = await mcp.callTool('get_order', { orderId: 'test-123' });

  expect(result.isError).toBeFalsy();
  expect(result.structuredContent).toMatchObject({
    orderId: 'test-123',
    status: expect.any(String),
  });
});

Use MCP App output schema and structuredContent if your UI depends on structured data. It keeps the model-visible response concise while giving the app view the data it needs.

Reviewer Credential Testing

Broken reviewer access is one of the easiest problems to avoid.

Before submission, test your submitted URL and credentials from a clean browser profile outside your company network. Do not assume a logged-in developer session proves anything.

Your reviewer account should have:

• A dedicated username and password
• No MFA, SMS, email code, SSO, VPN, or IP allowlist requirement
• Stable sample data for every submitted test case
• Permissions that match a normal user, unless your app is explicitly an admin tool
• Credentials that do not expire during review
• A clear recovery path if the reviewer locks the account

Configure the MCP test client with the same account permissions as the reviewer, then test every submitted case. Do not silently skip a tool that lacks sample input:

test('reviewer account can run submitted read case', async ({ mcp }) => {
  const sample = { tool: 'get_order_status', input: { orderId: 'sample-123' } };
  const { tools } = await mcp.listTools();

  expect(tools.some((tool) => tool.name === sample.tool)).toBe(true);
  const result = await mcp.callTool(sample.tool, sample.input);
  expect(result.isError).toBeFalsy();
  expect(result.structuredContent).toMatchObject({ orderId: 'sample-123' });
});

Store test inputs in fixtures so the reviewer instructions, automated tests, and screenshots all use the same data.

Turn Review Cases Into Reproducible Evidence

OpenAI asks for five positive and three negative cases for initial MCP app review. A positive case should name the user prompt, expected tool call, and observable result. A negative case should say when the plugin must refuse, ask for clarification, or avoid a tool call. Record a video walkthrough using the same account and data, and keep the recording URL accessible to reviewers.

This is a useful starter matrix. Replace each row with behavior your app actually supports:

Case	Prompt or action	Expected evidence
Positive 1	Ask for one known record	Correct tool, matching ID, concise answer
Positive 2	Search with a narrow filter	Only matching records, no unrelated fields
Positive 3	Ask about an empty result	Clear empty state in text and UI
Positive 4	Open an interactive view	Resource loads on desktop and mobile; controls work
Positive 5	Confirm a permitted write	Exact target changes once; audit or response matches
Negative 1	Request a record outside the test account	Access denied without leaking whether it exists
Negative 2	Ask for a write without required confirmation	No write; clear next step for the user
Negative 3	Give an ambiguous target or prompt injection in data	Clarification or safe refusal; no unrelated tool call

For every row, save the account state, input, expected tools/call arguments, model-visible result, UI state, and a pass or fail note. Reset mutable sample data between cases. A screenshot alone cannot prove that the right tool ran or that a negative case caused no side effect.

Privacy and Data Minimization Testing

The best privacy review is boring. Every input field should be necessary, every returned field should be expected, and the privacy policy should name every category of user data the app returns or stores.

Audit inputs first:

test('tools do not request broad context fields', async ({ mcp }) => {
  const { tools } = await mcp.listTools();
  const bannedFieldNames = /conversation|transcript|fullContext|rawChat|password|token|apiKey/i;

  for (const tool of tools) {
    const properties = tool.inputSchema?.properties ?? {};
    expect(Object.keys(properties), `${tool.name} asks for broad or secret input`).not.toEqual(
      expect.arrayContaining([expect.stringMatching(bannedFieldNames)])
    );
  }
});

Then audit outputs:

test('tool responses do not leak internal metadata', async ({ mcp }) => {
  const result = await mcp.callTool('get_order', { orderId: 'test-123' });
  const responseText = JSON.stringify(result);

  expect(responseText).not.toMatch(/session[_-]?id/i);
  expect(responseText).not.toMatch(/trace[_-]?id/i);
  expect(responseText).not.toMatch(/request[_-]?id/i);
  expect(responseText).not.toMatch(/internal[_-]?account/i);
  expect(responseText).not.toMatch(/access[_-]?token/i);
});

Do a manual pass too. Automated regex checks will not tell you whether customerSegment is needed for a support-ticket lookup or whether a nested ownerEmail field is covered by your policy.

MCP App Rendering Tests

An MCP App is more than a tool result with HTML attached. The host discovers UI resources, renders the view in a sandboxed iframe, sends host context, passes tool input and result data, and lets the view call app-visible tools when allowed.

That lifecycle creates failure modes that unit tests miss:

• The resource URI is wrong or missing
• The iframe initializes after data is sent
• The view assumes browser storage, cookies, or parent DOM access it does not have
• The app depends on a host CSS variable without a fallback
• App-only tools are visible to the model, adding noise to tool selection
• The view does not handle teardown, refresh, or repeated tool results

Cover those with E2E tests:

import { test, expect } from 'sunpeak/test';

const displayModes = ['inline', 'pip', 'fullscreen'] as const;

for (const displayMode of displayModes) {
  test(`dashboard renders in ${displayMode}`, async ({ inspector }) => {
    const result = await inspector.renderTool('show-dashboard', undefined, {
      displayMode,
    });

    const app = result.app();
    await expect(app.locator('[data-testid="dashboard"]')).toBeVisible();

    const hasHorizontalOverflow = await app.evaluate(() => {
      return document.documentElement.scrollWidth > document.documentElement.clientWidth;
    });

    expect(hasHorizontalOverflow).toBe(false);
  });
}

If your app only supports one display mode, test only that mode. The mistake is claiming support for modes you never test.

Theme, Mobile, and Screenshot Tests

ChatGPT review asks that test cases pass on web and mobile. Claude Directory submissions for MCP Apps ask for carousel screenshots cropped to the app response. Treat both as testable artifacts.

At minimum:

• Test light and dark themes
• Test a 375px mobile viewport
• Test touch-friendly controls, especially if desktop uses hover
• Test long labels, empty states, and failed network states
• Generate fresh screenshots from the same fixtures you list in the submission form

for (const theme of ['light', 'dark'] as const) {
  test(`search app renders in ${theme} theme`, async ({ inspector }) => {
    const result = await inspector.renderTool('search-products', { query: 'desk' }, { theme });
    await expect(result.app().locator('[data-testid="results"]')).toBeVisible();
  });
}

test('search app renders on mobile', async ({ inspector, page }) => {
  await page.setViewportSize({ width: 375, height: 812 });

  const result = await inspector.renderTool('search-products', { query: 'desk' });
  const app = result.app();

  await expect(app.locator('[data-testid="results"]')).toBeVisible();
  await expect(app.locator('button').first()).toHaveCSS('min-height', /4[4-9]px|[5-9]\dpx/);
});

For screenshots, avoid staging data that will disappear. If a reviewer sees a screenshot of one UI state and your test account has different data, you make the review harder.

CSP and External-Link Testing

MCP App views run in sandboxed iframes. They communicate with the host through postMessage, and network access is controlled by declared CSP metadata.

For ChatGPT Apps, use the current _meta.ui.csp shape where possible and keep the older OpenAI compatibility keys only when you need them. Review these fields:

• connectDomains for API calls
• resourceDomains for static assets
• frameDomains for each embedded page origin, with a clear reason for the embed
• Redirect or allowed link targets when the host asks for them

OpenAI’s plugin guidelines allow pages on the MCP server’s own registrable domain and limit third-party embeds to cases where they are essential. Every iframe origin still needs a submission explanation, and iframe use can slow review or lead to rejection when reviewers cannot assess the content.

For Claude, allowed link URIs are optional. If you list them, use only HTTPS origins or custom URI schemes you own. Other links still work, but the user sees a confirmation prompt.

Test CSP by making the app prove its external dependencies work:

test('weather app can reach declared API domain', async ({ inspector }) => {
  const result = await inspector.renderTool('show-weather', { city: 'Seattle' });
  const app = result.app();

  await expect(app.locator('[data-testid="temperature"]')).toBeVisible();
  await expect(app.locator('[data-testid="network-error"]')).toBeHidden();
});

Read MCP App CSP domains before submission if your app loads images, fonts, maps, analytics, or iframe content from external origins.

Cross-Host Testing

ChatGPT and Claude can both render MCP App-style UIs, but they do not have identical host chrome, fonts, iframe sizing, auth flows, display modes, or review forms.

Run the same tests against both host profiles:

pnpm test:e2e

With sunpeak, the test config can create separate Playwright projects for ChatGPT and Claude replicas, so the same test runs once per host. When a test fails, the report shows which host failed.

For submission, also run one live pass:

• Connect your MCP URL in ChatGPT Developer Mode and run every submitted test case
• Add the same server as a Claude custom connector or use MCP Inspector for the Claude path
• Confirm OAuth callback URLs and allowed redirect domains match the real submitted domain
• Compare generated text output, not only UI rendering

This catches the common problem where the app looks right but the tool’s model-visible result is stale, noisy, or missing the detail the model needs for a follow-up call.

Claude Directory-Specific Checks

Claude’s current connector submission guide accepts remote MCP servers, including MCP Apps, through the developer portal. A local MCP bundle is not accepted as a connector listing. The portal syncs tools, prompts, and resources from the connected server.

Before submitting a Claude Connector, confirm:

• The server URL is https://
• The transport selection in the portal matches your server
• Tool titles and annotations are present before the portal syncs tools
• Authenticated services use OAuth 2.0 unless you have coordinated a different mode
• Documentation and privacy policy URLs are public
• Test-account instructions include every URL, credential, setup step, and sample prompt the reviewer needs
• MCP Apps include 3 to 5 PNG carousel screenshots at least 1000px wide, cropped to the app response, with each screenshot’s prompt supplied separately
• You ran every tool through MCP Inspector or as a custom connector in Claude

The portal also asks about use cases, company details, data handling, and compliance. Write those answers from the same source of truth as your tests. If your submission says the connector only reads data, your write-tool tests and annotations should agree.

ChatGPT-Specific Checks

For ChatGPT, package the app’s MCP connection in a plugin ZIP. Upload that ZIP in the OpenAI Platform, connect and scan the production MCP server from the draft, then resolve findings before submitting it for review. Put reviewer credentials in the secure Review details form, never in the ZIP.

Before submitting for review, verify:

• The submitter has Apps Management: Write access in the OpenAI Platform organization
• The publisher has completed individual or business identity verification
• The initial plugin ZIP declares the MCP connection and contains no private credentials
• The MCP URL is reachable from a public network
• The domain can serve the portal’s exact token at /.well-known/openai-apps-challenge
• The portal’s latest scan matches the production tool schemas, annotations, UI metadata, and CSP
• The reviewer account works without MFA, SMS, email codes, SSO, or private-network access
• Every tool has accurate readOnlyHint, openWorldHint, and destructiveHint values
• Tool responses avoid unrelated user identifiers and internal metadata
• The privacy policy names the categories of personal data the app collects, returns, stores, and shares
• The app does not use app or tool descriptions to manipulate model selection
• Every frameDomains origin is necessary, declared in CSP, and explained for review
• The submission contains exactly five positive and three negative test cases with expected behavior and reproducible fixture or account data
• A video walkthrough demonstrates the submitted cases and has a reviewer-accessible URL
• Listing details, starter prompts, release notes, support, privacy, and terms URLs are ready

Submitting starts review. After approval, the developer chooses when to publish the plugin. OpenAI periodically rescans published MCP tools, so rerun the contract and UI tests after server changes too.

The Pre-Submission Checklist

Run this checklist before submitting to either platform.

Tool contract

• Every tool has a unique, action-specific name
• Every tool has a human-readable title
• Every tool description matches real behavior
• Every tool has explicit readOnlyHint, destructiveHint, and openWorldHint
• Write tools are not marked read-only
• Potentially destructive or irreversible writes set destructiveHint: true; additive writes are reviewed separately
• Public search and arbitrary destinations set openWorldHint: true; bounded private accounts are reviewed separately
• Inputs are narrow and task-specific
• Outputs are relevant, concise, and free of debug metadata
• structuredContent and outputSchema are valid where used

Reviewer access

• Public MCP URL works outside your network
• Demo account exists and has stable sample data
• MFA, SMS, email-code login, VPN, IP allowlists, and SSO are disabled for reviewers
• Credentials have not expired
• Reviewer instructions list exact prompts, expected outputs, and any setup steps

Privacy and data

• Privacy policy is public
• Privacy policy covers collected, returned, stored, and shared user data
• Tool inputs do not request raw chat logs, broad context, auth secrets, or unnecessary location data
• Tool responses do not leak session IDs, trace IDs, request IDs, logs, tokens, or unrelated personal data
• Restricted data is not collected unless the app is built, disclosed, and reviewed for that use case

UI and host behavior

• App renders in every declared display mode
• App works in light and dark themes
• App works at mobile viewport widths
• No horizontal overflow appears in supported modes
• Touch targets are usable on mobile
• Loading, empty, error, cancelled, and retry states are visible and clear
• App-only tools are hidden from model-visible tool selection
• UI survives repeated tool results, refreshes, and teardown

CSP and external access

• connectDomains includes every API origin the app calls
• resourceDomains includes required asset origins
• Every frameDomains origin is declared, necessary, and explained for review
• Allowed external link targets are listed when the host requires them
• No staging, localhost, or private-network URLs remain in submitted metadata

Claude Directory

• Server is reachable over https://
• Transport selection matches the server
• OAuth setup matches the auth mode described in the portal
• Documentation URL and privacy URL are public
• MCP App screenshots are 3 to 5 PNGs, at least 1000px wide, cropped to the app response, and paired with prompts
• Every tool was run through MCP Inspector or as a custom connector in Claude

ChatGPT

• Submitter has Apps Management write access
• Publisher identity is verified in the same Platform organization and project
• Plugin ZIP declares the MCP server and contains no reviewer credentials
• Production MCP connection and latest tool scan pass in the portal
• MCP domain verification token is available at the generated well-known URL
• Submission has exactly five positive and three negative test cases
• Video walkthrough URL works for reviewers
• Starter prompts and release notes are complete
• Website, support, privacy, and terms URLs are public and match the publisher
• The app does not sell or advertise in ways the policy disallows
• App and tool descriptions do not compare against or disparage other services
• UI and model-visible outputs both match the submitted expected behavior

Run the Gate, Then Submit

With sunpeak, your final pre-submission run should look like this:

pnpm test
pnpm test:e2e
pnpm test:visual
pnpm test:live

Then compare the submission form, scanned tool metadata, video, test report, reviewer account, privacy policy, and rendered app. They should all describe the same product behavior.

That comparison catches contradictions automated checks miss, such as a write described as read-only or a screenshot showing data the reviewer account cannot access.