AI Agent Video APIs: MCP Rendering vs Generation

An AI agent that makes videos usually needs two layers. A generator makes footage that does not exist yet, from a prompt. A renderer turns exact values - text, brand colours, layout, captions, media URLs - into the final MP4. Both now come as MCP servers. Runway runs a generation MCP. Renderly runs a hosted rendering MCP at https://renderly.video/api/mcp with ten tools and a free preview. This post explains when your agent needs each layer, compares Remotion on your own server with a hosted renderer, and shows the setup for Claude, Cursor and a custom agent.

Key Takeaways

• Use a generator only for footage that does not exist. Use a renderer for captions, text, layout, brand and the final MP4.
• Runway's MCP makes images and video from a prompt. Renderly's MCP fills templates with exact values and renders the MP4.
• Remotion's own MCP is a documentation tool, and Remotion now marks it as deprecated. MCP servers for Remotion compositions are community packages, some local and some hosted.
• Running Remotion on your own server means you supply Chrome, the queue and storage. Renderly's hosted MCP adds OAuth or API-key auth, free previews and webhooks.
• One endpoint works in Claude, Cursor and any custom agent that speaks Streamable HTTP.

Which video API should your AI agent use?

Runway describes its MCP like this: "Generate high-quality images and videos right from where you're already working" (Runway, Runway MCP, 2026). That is the generation layer. It answers prompts such as "a drone shot over a misty pine forest".

Most agent video jobs are not that. They are "make this week's product video with these six prices", "render a welcome clip for each of these 40 new customers" or "add captions to this narration and export it at 9:16". The footage either exists already or does not matter. What matters is that the price on screen is the right price, the logo is the real logo, and the text does not run off the frame. That is the rendering layer.

So the answer depends on the job:

What does a rendering MCP do that a generator cannot?

The Renderly MCP server registers ten tools, and every one of them works on templates and projects that already exist (lib/mcp/tools.ts). Its own server card describes the scope in one line: "List templates and projects, read their variable contracts, preview a payload for free, then render an MP4."

The difference from a generator is determinism. A template has named variables. The agent reads them with renderly_get_variables, and each text variable reports maxCharacters and linesAvailable. The tool description warns the agent that a longer value "will have its tail CLIPPED by the renderer", so the agent can shorten a headline before it spends a credit. A generative model has no such contract. It can misspell the product name, and a retry gives a different shot.

The ten tools fall into four groups:

The preview tool is the one an agent should call most. It "COSTS NO CREDITS", resolves the values exactly as a render would, and returns a public previewUrl, the resolved map of which values applied, a credit quote and warnings. An unknown variable name comes back as UNKNOWN_REPLACEMENT instead of a silent wrong video. Our agent preview guide covers the frame grid that lets a vision model look at the result first.

Is there an MCP server for video rendering with Remotion?

Remotion's own MCP page now opens with "The Remotion MCP is deprecated." It adds: "The hosted MCP will shut down no earlier than August 31, 2026. New installations are not recommended." (Remotion, Model Context Protocol, 2026). The page gives the reason: "It duplicates the /remotion-docs skill." It was a documentation tool, not a renderer.

The MCP servers that work with Remotion compositions are community packages, and they differ. One example, the Remotion MCP App, says "Connect to the hosted instance" and describes how "a live video player renders the result directly inside the chat" (mcp-use, Remotion MCP App, 2026, github.com/mcp-use/remotion-mcp-app). Others run on your own machine. Read what each one returns before you build on it: a player in the chat, or an MP4 file at a URL.

If you run Remotion on your own server instead, you supply the browser. Remotion renders each frame in a browser, and its docs describe two browser modes, of which "Chrome Headless Shell: Is faster for CPU-bound video rendering, and requires less dependencies and setup" (Remotion, Chrome Headless Shell, 2026, remotion.dev/docs/miscellaneous/chrome-headless-shell). So a renderer on your own server brings Chrome, CPU and an output store with it.

Check the license before you put Remotion in a product. Remotion is free for "an individual", "a for-profit organization with up to 3 employees" and "a non-profit or not-for-profit organization". Outside that group, "You are required to obtain a Company License to use Remotion" (Remotion, License, 2026).

Remotion on your own server or a hosted rendering MCP?

The MCP spec draws the line itself. In the stdio transport, "the client launches the MCP server as a subprocess" (Model Context Protocol, stdio transport, 2026, modelcontextprotocol.io/specification/2026-07-28/basic/transports/stdio). In Streamable HTTP, "the server operates as an independent process that can handle multiple client connections" (Model Context Protocol, Streamable HTTP transport, 2026). A Remotion MCP server that you start on your own machine is the first kind. A hosted renderer is the second.

Pick your own server when you already write Remotion components, you have a render machine, and the agent runs on that same machine. Pick hosted when the agent runs in Claude.ai, in a cloud function, or in a teammate's editor, and nobody wants to operate Chrome. The hosted route also matters for long jobs. The Renderly MCP endpoint caps each request at 60 seconds (maxDuration in app/api/[transport]/route.ts), so a render returns a jobId at once and the agent polls renderly_get_render_status. Our MCP Tasks post explains how that maps onto the new Tasks extension. For the wider build-or-buy question, see FFmpeg vs a video API.

How do you connect a rendering MCP to Claude, Cursor or your own agent?

The Renderly endpoint accepts two credentials: a Renderly rnd_... API key as a Bearer token, or a WorkOS AuthKit access token from the OAuth flow (lib/mcp/auth.ts). An unauthenticated call gets a 401 that points to /.well-known/oauth-protected-resource, so OAuth clients find the sign-in on their own.

Claude

In Claude.ai or Claude Desktop, add a custom connector with the URL https://renderly.video/api/mcp and click Sign in with Renderly. These two apps are OAuth only. In Claude Code, a key is the fastest path:

claude mcp add --transport http --scope user renderly \
  https://renderly.video/api/mcp \
  --header "Authorization: Bearer rnd_YOUR_API_KEY"

The Claude connector guide has the menu steps for each plan.

Cursor

Put this in ~/.cursor/mcp.json, or in .cursor/mcp.json to share it with a project. Leave out headers to sign in with OAuth instead:

{
  "mcpServers": {
    "renderly": {
      "url": "https://renderly.video/api/mcp",
      "headers": {
        "Authorization": "Bearer ${env:RENDERLY_API_KEY}"
      }
    }
  }
}

The Cursor setup page covers team allowlists and environment variables.

A custom agent over HTTP

Any MCP client library can connect. This uses the official TypeScript SDK (@modelcontextprotocol/sdk) and an API key:

import { Client } from "@modelcontextprotocol/sdk/client/index.js";
import { StreamableHTTPClientTransport } from "@modelcontextprotocol/sdk/client/streamableHttp.js";
 
const transport = new StreamableHTTPClientTransport(
  new URL("https://renderly.video/api/mcp"),
  {
    requestInit: {
      headers: { Authorization: `Bearer ${process.env.RENDERLY_API_KEY}` },
    },
  },
);
 
const client = new Client({ name: "my-video-agent", version: "1.0.0" });
await client.connect(transport);
 
// Give these tool definitions to your model.
const { tools } = await client.listTools();
 
// When the model picks a tool, run it and return the text result.
const preview = await client.callTool({
  name: "renderly_preview_render",
  arguments: {
    templateId: "<template id>",
    replacements: { headline: "Spring sale ends Friday" },
  },
});

renderly_create_render takes the same arguments plus an optional webhookUrl. It returns a jobId, status, creditsUsed and estimatedDurationMinutes, and the finished outputUrl arrives through renderly_get_render_status or the webhook (lib/services/renders.ts).

How does one agent use both layers in a single job?

Renderly media variables take a public URL: "text variables take a string, media variables take a public URL" (renderly_create_render tool description). That URL is the hand-off point between the two layers.

1. The agent asks the generator for the one shot that does not exist, for example a ten-second product hero clip.
2. It puts the finished clip at a public URL. The Renderly MCP cannot upload a local file yet, so the clip must already be reachable on the web.
3. It calls renderly_preview_render with a template, the clip URL on the video variable, and the exact title, price and call to action.
4. It reads warnings and resolved, fixes anything wrong, and updates the same preview with its previewId.
5. It calls renderly_create_render and waits for the outputUrl.

The same template then renders the next 50 variants with new text and the same clip, with no new generation. The voiceover and captions can come in the same call: a sound variable accepts {"tts": {"text": "..."}} and a caption variable accepts {"transcribe": {"source": "<overlay name>"}}. Captions are free when they transcribe narration made with tts in the same request. Our faceless video API post shows that request in full.

What does an agent render cost?

Runway says of its MCP: "The cost depends on the model, resolution and other settings" (Runway, Runway MCP, 2026). Check its current rates in your Runway account before an agent runs a batch.

Renderly charges by output minute (config/paddle.ts):

• About 1 credit per minute of 1080p video, rounded up to the nearest 0.5 credit. Output up to 2560x1440 costs 2x, and larger output costs 4x.
• A credit costs $0.20 on pay-as-you-go (minimum 50 credits). The Creator plan is $29 a month for 200 credits. The Business plan is $99 a month for 1,000 credits, which is $0.099 a credit.
• AI voiceover costs 0.5 credits per started 1,000 characters. Captions cost 0.25 credits per started minute of audio, and nothing when they transcribe narration made in the same request.
• Previews cost nothing, and the connector itself is free to add.

That makes the rendering layer cheap to call often. The useful rule for an agent: generate rarely, preview always, render once.