Model Gateway ​

Model Gateway is a single point of control between your applications and LLM providers. Applications connect through an OpenAI-compatible or Anthropic-compatible API, and the gateway handles provider authentication, routing, and policy enforcement so your applications don't have to.

The following diagram shows the path a request takes through Model Gateway:

You can use Model Gateway to perform the following:

• Issue scoped access keys to teams: Generate access keys for each team, project, or application instead of sharing provider credentials. Revoke or rotate a key without touching downstream applications, and keep provider keys out of environment variables, configuration files, and CI pipelines.
• Swap providers without rewriting applications: Route Workato recipes, genies, copilots, and external applications through a single gateway that accepts OpenAI-compatible and Anthropic-compatible requests. Both APIs work with any provider, such as OpenAI, Anthropic, Azure OpenAI, or AWS Bedrock, so you can switch models and providers without code changes in the calling applications.
• Apply usage limits: Define rate limits and token limits in a policy, then assign the policy to the access keys a team or application uses to cap how much they can consume.
• Restrict access by IP address: Allow or block requests from specific IP addresses or ranges in a policy.

Key components ​

Model Gateway includes the following components:

• Providers: Connections to LLM services, such as Anthropic, OpenAI Compatible, Azure OpenAI, and AWS Bedrock, along with the credentials Model Gateway uses to authenticate. Providers are stored in a project and can be reused in genies.
• Routes: The targets that serve requests. A route resolves to a provider and model, either directly or through rules that match on tags, keys, parameters, or request complexity. Applications reference a route by its route ID.
• Access keys: Credentials that applications present to Model Gateway. Each access key grants access to one or more routes and attributes usage to a team or application.
• Policies: Reusable rate limits, token limits, and IP access restrictions that you assign to routes and access keys.
• Dashboard: Workspace-level visibility into request volume, token usage, and success and failure rates.

Refer to Configure Model Gateway to add providers, create routes, issue access keys, and define policies, the Route builder to define the rules a rule-based or intelligent route uses, or the Model Gateway dashboard to monitor traffic and token usage.

How Model Gateway works ​

Applications send a request to Model Gateway and authenticate with an access key. The access key determines which routes the application can use, and the route it names determines the provider and model that serves the request. When a policy applies, Model Gateway enforces its rate limits, token limits, and IP access restrictions before the request reaches the provider.

Supported applications ​

Model Gateway exposes two APIs that clients, such as genies, Workato recipes, or external applications, can route through without code changes:

• OpenAI-compatible API: Any client that's compatible with the OpenAI SDK, including Workato recipes, genies, copilots, and external applications.
• Anthropic-compatible API: Clients that use the Anthropic Messages API, such as Claude Desktop, Claude Code, and applications built with the Anthropic SDK. Set the client's base URL to <model-gateway-host>/anthropic, for example https://ai.workato.com/anthropic. This API supports the /anthropic/v1/messages and /anthropic/v1/messages/count_tokens endpoints. Refer to Connect Anthropic-compatible clients to configure Claude Desktop, Claude Code, and the Anthropic SDKs.

Both APIs work with every route, regardless of the provider behind it. For example, an application built with the Anthropic SDK can send requests through the Anthropic-compatible API to a route backed by OpenAI or AWS Bedrock. You aren't locked into a provider by the API format your clients use.

Model Gateway supports Server-Sent Events (SSE) end to end on both APIs to allow streaming responses to render incrementally rather than waiting for the full payload.