Run an inference gateway that trusts GitHub Actions OIDC

This guide is for the platform operator who runs the inference gateway that fullsend's agents call (pi, codex and Claude Code). When you finish, you have an agentgateway service on Cloud Run that:

• accepts a GitHub Actions job's OIDC token as its only credential, and only for the repositories you list;
• holds the upstream model credential itself, so no forge secret stores a reusable key;
• refuses an expired token with 401 (agentgateway's jwtAuth does, after a 60 s leeway). Claude Code recovers from a stale credential only on a 401 or 403, never on a 5xx;
• serves the three APIs the runtimes use: /v1/messages (pi, Claude Code), /v1/chat/completions (pi) and /v1/responses (pi, codex).

Security boundary: securing the credential becomes your job. The gateway keeps model keys out of every repository, but the credential still exists: the upstream key in Secret Manager, and the runtime service account's access to Vertex AI. Both live in your GCP project, and protecting them is your job as the operator:

• Anyone who can deploy a revision that runs as the runtime service account (roles/run.developer on the service plus roles/iam.serviceAccountUser on the account) can read the upstream key and call Vertex AI as the gateway. Treat those roles as holding the key.
• Anyone who can read the key secret (roles/secretmanager.secretAccessor or an admin role on it or the project), or act as the runtime service account by any other route (another Cloud Run service or job, token creation, service account keys), also holds the key.
• Anyone who can add a version to the gateway-config secret also holds the key. The config decides which repositories the gateway serves and which signing keys (jwks) it trusts, and also where each model sends the upstream key: a model whose baseUrl points at the writer's own server, with apiKey.file naming the mounted key, hands the key over on the first call. A revision that mounts latest loads a new version on its next start, with no deploy needed. Let the people who approve enrolments edit only the allowlist, and have a trusted step generate, check and roll out the full config.
• Project owners and editors hold all of these. Keep that list short, and review the project's IAM and audit logs as you would review access to the key itself.

The runner side (how a fullsend run fetches the token and points the agent's runtime at the gateway) is in the user guide for the hosted gateway route. This page covers only the gateway. It uses agentgateway v1.7.0, and every config block on it is validated against that release.

See it in practice. The fullsend-ai organization runs its own gateway this way. Its config and operations live in the fullsend-ai/org inference gateway README.

What you end up with

You hand these three values to the people who enrol repositories. None of them is a secret.

Value	Example in this guide	What it is
Gateway URL	https://gateway.example.com	The Cloud Run service URL, or your own domain in front of it. Behind a shared ingress it can carry a path prefix (Serve the gateway under a path prefix).
Audience	fullsend-inference	The string every job asks GitHub to put in its token's aud claim. The gateway refuses any other value. You choose it; it has no required format.
Allowed repositories	example-org/example-repo	The owner/name of each repository the gateway serves, matched exactly against the token's repository claim.

Prerequisites

• A GCP project where you can create Artifact Registry repositories, service accounts, Secret Manager secrets and Cloud Run services, and grant project IAM roles.
• gcloud, signed in to that project.
• skopeo, to copy the release image.
• podman or docker, to validate the config before deploying (optional but recommended).
• For a Vertex AI upstream: the models you plan to serve enabled in the project. Model availability varies by project.
• A repository where you can add a workflow, to run the checks in step 8.

Set the names once. Every command below uses them:

export PROJECT=example-project
export REGION=us-east5
export AR_REPO=example-images
export SERVICE=inference-gateway
export SA_NAME=gateway-runtime
export SA="$SA_NAME@$PROJECT.iam.gserviceaccount.com"
export AUDIENCE=fullsend-inference

1. Copy the release image into Artifact Registry

Cloud Run can't pull from ghcr.io, so copy the agentgateway release image into your project unchanged.

gcloud artifacts repositories create "$AR_REPO" --project "$PROJECT" --location "$REGION" \
  --repository-format docker
skopeo copy --all --quiet \
  --dest-creds "oauth2accesstoken:$(gcloud auth print-access-token)" \
  docker://ghcr.io/agentgateway/agentgateway:v1.7.0 \
  "docker://$REGION-docker.pkg.dev/$PROJECT/$AR_REPO/agentgateway:v1.7.0"

Create request issued for: [example-images]
Waiting for operation [projects/example-project/locations/us-east5/operations/...] to complete...
.....done.
Created repository [example-images].

--all copies every architecture in the release. Check that the copy is byte-for-byte the release:

skopeo inspect --raw docker://ghcr.io/agentgateway/agentgateway:v1.7.0 | shasum -a 256
skopeo inspect --raw --creds "oauth2accesstoken:$(gcloud auth print-access-token)" \
  "docker://$REGION-docker.pkg.dev/$PROJECT/$AR_REPO/agentgateway:v1.7.0" | shasum -a 256

The release's digest:

60216aafb03d37f94e952ff3d51d2f630a65b10ecaebdc0a864882f92610c45b  -

Both commands must print this digest.

2. Create the runtime service account

The gateway calls Vertex AI as its own service account. No key file is created.

gcloud iam service-accounts create "$SA_NAME" --project "$PROJECT" \
  --display-name "inference gateway runtime"
gcloud projects add-iam-policy-binding "$PROJECT" --member "serviceAccount:$SA" \
  --role roles/aiplatform.user --condition=None --format='value(etag)'

Created service account [gateway-runtime].
Service account email: gateway-runtime@example-project.iam.gserviceaccount.com
Updated IAM policy for project [example-project].
BwZd...

Skip the roles/aiplatform.user binding if you serve no Vertex models.

3. Write the gateway config

Save this as config.tmpl.yaml. Each rule in the comments is required, and the reason is on the line it applies to.

gateways:
  default: { port: 8080, bindAddress: 0.0.0.0 }
llm:
  gateways: [default]
  discovery: disabled
  policies:
    jwtAuth:
      mode: strict                                   # a request without a valid token is refused
      providers:
      - issuer: https://token.actions.githubusercontent.com
        audiences: [AUDIENCE]                        # a fixed audience: a token minted for another service is refused
        jwks: { url: https://token.actions.githubusercontent.com/.well-known/jwks }   # remote JWKS: GitHub rotates its keys
  models:
  - name: claude-sonnet-4-6
    provider: vertex
    params: { vertexProject: VERTEX_PROJECT, vertexRegion: global }
    requestHeaders: { remove: [x-api-key] }          # the gateway forwards other caller headers unchanged
    authorization:
      rules:
      - allow: 'jwt.repository == "ALLOWED_REPO"'    # exact match; never a prefix or owner-only match
  - name: gemini-2.5-flash
    provider: vertex
    params: { vertexProject: VERTEX_PROJECT, vertexRegion: global }
    requestHeaders: { remove: [x-api-key] }
    authorization:
      rules:
      - allow: 'jwt.repository == "ALLOWED_REPO"'

What the config does, and what it must not do:

• Validate the token, then drop it. jwtAuth checks the signature against GitHub's published keys, the issuer, the audience and the expiry. After that, the caller's Authorization header is not forwarded upstream; the gateway sends its own credential instead. This is the default. Never set preserveToken: true or a passthrough backend auth: either one hands the job's token to the model provider.
• Authorise per repository. Each model's authorization rule decides who may call it, and /v1/models lists for each caller only the models it may call. Allow more repositories with ||: jwt.repository == "example-org/a" || jwt.repository == "example-org/b".
• Give every model an allow rule that tests the repository. Any GitHub repository can mint a token for your audience and pass jwtAuth, so the authorization rules are the only thing that refuses one. agentgateway lets every such caller through when a model has:
  • no authorization block, or an empty rules list;
  • only deny rules: everything not denied is allowed;
  • an allow rule that does not test the repository, such as 'true'. A bare string in rules (- 'true') is an allow rule;
  • only require rules, none of which tests the repository.
• Pin the repository ids too, if a name may be reused. GitHub's token also carries the numeric repository_id and repository_owner_id claims, sent as strings (jwt.repository_id == "123", not == 123). The ids stay with a repository through a rename; a new repository that reuses an old name gets a new repository_id. Matching the ids as well stops a reused name from inheriting access, and a renamed repository is refused until you update its entry. This walkthrough matches the name only.
• Strip x-api-key. agentgateway removes the credential it validated, but passes every other caller header through to the upstream, x-api-key included (step 9 shows both cases). Add requestHeaders.remove to every model.
• Pick the Vertex location your project serves the model in. This walkthrough used global for both models. Model availability varies by project and location.

Fill in the placeholders:

sed -e "s/VERTEX_PROJECT/$PROJECT/g" -e "s/AUDIENCE/$AUDIENCE/" \
    -e "s#ALLOWED_REPO#example-org/example-repo#g" config.tmpl.yaml > config.yaml

This is enough for a first deploy. Once you serve more than a few repositories, keep the allowed repositories in a file of their own and generate config.yaml from it with a script. The script should refuse a config that breaks the rules above, such as a model whose authorization rules do not test the repository. Then every change to the allowlist is a one-line, reviewable diff.

An upstream that needs an API key

For a provider that takes a key rather than the service account (OpenAI, for example), the gateway reads the key from a file. You mount that file from Secret Manager in step 6.

  - name: gpt-6-luna
    provider: openAI
    params: { apiKey: { file: /etc/agw-upstream/key } }
    requestHeaders: { remove: [x-api-key] }
    authorization:
      rules:
      - allow: 'jwt.repository == "ALLOWED_REPO"'

Not executed in this walkthrough: no OpenAI key was available for this run, so the OpenAI model and the native /v1/responses path are not shown here. The file-mounted key pattern itself was exercised with the custody-check upstream in step 9. If a file referenced by apiKey.file is missing, the gateway exits at startup.

Accept a static key as well (auth: api-key)

A repository that sets auth: api-key sends a key you issue instead of an OIDC token, in the same Authorization: Bearer header. (A repository can instead send its key as x-api-key on /v1/messages with messages_auth_header: x-api-key; that needs a gateway that reads the header, see Header custody for an x-api-key gateway.) Serve these repositories only if you must: a key is a long-lived secret, and a leaked key works until you revoke it. Prefer OIDC wherever it is possible.

Not executed here: this section has no live run, so no output is shown. Validate the config as in step 4 before you deploy it. To repeat the step 9 custody check with a key in place of the token, grant a test key the echo models for the check only: add echo and echo-noremove to its allowedModels, add its apiKey.purpose rule to both echo models, and remove both grants once the check passes.

Add an agentgateway apiKey policy under llm.policies. Store each key's SHA-256 hash, never the key, and list the models each key may call:

  policies:
    jwtAuth:
      mode: permissive                               # only on a gateway that serves both modes; see below
      providers:
      - issuer: https://token.actions.githubusercontent.com
        audiences: [AUDIENCE]
        jwks: { url: https://token.actions.githubusercontent.com/.well-known/jwks }
    apiKey:
      mode: optional
      keys:
      - keyHash: "sha256:KEY_HASH"                   # the key's hex SHA-256 digest, never the key itself
        metadata: { name: example-repo, purpose: example-repo-key }
        allowedModels: [claude-sonnet-4-6]           # the models this key may call

Admit the key on each model it may call, with a second rule next to the repository rule:

    authorization:
      rules:
      - allow: 'jwt.repository == "ALLOWED_REPO"'
      - allow: 'apiKey.purpose == "example-repo-key"'

• Strip the caller's key before the upstream, as you strip the bearer. The key must never reach the model provider. Keep requestHeaders.remove: [x-api-key] on every model, never forward the caller's Authorization header, and check with the step 9 custody models (with the temporary test-key grants above) that the upstream sees only the gateway's own key.
• A gateway that serves only OIDC keeps strict jwtAuth. Leave out the apiKey policy, keep mode: strict as in step 3, and the gateway stays as tested above.
• Serving both modes on one gateway weakens authentication. jwtAuth must be permissive so that a key, which is not a JWT, gets past it, and apiKey must be optional so that a request authenticated by its token, which carries no key, gets past the key check. A request with no credential at all then passes both and reaches the per-model authorization rules. Those rules are the only thing that refuses it, so every model must have them, and none may admit an anonymous caller. Where you can, run a separate gateway for api-key repositories, or keep the gateway OIDC-only with strict jwtAuth.

4. Validate the config locally

mkdir -p v/config v/upstream && cp config.yaml v/config/ && chmod -R a+rX v
podman run --rm -v "$PWD/v/config:/etc/agw-config:ro" -v "$PWD/v/upstream:/etc/agw-upstream:ro" \
  ghcr.io/agentgateway/agentgateway:v1.7.0 -f /etc/agw-config/config.yaml --validate-only

Configuration is valid!

Both the config above and the version with the custody-check models from step 9 returned this. The container runs as a non-root user, so the mounted files must be world-readable. On an Apple silicon Mac, add --platform linux/arm64; amd64 emulation hangs. If you configured an apiKey.file, put a placeholder file at that path in v/upstream/ first.

5. Store the config and the upstream key in Secret Manager

gcloud secrets create gateway-config --project "$PROJECT" --replication-policy automatic \
  --data-file=config.yaml
gcloud secrets create gateway-upstream-key --project "$PROJECT" --replication-policy automatic \
  --data-file=upstream-key
for s in gateway-config gateway-upstream-key; do
  gcloud secrets add-iam-policy-binding "$s" --project "$PROJECT" --member "serviceAccount:$SA" \
    --role roles/secretmanager.secretAccessor --format='value(bindings)'
done

Created version [1] of the secret [gateway-config].
Created version [1] of the secret [gateway-upstream-key].
Updated IAM policy for secret [gateway-config].
{'members': ['serviceAccount:gateway-runtime@example-project.iam.gserviceaccount.com'], 'role': 'roles/secretmanager.secretAccessor'}
Updated IAM policy for secret [gateway-upstream-key].
{'members': ['serviceAccount:gateway-runtime@example-project.iam.gserviceaccount.com'], 'role': 'roles/secretmanager.secretAccessor'}

upstream-key is the file holding your provider key, or the throwaway key from step 9. Create it with the provider's console or CLI, and delete the local copy once the secret exists.

If every model uses the service account (Vertex only, no custody check), skip gateway-upstream-key: create and bind only gateway-config, and in step 6 drop ,/etc/agw-upstream/key=gateway-upstream-key:latest from --set-secrets.

6. Deploy to Cloud Run

gcloud run deploy "$SERVICE" --project "$PROJECT" --region "$REGION" \
  --image "$REGION-docker.pkg.dev/$PROJECT/$AR_REPO/agentgateway:v1.7.0" \
  --service-account "$SA" --port 8080 \
  --allow-unauthenticated --no-invoker-iam-check \
  --cpu 1 --memory 512Mi --max-instances 1 \
  --args=-f,/etc/agw-config/config.yaml \
  --set-secrets "/etc/agw-config/config.yaml=gateway-config:latest,/etc/agw-upstream/key=gateway-upstream-key:latest"

Deploying container to Cloud Run service [inference-gateway] in project [example-project] region [us-east5]
Deploying new service...
Setting IAM Policy...........done
Creating Revision...................................................done
Routing traffic.....done
Done.
Service [inference-gateway] revision [inference-gateway-00001-abc] has been deployed and is serving 100 percent of traffic.
Service URL: https://gateway.example.com

• --no-invoker-iam-check is required. The gateway does its own authentication. Without this flag, Cloud Run's front end tries to verify every Authorization: Bearer as a Google token and rejects a GitHub token with an HTML 401 before it reaches the gateway, even with --allow-unauthenticated. This is masked for the first minutes after a deploy while IAM settles.
• Each secret gets its own directory. Cloud Run mounts one secret per directory.
• latest is for the first deploy only. This revision reads whichever config version is newest, so it is not a rollback target. Every later change mounts a version number; see Change the config.
• Size the service for your load. The values above are what this walkthrough ran with. See Watch health and capacity for what to measure.

Check that the gateway, not Cloud Run, answers a non-Google bearer:

curl -s -i https://gateway.example.com/v1/models -H 'authorization: Bearer not-a-google-token'  # gitleaks:allow

HTTP/2 401
content-type: text/plain
server: Google Frontend
content-length: 21

authentication failed

A 401 with a text/plain body comes from agentgateway. It returns authentication failed for every jwtAuth refusal, and writes the reason to its access log (see Troubleshooting). A text/html body means the invoker check is still on.

7. Know what a healthy start looks like

gcloud logging read "resource.type=\"cloud_run_revision\" AND resource.labels.service_name=\"$SERVICE\" AND textPayload:\"started\"" \
  --project "$PROJECT" --freshness 30m --limit 5 --format 'value(textPayload)'

2026-10-09T22:47:31.427378Z	info	proxy::gateway	started bind	bind="bind/8080"
2026-10-09T22:47:31.427319Z	info	proxy::gateway	started bind	bind="bind/9000"
Starting new instance. Reason: DEPLOYMENT_ROLLOUT - Instance started due to traffic shifting between revisions due to deployment, traffic split adjustment, or deployment health check.

bind/8080 is the gateway. bind/9000 appears only with the custody-check listener from step 9. This output was recorded on v1.6.0. v1.7.0 also logs two warnings before started bind when the config is mounted read-only, as it is on Cloud Run. This is from a local run of the v1.7.0 image with the step 3 config:

warn	config	failed to write generated session key	err=failed to write to file `/etc/agw-config/config.yaml`: Read-only file system (os error 30) path=/etc/agw-config/config.yaml
warn	config	no session key configured; MCP session IDs are not encrypted. Set SESSION_KEY or config.session.key
info	proxy::gateway	started bind	bind="bind/8080"

Both are harmless for this setup: the session key only encrypts MCP session IDs, and this gateway serves no MCP. If you add MCP, set SESSION_KEY or config.session.key.

8. Check the gateway from GitHub Actions

Copy gateway-check.yml to .github/workflows/ in an allowed repository, set a repository variable GATEWAY_URL to the service URL, and run it. It mints a token for your audience and checks one prompt per API, then the refusals. The job needs id-token: write to mint the token. Set AUDIENCE to your audience:

permissions:
  id-token: write
  contents: read
jobs:
  check:
    runs-on: ubuntu-latest
    timeout-minutes: 10
    env:
      GATEWAY_URL: ${{ vars.GATEWAY_URL }}
      AUDIENCE: fullsend-inference

The job's one step defines two shell helpers. tok fetches a token for an audience, and call prints each call's label, HTTP status and the first 700 bytes of the response body. It then calls each API, the custody-check models and the refusal cases, and finally checks whether any response body contains the token.

gh variable set GATEWAY_URL -R example-org/example-repo -b https://gateway.example.com
gh workflow run gateway-check.yml -R example-org/example-repo
id=$(gh run list -R example-org/example-repo -w gateway-check.yml -L1 --json databaseId -q '.[0].databaseId')
gh run watch "$id" -R example-org/example-repo
gh run view "$id" -R example-org/example-repo --log

The echo calls need the custody-check models from step 9; drop those lines if you skip it.

Output from the allowed repository. The claims line comes from a GitHub Actions run. The rest of this step and the custody output in step 9 come from a local run of the v1.7.0 image. That run used this page's config, Vertex AI credentials, and tokens carrying the same claims, signed by a local key set that jwtAuth was pointed at:

== claims (non-secret)
{"iss":"https://token.actions.githubusercontent.com","aud":"fullsend-inference","repository":"example-org/example-repo","ttl":300}

== GET /v1/models
HTTP 200 {"object":"list","data":[{"id":"claude-sonnet-4-6","object":"model","created":...,"owned_by":"gcp.vertex_ai"},{"id":"gemini-2.5-flash","object":"model","created":1750118400,"owned_by":"gcp.vertex_ai","name":"Gemini 2.5 Flash","context_window":1048576,"context_length":1048576,"max_input_tokens":1048576,"max_tokens":65536,"architecture":{...},"capabilities":{...},"supported_parameters":[...]},{"id":"echo-noremove",...},{"id":"echo",...}]}

== POST /v1/messages (claude-sonnet-4-6)
HTTP 200 {"id":"msg_vrtx_...","type":"message","role":"assistant","model":"claude-sonnet-4-6","stop_reason":"end_turn","stop_sequence":null,"usage":{"input_tokens":13,"output_tokens":5,"cache_creation_input_tokens":0,"cache_read_input_tokens":0,"cache_creation":{"ephemeral_5m_input_tokens":0,"ephemeral_1h_input_tokens":0}},"content":[{"text":"pong","type":"text"}],"container":null,"stop_details":null}

== POST /v1/chat/completions (gemini-2.5-flash)
HTTP 200 {"model":"gemini-2.5-flash","service_tier":"default","usage":{"prompt_tokens":5,"completion_tokens":22,"total_tokens":27,"completion_tokens_details":{"reasoning_tokens":21}},"choices":[{"message":{"content":"pong","role":"assistant"},"index":0,"finish_reason":"stop"}],"id":"...","created":...,"object":"chat.completion"}

== POST /v1/responses (echo)
HTTP 200 {"id":"resp_...","status":"completed","output":[{"type":"message","content":[{"type":"output_text","annotations":[],"logprobs":null,"text":"{\"authorization\":\"GATEWAY_KEY\",\"x_api_key_present\":false,\"canary_present\":false}"}], ...

== POST /v1/responses (claude-sonnet-4-6)
HTTP 400 failed to process LLM request: unsupported conversion: from Responses to provider gcp.vertex_ai (supported: [AnthropicMessages])

== negative: token minted for another audience
HTTP 401 authentication failed

== negative: token only in x-api-key
HTTP 401 authentication failed

== negative: no credential
HTTP 401 authentication failed

== does any body contain the token? 0

• The token lives 300 seconds (ttl:300). The fullsend runner re-fetches it before it expires, so runs longer than that keep working.

• /v1/responses works only for upstreams that accept it. The gateway translates Responses to Chat Completions for a Chat-only upstream (the echo call), but not to Vertex Claude. Map Claude models to anthropic-messages in the runner's model list, and Gemini models to openai-completions.

• Every 401 body reads authentication failed. The reason is in the gateway's access log, as error="authentication failure: ..." with reason=JwtAuth; see Troubleshooting.

• /v1/models lists each model with its provider in owned_by (gcp.vertex_ai here). Models in agentgateway's catalog, such as Gemini, also carry fields like context_window and max_tokens.

• The gateway's log line for each call names the model, the upstream and the caller. The repository appears in jwt.sub. Its format depends on the repository: one that uses GitHub's immutable subject claims logs repo:example-org@1111/example-repo@2222:ref:... instead. Write authorization rules on jwt.repository and the id claims, never by parsing jwt.sub:

  info	request ... endpoint=aiplatform.googleapis.com:443 ... http.path=/v1/chat/completions http.status=200 jwt.sub=repo:example-org/example-repo:ref:refs/heads/main protocol=llm gen_ai.operation.name=chat gen_ai.provider.name=gcp.vertex_ai gen_ai.request.model=gemini-2.5-flash ...

Check that another repository is refused

Run the same workflow, with the same audience, in a repository that is not in the allow rules:

== claims (non-secret)
{"iss":"https://token.actions.githubusercontent.com","aud":"fullsend-inference","repository":"example-org/other-repo","ttl":300}

== GET /v1/models
HTTP 200 {"object":"list","data":[]}

== POST /v1/messages (claude-sonnet-4-6)
HTTP 403 {"error":{"message":"Model authorization denied","type":"invalid_request_error","code":"model_authorization_denied"}}

== POST /v1/chat/completions (gemini-2.5-flash)
HTTP 403 {"error":{"message":"Model authorization denied","type":"invalid_request_error","code":"model_authorization_denied"}}

A valid GitHub token from the wrong repository gets an empty model list and 403 on every model. Every other call in that run also returned 403 model_authorization_denied, and the audience and missing-token negatives returned the same 401s as above.

9. Check that no caller credential reaches the upstream

Order: this step changes the gateway config and adds a throwaway upstream key that the echo listener compares against. Apply both before steps 4–6 (validate, store, deploy), so the deployed service mounts that key and step 8's custody check can pass.

This optional check adds two models whose upstream is a second listener inside the same container. It answers every request with what it received: whether authorization equals the gateway's own upstream key (GATEWAY_KEY), and whether x-api-key and a canary header arrived. Its key is read from the mounted gateway-upstream-key file, so it also proves the file-mounted key works.

Append the binds block in echo-upstream.yaml to config.tmpl.yaml, at the top level. It defines the echo listener on port 9000 as a directResponse route, whose CEL body reports what the request carried. Then add the two models under llm.models:

  - name: echo
    provider: { custom: { formats: [ { type: completions } ] } }
    params: { baseUrl: http://127.0.0.1:9000/v1, apiKey: { file: /etc/agw-upstream/key } }
    requestHeaders: { remove: [x-api-key] }
    authorization:
      rules:
      - allow: 'jwt.repository == "ALLOWED_REPO"'
  - name: echo-noremove
    provider: { custom: { formats: [ { type: completions } ] } }
    params: { baseUrl: http://127.0.0.1:9000/v1, apiKey: { file: /etc/agw-upstream/key } }
    authorization:
      rules:
      - allow: 'jwt.repository == "ALLOWED_REPO"'

Generate a throwaway key for the check and put it in both places. The config is itself stored as a secret:

openssl rand -hex 16 > upstream-key
sed -e "s/VERTEX_PROJECT/$PROJECT/g" -e "s/AUDIENCE/$AUDIENCE/" -e "s/ECHO_KEY/$(cat upstream-key)/" \
    -e "s#ALLOWED_REPO#example-org/example-repo#g" config.tmpl.yaml > config.yaml

The workflow in step 8 sends a valid bearer plus x-api-key: canary and x-canary: 1:

== custody: echo, requestHeaders.remove [x-api-key]
HTTP 200 {... "content":"{\"authorization\":\"GATEWAY_KEY\",\"x_api_key_present\":false,\"canary_present\":true}" ...}

== custody: echo-noremove, no requestHeaders
HTTP 200 {... "content":"{\"authorization\":\"GATEWAY_KEY\",\"x_api_key_present\":true,\"canary_present\":true}" ...}

• authorization is GATEWAY_KEY in both: the job's token was replaced by the gateway's own key.
• x-api-key reached the upstream only on the model without requestHeaders.remove.
• The canary reached the upstream in both: other caller headers pass through unchanged.

Remove the two echo models and the binds block once the check passes, then roll the change out as described in Change the config.

Serve the gateway under a path prefix

Not executed in this walkthrough: the steps above give the gateway its own origin. This section describes what fullsend expects when it shares one; the ingress configuration depends on your load balancer and is not shown.

A gateway behind a shared ingress often gets a path prefix rather than its own host name, for example https://ingress.example.com/team-a/v1/.... Fullsend supports this: hand out the gateway URL with the prefix and without the trailing /v1:

Gateway serves	Gateway URL to hand out
https://gateway.example.com/v1/...	https://gateway.example.com
https://ingress.example.com/team-a/v1/...	https://ingress.example.com/team-a
https://ingress.example.com/api/v1/...	https://ingress.example.com/api

The runner then calls <url>/v1/responses, <url>/v1/messages and <url>/v1/chat/completions, and the run's egress profile binds the credential to <prefix>/v1/** on the ingress host only, not to the whole host, so the job's token never reaches another service behind the same ingress.

• The ingress must forward <prefix>/v1/... to the gateway's /v1/... (strip the prefix) unless the gateway itself serves the prefixed paths. Check with the step 8 workflow, setting GATEWAY_URL to the URL with the prefix: GET /v1/models then calls <prefix>/v1/models.
• The prefix must be a clean path: no .., . or empty segments, no percent-encoding, and only letters, digits, -, ., _ and ~ in each segment. The host must still serve https on port 443.
• One gateway per repository still applies: a repository's inference.gateway.url names one host and prefix.

Header custody for an x-api-key gateway

Not executed in this walkthrough: the agentgateway setup above reads the caller's credential from Authorization and needs none of this. This section describes what fullsend expects of a gateway that reads it from x-api-key; the configuration depends on the gateway and is not shown.

Some gateways read the caller's key on the Messages API (/v1/messages) from x-api-key, the header Anthropic documents for API keys on that API. A repository in the api-key mode can send its key there by setting messages_auth_header: x-api-key in its inference.gateway block. Then:

• Claude Code, and pi models with api: anthropic-messages, send the key as x-api-key and set no credential in Authorization.
• pi openai-responses and openai-completions models, and codex, keep Authorization: Bearer on /v1/responses and /v1/chat/completions.
• The setting is refused with auth: oidc, and fullsend never tries one header and retries in the other after a 401.

On such a gateway, x-api-key is first the caller's credential and then the header an Anthropic upstream reads for the provider key. The gateway must therefore:

• authenticate the caller from x-api-key, then remove that header before the request goes upstream;
• set the upstream's x-api-key from its own provider key by replacing the header, never by appending to it, so the caller's value cannot reach the upstream as a second header or as the first of two;
• still remove any caller Authorization header, on every path and toward every upstream;
• pass the step 9 custody check with a canary value in the caller's x-api-key on /v1/messages, and show that the upstream sees only the gateway's own key.

Change the config

Not executed in this walkthrough: the service was deleted after the checks, so these commands were not run here.

The first time only, pin the running config first. The step 6 revision mounts latest, and Cloud Run reads a mounted secret when the file is read, not when the revision is created. Adding a version would change what that revision loads on its next start, and leave no revision to roll back to. Roll a revision that mounts the running version, 1, before you add a new one:

gcloud run services update "$SERVICE" --project "$PROJECT" --region "$REGION" \
  --update-secrets "/etc/agw-config/config.yaml=gateway-config:1"

For every change, add config.yaml as a new secret version, roll a revision that mounts that version by number, and send all traffic to it:

v=$(gcloud secrets versions add gateway-config --project "$PROJECT" --data-file=config.yaml \
  --format='value(name.basename())')
gcloud run services update "$SERVICE" --project "$PROJECT" --region "$REGION" \
  --update-secrets "/etc/agw-config/config.yaml=gateway-config:$v"
gcloud run services update-traffic "$SERVICE" --project "$PROJECT" --region "$REGION" --to-latest

Then rerun the workflow from step 8. The --to-latest step matters only after a rollback, which pins traffic to an older revision; otherwise it changes nothing.

• Mount a version number, not latest. A revision that mounts a numbered version keeps the config it was deployed with. A revision that mounts latest reads the newest version instead, so it cannot restore an older config.

• To roll back, list the recent revisions, then send all traffic to the previous one:

  gcloud run revisions list --service "$SERVICE" --project "$PROJECT" --region "$REGION" --limit 3
  gcloud run services update-traffic "$SERVICE" --project "$PROJECT" --region "$REGION" \
    --to-revisions <previous-revision>=100

• Keep the older version enabled while any revision you may roll back to mounts it. If the version is disabled, that revision cannot read its config, and the gateway exits at startup.

Remove a repository

Not executed in this walkthrough: the service was deleted after the checks, so no repository was removed from a running gateway here.

1. Remove every grant the repository has in the gateway config (step 3):

   • its jwt.repository rule on every model. Where a rule allows several repositories with ||, delete only that repository's term;
   • if you issued it a gateway key (auth: api-key), its apiKey.purpose rule on every model and its entry under apiKey.keys. Removing only the jwt.repository rule leaves the key working.

   If that was a model's last allow rule, remove the model too. A model left with no authorization block or an empty rules list lets every caller through (see step 3).

2. Roll the change out as in Change the config.

3. Check that the repository is refused:

   Mode	Check	Refused looks like
   oidc	The step 8 workflow in that repository, or fullsend inference gateway status in a job there	GET /v1/models returns {"object":"list","data":[]} or 403 model_authorization_denied, as in Check that another repository is refused. status prints Gateway authenticated the assertion, but no models are authorised for <owner/repo> or Gateway refused the assertion (HTTP 403)
   api-key	GET /v1/models with Authorization: Bearer <old key> from your own machine (status stops at the config checks in this mode)	401 authentication failed, because the key's apiKey.keys entry is gone. Then ask the repo admin to delete the FULLSEND_INFERENCE_GATEWAY_API_KEY secret

Watch health and capacity

Not executed in this walkthrough: the service was deleted after the checks.

• Every model call leaves one log line. It carries http.status, the calling repository in jwt.sub, the model, token counts (gen_ai.usage.*) and duration. The sample under step 8 shows the shape. Filter it for 429 and 5xx. A 429 with reason=RateLimit is the gateway's own localRateLimit (below); any other 429 or 5xx usually comes from the upstream.
• Cloud Run's own 429s never reach the gateway. When every request slot on every instance up to --max-instances is busy, Cloud Run refuses the request itself. It shows only in Cloud Run's request log (run.googleapis.com/requests), not in the gateway's lines. Step 6 sets no --concurrency, so Cloud Run's default applies: 80 slots per vCPU, so 80 with --cpu 1.
• Cap bursts with agentgateway's localRateLimit (requests and tokens per fillInterval). Set its key so that callers outside your allow rule draw from a separate bucket. Any GitHub Actions job can mint a token for your audience (only pull-request runs from forks cannot: they get no id-token: write). That token passes jwtAuth and reaches the rate limit before the model's rule refuses it, so with one shared bucket an outside repository can use up your repositories' share. The caps apply per instance and reset on each new instance, so they are a burst guard, not a spend limit. Set a budget alert at the model provider as well.
• Scale from what you measure. Raise the rate caps on reason=RateLimit 429s. Raise --concurrency on Cloud Run 429s; Cloud Run sets the default only when it creates the service, so raise --concurrency yourself after a CPU change. Raise CPU or memory when the instance runs hot. If you raise --max-instances, divide the rate caps by the same factor: each instance has its own buckets. A streamed model call holds its request slot for the whole response.

Delete the gateway

gcloud run services delete "$SERVICE" --project "$PROJECT" --region "$REGION" --quiet
gcloud secrets delete gateway-config --project "$PROJECT" --quiet
gcloud secrets delete gateway-upstream-key --project "$PROJECT" --quiet
gcloud projects remove-iam-policy-binding "$PROJECT" --member "serviceAccount:$SA" \
  --role roles/aiplatform.user --condition=None --format='value(etag)'
gcloud iam service-accounts delete "$SA" --project "$PROJECT" --quiet
gcloud artifacts repositories delete "$AR_REPO" --project "$PROJECT" --location "$REGION" --quiet

Deleting [inference-gateway]...
...........................done.
Deleted service [inference-gateway].
Deleted secret [gateway-config].
Deleted secret [gateway-upstream-key].
Updated IAM policy for project [example-project].
BwZd...
deleted service account [gateway-runtime@example-project.iam.gserviceaccount.com]
Delete request issued for: [example-images]
Waiting for operation [projects/example-project/locations/us-east5/operations/...] to complete...
.....done.
Deleted repository [example-images].

Remove the project binding before deleting the service account. Deleting a secret also removes its accessor binding. Afterwards, describe on each resource returns NOT_FOUND, and the service URL returns 404.

Troubleshooting

Every jwtAuth refusal returns 401 with the body authentication failed. The reason is in the gateway's access log line for that request, as error="authentication failure: ..." with reason=JwtAuth. Find it with:

gcloud logging read "resource.labels.service_name=\"$SERVICE\" AND textPayload:\"reason=JwtAuth\"" \
  --project "$PROJECT" --freshness 1h --limit 5 --format 'value(textPayload)'

For the step 6 check's non-JWT bearer, on v1.7.0:

2026-10-30T15:07:24.725330Z	error	request gateway=default/default listener=default route=internal/llm:request ... http.method=GET ... http.path=/v1/models http.version=HTTP/1.1 http.status=401 ... protocol=http error="authentication failure: the token header is malformed: Error(InvalidToken)" reason=JwtAuth duration=0ms

The rows below are keyed on that error= text.

You see	Cause	Action
401 text/html … Your client does not have permission to the requested URL	Cloud Run's invoker IAM check rejected a non-Google bearer before it reached the gateway.	gcloud run services update "$SERVICE" --project "$PROJECT" --region "$REGION" --no-invoker-iam-check
401; log error="authentication failure: the token is invalid or malformed: Error(InvalidAudience)"	The job minted its token for a different audience.	Make the runner's configured audience equal the gateway's audiences value, character for character.
401; log error="authentication failure: no bearer token found"	The request carried no Authorization: Bearer, or sent the token only in x-api-key.	agentgateway reads the token from Authorization: Bearer by default (configurable with location). For pi, set authHeader for anthropic-messages to authorization; the fullsend runner does this for you.
401; log error="authentication failure: the token header is malformed: Error(InvalidToken)"	The bearer is not a JWT.	Check the runner fetched an OIDC token (id-token: write permission) rather than passing another secret.
401; log error ends in Error(ExpiredSignature) (seen in an earlier live test, not this walkthrough)	The token is past its exp (GitHub tokens live exp − iat, 300 s in practice, plus about 60 s of gateway leeway).	The runner must re-fetch the token before exp.
GET /v1/models returns {"object":"list","data":[]} and every model 403 model_authorization_denied	The token is valid but its repository matches no authorization rule.	Add the repository to the rules (exact owner/name), then change the config.
400 failed to process LLM request: unsupported conversion: from Responses to provider gcp.vertex_ai (supported: [AnthropicMessages])	A Vertex Claude model was called on /v1/responses.	Map Claude models to anthropic-messages in the runner's model list.
Permission denied (os error 13) from --validate-only	The mounted file isn't readable by the container's non-root user.	chmod -R a+rX the mounted directories.
Gateway exits at startup after adding an apiKey.file model	The file it names is not mounted.	Mount the key secret at that path in its own directory (--set-secrets).

Related

• OpenAI Workload Identity: the route without a gateway, where the OIDC token is exchanged with OpenAI directly.
• pi-inference-gateway: the pi extension that calls the gateway, and its agentgateway guide.
• agentgateway v1.7.0 configuration reference: schema/config.md in the agentgateway repository.
• ADR 0137: the inference gateway credential route, including the gateway-side requirements this page implements.