Using Inferrs with OpenClaw

Prerequisites

• ✓An Inferrs installation with the `inferrs` CLI available on your PATH.
• ✓A local model that Inferrs can serve, such as `google/gemma-4-E2B-it`.
• ✓An OpenClaw setup where you can edit the agents and models configuration and run `openclaw infer model run`.

Steps

1. 1

   Start Inferrs with a local model

   Start the Inferrs server and bind it to a host and port that OpenClaw can reach. 1:8080` using the `metal` device, which the later OpenClaw config expects.

   bashCopy

   inferrs serve google/gemma-4-E2B-it \
     --host 127.0.0.1 \
     --port 8080 \
     --device metal

2. 2

   Verify the Inferrs server is reachable

   Before touching OpenClaw, confirm that Inferrs is actually listening and exposing the OpenAI-compatible endpoints. These curl checks hit the health probe and list models; if either fails, fix Inferrs networking or model loading first.

   bashCopy

   curl http://127.0.0.1:8080/health
   curl http://127.0.0.1:8080/v1/models

3. 3

   Configure Inferrs as an OpenClaw provider

   Add a provider entry that points OpenClaw at your Inferrs `/v1` base URL and describes the model capabilities. requiresStringContent` so OpenClaw flattens content into plain strings for Inferrs.

   jsonCopy

   {
     agents: {
       defaults: {
         model: { primary: "inferrs/google/gemma-4-E2B-it" },
         models: {
           "inferrs/google/gemma-4-E2B-it": {
             alias: "Gemma 4 (inferrs)",
           },
         },
       },
     },
     models: {
       mode: "merge",
       providers: {
         inferrs: {
           baseUrl: "http://127.0.0.1:8080/v1",
           apiKey: "inferrs-local",
           api: "openai-completions",
           models: [
             {
               id: "google/gemma-4-E2B-it",
               name: "Gemma 4 E2B (inferrs)",
               reasoning: false,
               input: ["text"],
               cost: { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 },
               contextWindow: 131072,
               maxTokens: 4096,
               compat: {
                 requiresStringContent: true,
               },
             },
           ],
         },
       },
     },
   }

4. 4

   Run a manual Inferrs chat completion smoke test

   Test Inferrs directly with a minimal `/v1/chat/completions` request to confirm the model responds before involving OpenClaw. This isolates Inferrs issues from OpenClaw configuration problems and gives you a known-good baseline.

   bashCopy

   curl http://127.0.0.1:8080/v1/chat/completions \
     -H 'content-type: application/json' \
     -d '{"model":"google/gemma-4-E2B-it","messages":[{"role":"user","content":"What is 2 + 2?"}],"stream":false}'

5. 5

   Run an OpenClaw model inference against Inferrs

   Once the direct curl call works, exercise the full OpenClaw → Inferrs path. This command uses the configured `inferrs/google/gemma-4-E2B-it` model and returns JSON so you can see exactly what OpenClaw got back.

   bashCopy

   openclaw infer model run \
     --model inferrs/google/gemma-4-E2B-it \
     --prompt "What is 2 + 2? Reply with one short sentence." \
     --json

6. 6

   Tune Inferrs compatibility flags for Gemma and tools

   If you see schema errors or tool-related crashes, adjust the `compat` block for your Inferrs model. Disabling tools with `supportsTools: false` can help when Gemma accepts small direct calls but fails on full agent turns.

   textCopy

   compat: {
     requiresStringContent: true,
     supportsTools: false
   }

Configuration

Troubleshooting

compat: {
  requiresStringContent: true
}

compat: {
  requiresStringContent: true,
  supportsTools: false
}

Frequently asked questions