> ## Documentation Index
> Fetch the complete documentation index at: https://docs.truffle.net/llms.txt
> Use this file to discover all available pages before exploring further.

# Inference API

> Use the Truffle inference HTTP endpoints directly for model listing and chat completions

Your Truffle exposes an OpenAI-compatible inference API at `/if2/v1/*`. You can use it via the CLI or call the HTTP endpoints directly.

<Note>
  For the interactive CLI experience (`truffile infer`), see the [CLI guide](/sdk/cli#infer-mode). This page covers the raw HTTP endpoints.
</Note>

***

## List Models

```bash theme={null}
curl -sS http://<device-host>/if2/v1/models | python -m json.tool
```

Or via the CLI:

```bash theme={null}
truffile models
```

***

## Chat Completions

```bash theme={null}
curl -sS http://<device-host>/if2/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<model-id-or-uuid>",
    "messages": [
      {"role": "user", "content": "Give me one sentence about Truffle."}
    ]
  }'
```

### Request body

| Field         | Type     | Description                                      |
| ------------- | -------- | ------------------------------------------------ |
| `model`       | `string` | Model ID or UUID (from `/models`)                |
| `messages`    | `array`  | Conversation messages (`role` + `content`)       |
| `stream`      | `bool`   | Stream response tokens via SSE (default `false`) |
| `max_tokens`  | `int`    | Maximum output tokens                            |
| `temperature` | `float`  | Sampling temperature                             |
| `top_p`       | `float`  | Nucleus sampling parameter                       |
| `reasoning`   | `object` | `{"enabled": true}` to enable reasoning output   |
| `tools`       | `array`  | OpenAI-format tool definitions                   |
| `tool_choice` | `string` | `"auto"` to let the model decide                 |

### Streaming

Set `"stream": true` and optionally `"stream_options": {"include_usage": true}` to get token usage in the final event.

The response is an SSE stream of `data: {...}` lines, terminated by `data: [DONE]`.

Each event contains a `choices[0].delta` object with incremental `content`, `reasoning`, and/or `tool_calls`.

### Image input

Pass images as part of a message's `content` array:

```json theme={null}
{
  "role": "user",
  "content": [
    {"type": "text", "text": "What is in this image?"},
    {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}
  ]
}
```

Supported image types: `image/jpeg`, `image/png`, `image/bmp`.

***

## MCP Testing with the CLI

`truffile infer` doubles as an MCP test harness. Start your MCP server locally, then:

```
truffile infer
> /mcp connect http://127.0.0.1:8000/mcp
> /mcp tools
> use the get_markets tool to show me current markets
```

See the [CLI guide](/sdk/cli#mcp-integration) for full details.