Inference API

Use the Truffle inference HTTP endpoints directly for model listing and chat completions

Your Truffle exposes an OpenAI-compatible inference API at /if2/v1/*. You can use it via the CLI or call the HTTP endpoints directly.

For the interactive CLI experience (truffile infer), see the CLI guide. This page covers the raw HTTP endpoints.

---

List Models

curl -sS http://<device-host>/if2/v1/models | python -m json.tool

Or via the CLI:

truffile models

---

Chat Completions

curl -sS http://<device-host>/if2/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<model-id-or-uuid>",
    "messages": [
      {"role": "user", "content": "Give me one sentence about Truffle."}
    ]
  }'

Request body

Field	Type	Description
model	string	Model ID or UUID (from /models)
messages	array	Conversation messages (role + content)
stream	bool	Stream response tokens via SSE (default false)
max_tokens	int	Maximum output tokens
temperature	float	Sampling temperature
top_p	float	Nucleus sampling parameter
reasoning	object	{"enabled": true} to enable reasoning output
tools	array	OpenAI-format tool definitions
tool_choice	string	"auto" to let the model decide

Streaming

Set "stream": true and optionally "stream_options": {"include_usage": true} to get token usage in the final event.

The response is an SSE stream of data: {...} lines, terminated by data: [DONE].

Each event contains a choices[0].delta object with incremental content, reasoning, and/or tool_calls.

Image input

Pass images as part of a message’s content array:

{
  "role": "user",
  "content": [
    {"type": "text", "text": "What is in this image?"},
    {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}
  ]
}

Supported image types: image/jpeg, image/png, image/bmp.

---

MCP Testing with the CLI

truffile infer doubles as an MCP test harness. Start your MCP server locally, then:

truffile infer
> /mcp connect http://127.0.0.1:8000/mcp
> /mcp tools
> use the get_markets tool to show me current markets

See the CLI guide for full details.