# llm_ollama

A RocketRide LLM node that routes pipeline traffic through a locally-hosted Ollama server.

## What it does

Provides text generation against an Ollama server running on your own hardware. The node acts as an `llm` invoke connection for agents and other nodes that need an LLM, and can also be driven directly via its `questions` / `answers` lane pair. Because all inference happens on-premise, no external API key is required, making it a natural fit for privacy-sensitive or air-gapped deployments.

Internally, the node talks to Ollama through its **OpenAI-compatible `/v1` endpoint** using **langchain-openai** (`ChatOpenAI`), with temperature fixed at `0`. If the configured `serverbase` URL does not end in `/v1`, the node appends it automatically, so both `http://localhost:11434` and `http://localhost:11434/v1` are accepted. The OpenAI client requires a non-empty API key field; the node sends the placeholder string `dummy-key`, which Ollama ignores.

---

## Configuration

### Lanes

| Lane in     | Lane out  | Description                                          |
| ----------- | --------- | ---------------------------------------------------- |
| `questions` | `answers` | Send a question directly, receive a generated answer |

### Fields

Pick a profile from the dropdown; the profile pre-fills `model`, `serverbase`, and `modelTotalTokens`. All three fields are individually overridable when using the `custom` profile.

| Field | Type | Description |
|---|---|---|
| `model` | string | Ollama model |
| `modelTotalTokens` | number | Total Tokens |
| `profile` | string | Default "llama3_3". LLM model |

---

## Profiles

**Llama**

| Profile               | Model             | Context    |
| --------------------- | ----------------- | ---------- |
| Llama 4 Latest        | `llama4:latest`   | 10,000,000 |
| Llama 3.3 _(default)_ | `llama3.3:latest` | 128,000    |
| Llama 3.1 405B        | `llama3.1:405b`   | 128,000    |
| Llama 3.1 70B         | `llama3.1:70b`    | 128,000    |
| Llama 3.1 8B          | `llama3.1:8b`     | 128,000    |
| Llama 3.2 3B          | `llama3.2`        | 128,000    |
| Llama 3.2 1B          | `llama3.2:1b`     | 128,000    |

**Qwen**

| Profile       | Model          | Context |
| ------------- | -------------- | ------- |
| Qwen 3 Latest | `qwen3:latest` | 128,000 |
| Qwen 2.5 72B  | `qwen2.5:72b`  | 128,000 |
| Qwen 2.5 32B  | `qwen2.5:32b`  | 128,000 |
| Qwen 2.5 14B  | `qwen2.5:14b`  | 128,000 |
| Qwen 2.5 7B   | `qwen2.5`      | 128,000 |
| Qwen 2.5 3B   | `qwen2.5:3b`   | 128,000 |
| Qwen 2.5 1.5B | `qwen2.5:1.5b` | 128,000 |
| Qwen 2.5 0.5B | `qwen2.5:0.5b` | 128,000 |

**DeepSeek**

| Profile          | Model              | Context |
| ---------------- | ------------------ | ------- |
| DeepSeek R1 671B | `deepseek-r1:671b` | 128,000 |
| DeepSeek R1 32B  | `deepseek-r1:32b`  | 128,000 |
| DeepSeek R1 14B  | `deepseek-r1:14b`  | 128,000 |
| DeepSeek R1 7B   | `deepseek-r1:7b`   | 128,000 |
| DeepSeek R1 1.5B | `deepseek-r1:1.5b` | 128,000 |

**Other**

| Profile    | Model     | Context |
| ---------- | --------- | ------- |
| Phi 4 14B  | `phi4`    | 16,000  |
| Mistral 7B | `mistral` | 32,000  |

**Custom**: supply any Ollama model tag, context token count, and server URL directly. The default context for a new custom profile is 16,385 tokens until you change it.

---

## Upstream docs

- [Ollama documentation](https://docs.ollama.com/)
- [Ollama model library](https://ollama.com/library)

---

<!-- ROCKETRIDE:GENERATED:PARAMS START -->
<!-- Generated by nodes:docs-generate. Do not edit by hand. -->

## Schema

| Field | Type | Description | Default |
|---|---|---|---|
| `model` | `string` | **Model**<br/>Ollama model |  |
| `modelTotalTokens` | `number` | **Tokens**<br/>Total Tokens |  |
| `ollama.profile` | `string` | **Model**<br/>LLM model | `"llama3_3"` |

## Dependencies

- `langchain-openai`
- `langchain-core`
- `langchain`

## Source

[<svg viewBox="0 0 16 16" width="15" height="15" fill="currentColor" aria-hidden="true" style="vertical-align:-0.15em;margin-right:0.35em"><path d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27.68 0 1.36.09 2 .27 1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.013 8.013 0 0016 8c0-4.42-3.58-8-8-8z"/></svg> View source](https://github.com/rocketride-org/rocketride-server/tree/develop/nodes/src/nodes/llm_ollama)
<!-- ROCKETRIDE:GENERATED:PARAMS END -->