> ## Documentation Index
> Fetch the complete documentation index at: https://koharu.rs/llms.txt
> Use this file to discover all available pages before exploring further.

# Runtimes, Models, and Hardware Acceleration in Koharu

> Learn how Koharu selects GPU backends, which drivers you need for CUDA, ROCm, and Metal, and where downloaded model files are cached on disk.

Koharu automatically detects your hardware at startup and selects the fastest available compute backend for its ML stack — text detection, OCR, translation, and inpainting all share a single chosen device. You do not need to configure anything manually in most cases, but understanding the selection order and driver requirements helps you get the best performance and troubleshoot issues when they arise.

## Runtime selection order

Koharu evaluates backends in the following priority order and uses the first one that is available and fully initialized:

| Priority | Runtime        | Platforms                |
| -------- | -------------- | ------------------------ |
| 1        | **Metal**      | macOS Apple Silicon      |
| 2        | **CUDA**       | Windows, Linux           |
| 3        | **ROCm / HIP** | Windows, Linux           |
| 4        | **Vulkan**     | Windows, Linux           |
| 5        | **CPU**        | All platforms (fallback) |

If a higher-priority backend is present but fails to initialize (for example, due to an outdated driver), Koharu falls back to the next option in the list. CPU fallback is always available and prioritizes correctness over speed.

## Platform and driver requirements

<Tabs>
  <Tab title="CUDA (NVIDIA)">
    Koharu supports **CUDA 13.0** on Windows and Linux.

    **Driver requirement**: CUDA 13.0 requires an NVIDIA driver from the **R580 series or newer**. Install the [latest NVIDIA driver](https://www.nvidia.com/en-us/drivers/) before launching Koharu. If your driver is older than R580, Koharu will skip CUDA and fall back to Vulkan or CPU.

    No separate CUDA Toolkit installation is required — Koharu bundles the runtime libraries it needs.
  </Tab>

  <Tab title="ROCm / HIP (AMD)">
    Koharu supports **ROCm/HIP** on Windows and Linux.

    **SDK requirement**: Download and install the [ROCm Core SDK with HIP](https://rocm.docs.amd.com/projects/HIP/en/latest/install/install.html) for your operating system before starting Koharu. Koharu expects the HIP runtime to be present on the system path.

    <Note>
      Not all AMD GPU generations are supported by ROCm on every platform. Consult AMD's ROCm compatibility matrix for your specific GPU model before installing.
    </Note>
  </Tab>

  <Tab title="Metal (Apple)">
    Metal is the primary backend on **macOS Apple Silicon** (M1 and later) and is selected automatically. No configuration or additional drivers are required — Metal is part of macOS.

    Intel Macs are not supported by Koharu's release builds.
  </Tab>

  <Tab title="Vulkan">
    Vulkan is used as a **cross-platform fallback** when CUDA and ROCm are both unavailable or fail to initialize. It works on **Windows** and **Linux** as long as a usable Vulkan device and an up-to-date graphics driver are present.

    If you are on a system without a dedicated GPU, Vulkan may still be available through a CPU-backed Vulkan implementation, though performance will be similar to the CPU runtime.
  </Tab>

  <Tab title="CPU">
    CPU is always available on every platform and requires no GPU SDK or driver beyond what the operating system already provides.

    CPU inference is significantly slower than accelerated backends for large models (particularly diffusion inpainting), but it is reliable and correct on all hardware. It is the right choice for systems without a supported GPU.
  </Tab>
</Tabs>

## Model downloads and the local cache

Koharu resolves runtime dependencies and model files on demand rather than bundling them in the installer. Three kinds of data are downloaded and cached:

* **Native runtime packages** — Torch, llama.cpp, and diffusion backend libraries compiled for your platform and selected runtime (e.g., CUDA or CPU)
* **Pinned model files** — detection, OCR, and inpainting model weights fetched from Hugging Face
* **GGUF quantizations** — local LLM files selected for the translation stage, also fetched from Hugging Face

All of these are stored under your operating system's standard cache directory:

```
<cache>/koharu/packages/
```

The exact path depends on your OS:

| Platform | Cache path                         |
| -------- | ---------------------------------- |
| Windows  | `%LOCALAPPDATA%\koharu\packages`   |
| macOS    | `~/Library/Caches/koharu/packages` |
| Linux    | `~/.cache/koharu/packages`         |

Project data (your `.khrproj` files and imported pages) is stored separately under `Documents/Koharu`. Application settings are stored in `~/.koharu/config.toml`.

Downloads are staged before being published to the cache. A failed or interrupted download will be retried automatically on the next launch or the next time the relevant model is used — you do not need to manually clear anything.

<Note>
  **WebGPU is required for the editor canvas.** Even when ML inference falls back to CPU, Koharu's canvas and editor interface use WebGPU inside an embedded webview. A working WebGPU adapter and an up-to-date graphics driver are required to launch the editor at all — not just for accelerated ML.
</Note>

## Choosing a quantization size

Koharu supports multiple GGUF quantization levels for local translation models. Smaller quantizations use less memory and load faster, usually with a small quality trade-off. Larger quantizations produce higher-quality translations but require more VRAM or system RAM.

<Tip>
  Start with a mid-size quantization (such as Q4 or Q5) rather than the largest file your disk can hold. You can always switch to a larger quantization later once you have a sense of the quality difference for your content.
</Tip>

## Troubleshooting startup and download failures

If Koharu fails to initialize a runtime or stalls on a model download, work through these steps:

1. **Check network access** — Koharu fetches packages from GitHub release assets and model weights from Hugging Face. Confirm both are reachable from your network. Corporate firewalls or VPNs may block these endpoints.
2. **Update your GPU driver** — an outdated driver is the most common reason CUDA or ROCm fails to initialize. Install the latest driver from your GPU vendor and relaunch Koharu.
3. **Retry the download** — failed downloads are retried automatically on next launch. You can also trigger a retry by navigating to the relevant feature and running it again.
4. **Check the activity center** — Koharu reports download and initialization errors in the activity center at the bottom of the window. The full error message there is useful when filing a bug report.

For persistent failures, capture the full error from the activity center and see the [Troubleshooting](/reference/troubleshooting) reference page.