> ## Documentation Index
> Fetch the complete documentation index at: https://koharu.rs/llms.txt
> Use this file to discover all available pages before exploring further.

# Hardware and runtimes

> Choose a usable device and understand what Koharu downloads and caches.

Koharu downloads the native libraries and model files needed by the features you use. You do not need a full CUDA or ROCm SDK installation to run a release.

## Device selection

Startup chooses a shared ML device from the available backends: Metal on Apple silicon, then CUDA, ROCm/HIP, Vulkan, or CPU where supported. The operating system, driver, model backend, and published runtime packages must all support the selected path.

<Tabs>
  <Tab title="NVIDIA">
    CUDA 13.0 support is available on Windows and Linux. Use a Turing-class or newer GPU and an
    R580-series or newer driver. See NVIDIA's [driver and architecture
    matrix](https://docs.nvidia.com/datacenter/tesla/drivers/cuda-toolkit-driver-and-architecture-matrix.html)
    and [driver downloads](https://www.nvidia.com/en-us/drivers/).
  </Tab>

  <Tab title="AMD">
    ROCm/HIP support depends on the exact GPU, operating system, and driver. Install a compatible
    [AMD driver](https://www.amd.com/en/support). Koharu resolves its required ROCm runtime packages
    automatically. Check AMD's [compatibility
    matrix](https://rocm.docs.amd.com/en/docs-10.0.0/compatibility/compatibility-matrix.html) for
    platform requirements.
  </Tab>

  <Tab title="Apple silicon">
    Koharu uses Metal for supported local inference on Apple-silicon macOS.
  </Tab>

  <Tab title="Vulkan and CPU">
    Vulkan provides an accelerator option on Windows and Linux. If no complete accelerator path
    initializes, Koharu falls back to CPU. CPU inference requires no GPU SDK and is slower.
  </Tab>
</Tabs>

<Note>
  The editor canvas uses WebGPU in the embedded CEF webview. It still needs a working WebGPU adapter
  and graphics driver when ML inference runs on CPU.
</Note>

## Downloads and caches

| Data                                                      | When it is needed                       |
| --------------------------------------------------------- | --------------------------------------- |
| Torch, llama.cpp, diffusion, and backend runtime packages | When their runtime is initialized       |
| Pinned or versioned model weights                         | On first use of the selected processor  |
| Local GGUF quantizations                                  | When you choose and use that model file |

Packages live under the operating system cache directory in `koharu/packages`. Projects live under `Documents/Koharu`; settings live in `~/.koharu/config.toml`. See [data locations](/en/reference/data).

Downloads are staged before being published to the cache. Retry an interrupted download on the next launch or model use.

## Memory and model residency

The resource monitor reports host memory, compute activity, and model residency. Models load lazily and may remain resident for reuse. Koharu can evict idle models when a later stage needs memory.

Start with a moderate local quantization. Smaller files usually need less memory, with a possible quality trade-off. A model that fits on disk can still exceed device memory.

If startup fails, check network access to the package repositories, update the driver, and retry once. Capture the full error if it repeats and follow [troubleshooting](/en/reference/troubleshooting).