Poros documentation

Fine-tune large models on small GPUs: 32B in ~11 GB of training peak, and 72B end-to-end on a single 32 GB card.

Poros streams the frozen quantized base through the GPU one block at a time, keeping only the LoRA adapters resident in VRAM. The streaming changes nothing — loss, adapter weights, and optimizer state all come out bit-for-bit identical to the same configuration trained fully resident.

Quickstart

from poros import PorosTrainer

trainer = PorosTrainer.from_pretrained(
    "Qwen/Qwen2.5-32B",
    quantization="nf4",
    lora_rank=16, lora_alpha=32,
)
trainer.train("OpenAssistant/oasst1", steps=200, seq_len=512)  # any HF dataset, or "synthetic"
trainer.save("./poros_output/qwen32b-lora")   # adapter + tokenizer + config

Five lines to a trained adapter. Full quickstart →

Getting started

Installationpip install poros-train, poros doctor, and the pinned torch 2.10+ / CUDA 12.8 stack.QuickstartZero to a trained adapter in under 5 minutes — PorosTrainer in 5 lines, the one-line poros.prepare HF drop-in, or the poros init / poros train CLI.Train on your own dataPoint at a .jsonl/.json/.csv file and run poros train. Scaling and troubleshooting.

Usage

poros.prepare Auto APIThe one-line drop-in — what prepare detects, the guarantee labels, and the gates that raise early.Adapter supportPoros trains frozen-base adapters only. Status matrix for PEFT LoRA, DoRA, and experimental types.API ReferencePorosTrainer, PorosModel, patch_hf_model, PorosConfig fields, the poros CLI, determinism, architecture registry, errors.BenchmarkingThe bench harness: shared configuration per arm, what is and is not held identical, and how the campaign numbers were measured.

Reference

Consumer GPUsMeasured peaks by model size and card.Architecture supportWhich model families are validated.Parity modesWhat each guarantee label means.LimitationsWhat Poros does not do.Official A100 resultsThe A100 parity + throughput matrix.Official B300 resultsThe B300 parity + throughput matrix.

Key properties

Bitwise parity: 0.00e+00 max diff on loss and adapter weights (NF4 QLoRA, gated 0.00e+00 across Qwen2.5, Qwen3, Gemma3, and Gemma4 on the platforms in the paper's parity matrix, Gemma on RTX 5090; plus Qwen3.5/Qwen3.6 validated on RTX 4090).
68–77% VRAM reduction vs fully-resident QLoRA; 45–60% less than Unsloth (NF4, 32B–72B scale).
Throughput tradeoff: hardware-dependent — about 1.6× slower per step on an RTX PRO 6000, up to ~2.7× on an A100, and ~3.5–4.3× on a B300 (where the resident baseline is so fast that PCIe streaming dominates the ratio). Memory for time.
Pure-python install: pip install poros-train works without torch. ML dependencies are optional extras.