# Quickstart

Get from zero to a trained adapter in under 5 minutes.

## Option 1: PorosTrainer (5 lines)

```python
from poros import PorosTrainer

trainer = PorosTrainer.from_pretrained(
    "Qwen/Qwen2.5-32B",
    quantization="nf4",
    lora_rank=16, lora_alpha=32,
)
trainer.train("OpenAssistant/oasst1", steps=200, seq_len=512)  # any HF dataset, or "synthetic"
trainer.save("./poros_output/qwen32b-lora")   # adapter + tokenizer + config
```

`block_size` defaults to the validated **4**; pass it explicitly only to
tune. For full configuration use
`PorosConfig` (see the API reference).

### Use your trained adapter

```python
import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, BitsAndBytesConfig
from peft import PeftModel

# Load the 32B base in NF4 4-bit — a dense load would need ~65 GB of VRAM.
bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)
base = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-32B", quantization_config=bnb, device_map="auto"
)
model = PeftModel.from_pretrained(base, "./poros_output/qwen32b-lora")
tok = AutoTokenizer.from_pretrained("./poros_output/qwen32b-lora")
```

Or merge into a standalone model for vLLM / TGI:
`poros export ./poros_output/qwen32b-lora -o ./merged`. See `examples/inference.py`.

Merging dequantizes the base back to bf16, so `poros export` needs the whole model resident on the GPU — the one thing blockwise streaming does not help with, since it is a training technique. If you trained a model bigger than your card, keep serving the adapter on top of the 4-bit base (Option A in `examples/inference.py`); Poros tells you this with a clear error rather than an OOM.

`from_pretrained` loads the model in NF4 (4-bit), attaches LoRA adapters,
and sets up blockwise residency.  `block_size=4` means 4 transformer layers
are resident at a time; the rest stream from CPU pinned memory.  If you omit
`block_size`, Poros uses the validated default **4** — it is **not** guessed
from available VRAM. `block_size` is a memory<->speed knob (larger = fewer
transfers = faster but more resident VRAM; smaller = less VRAM but slower); it
never affects parity.

## Option 2: HF drop-in (one line)

If you already have a Hugging Face training script:

```python
import torch
from transformers import AutoModelForCausalLM, BitsAndBytesConfig
from peft import get_peft_model, LoraConfig
import poros

bnb = BitsAndBytesConfig(
    load_in_4bit=True,
    bnb_4bit_quant_type="nf4",
    bnb_4bit_compute_dtype=torch.bfloat16,
    bnb_4bit_use_double_quant=True,
)
model = AutoModelForCausalLM.from_pretrained(
    "Qwen/Qwen2.5-32B", quantization_config=bnb
)
model = get_peft_model(
    model, LoraConfig(r=16, lora_alpha=32, target_modules=["q_proj", "v_proj"])
)

model = poros.prepare(model)   # the only Poros line

# standard HF Trainer now trains blockwise (the canonical 32B config peaks at 11.61 GB)
```

This is the same one line the README leads with. The prepared model is still
a compliant `nn.Module` — HF Trainer, eval callbacks, loss handlers, and AMP
all work unchanged. `prepare()` also detects precision, gates the adapter, and
reports which guarantee applies;
[`poros.patch_hf_model`](api.md#hf-drop-in-patch) is the lower-level engine
call it wraps, for when you want to skip the detection and gating.

## Option 3: CLI

The CLI trains from a YAML config, and writes you a ready-to-run one:

```bash
poros init              # writes train.yaml: model, dataset, output_dir
poros train train.yaml
```

The generated file has three active fields — everything omitted uses the
validated defaults (NF4, rank-16 LoRA, block size 4) — plus the common
knobs as comments, ready to uncomment. `poros init -m <model> -d <data>`
pre-fills it; `poros schema config` prints every available field.

To train on your own `.jsonl`/`.json`/`.csv` file, see
[poros-trainer.md](poros-trainer.md) — a local path in the `dataset:` field
works as-is. If a run is interrupted, `poros train train.yaml
--resume-from latest` resumes from the newest checkpoint in `output_dir`
(written every `save_steps` steps), bitwise-faithful to an uninterrupted run.

## What to expect

The 32B configuration trains at 11.61 GB peak (measured), which fits a
16 GB VRAM budget with 4.39 GB of headroom; the smallest physical card we
have measured on directly is 24 GB.  The per-step slowdown vs
fully-resident QLoRA is hardware- and scale-dependent — measured 1.7× at 7B
on an RTX 3090, about 1.6× at 32B on an RTX PRO 6000, 2.28–2.68× on an
A100, and ~3.5–4.3× on a B300 — the deliberate
memory-for-time tradeoff.  On a 24 GB card the resident 32B does not fit at
all; Poros does.

The training output is bitwise identical to the same setup run fully
resident: 0.00e+00 max difference on loss and adapter weights. (That is a
claim about Poros's streaming, not about matching a different framework's
numbers — see [limitations.md](limitations.md).)

## Next steps

- [Benchmark guide](bench.md) — run a reproducible comparison
- [API reference](api.md) — full configuration options
- [Supported architectures](architecture-support.md) — validated model families
- [Limitations](limitations.md) — what Poros does and does not do