# Poros — Official Validation Results (NVIDIA B300 SXM6)

Official, measured benchmark numbers for the architecture families gated on the
B300 (`qwen2`, `qwen3`). Every cell is **measured** — no number is interpolated
or estimated. The same-attention-path A100 matrix is included below.

## Configuration

| Field | Value |
|---|---|
| Hardware | NVIDIA B300 SXM6 (268 GB VRAM) |
| Precision | NF4 (the only bitwise-parity path) |
| Steps × passes | 200 steps × 3 passes (warmup discarded; median ± std across passes) |
| Sequence length | 512 |
| Batch / grad-accum | 1 / 1 (so **1 step = 1 optimizer step** for every method) |
| LoRA | rank 16, alpha 32 |
| Block size (poros) | 4 |
| Dataset | `synthetic` (deterministic, license-free random tokens) |
| Methods | poros (blockwise residency), resident (full QLoRA), unsloth |
| Software stacks | torch 2.10.0+cu128 / transformers 5.5.0 and torch 2.12.0+cu130 / transformers 5.10.1, bitsandbytes 0.49.2 |

**Scope.** The bitwise-parity guarantee (0.00e+00) holds on the deterministic,
math-only SDPA path — that is what Poros ships and what the poros cells below
run. The resident/unsloth baselines use their default flash attention, so the
slowdown column includes a math-vs-flash gap on top of the streaming cost;
same-path references are ~1.6× at 32B on an RTX PRO 6000 (all methods
math-only) and 2.28–2.68× on the A100 (all methods fast path, table below).
Bitwise parity is a separate gate asserting `torch.equal(resident, blockwise)`
— exact, not `allclose`.

## Results — B300

| Model | Arch | Stack | Bitwise parity | poros s/step (median±std) | resident s/step | unsloth s/step | Slowdown vs resident | vs unsloth | poros VRAM | resident VRAM | unsloth VRAM | VRAM cut vs resident | vs unsloth |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3-32B | qwen3 | torch 2.10.0+cu128 | PASS (0.00e+00) | 1.0585±0.0044 | 0.2945 | 0.4048 | 3.59× | 2.61× | 11.12 GB | 37.92 GB | 20.49 GB | 70.7% | 45.8% |
| Qwen3-32B | qwen3 | torch 2.12.0+cu130 | PASS (0.00e+00) | 1.0483±0.0096 | 0.2960 | — | 3.54× | — | 11.11 GB | 37.89 GB | — | 70.7% | — |
| Qwen2.5-32B | qwen2 | torch 2.10.0+cu128 | PASS (0.00e+00) | 1.0012±0.0082 | 0.2755 | 0.4051 | 3.63× | 2.47× | 11.04 GB | 36.53 GB | 20.54 GB | 69.8% | 46.2% |
| Qwen2.5-32B | qwen2 | torch 2.12.0+cu130 | PASS (0.00e+00) | 0.9900±0.0049 | 0.2731 | — | 3.62× | — | 11.04 GB | 36.54 GB | — | 69.8% | — |
| Qwen2.5-72B | qwen2 | torch 2.10.0+cu128 | PASS (0.00e+00) | 1.8680±0.0087 | 0.4489 | 0.5917 | 4.16× | 3.16× | 18.16 GB | 67.63 GB | 42.46 GB | 73.1% | 57.2% |
| Qwen2.5-72B | qwen2 | torch 2.12.0+cu130 | PASS (0.00e+00) | 1.8635±0.0101 | 0.4410 | — | 4.23× | — | 18.16 GB | 67.63 GB | — | 73.1% | — |

Raw per-cell artifacts (full provenance: torch/cuda/transformers/bitsandbytes
versions, GPU UUID, per-pass timings):
[`validation/official_b300_consolidated.jsonl`](validation/official_b300_consolidated.jsonl).

## Results — A100 80GB PCIe (same attention path for all methods)

All three methods on the same default fast path and the `synthetic` dataset, so
the slowdown column is the pure cost of blockwise residency. Full configuration
and notes: [official-a100-results.md](official-a100-results.md).

| Model | Arch | poros s/step (median±std) | resident s/step | unsloth s/step | Slowdown vs resident | vs unsloth | poros VRAM | resident VRAM | unsloth VRAM | VRAM cut vs resident | vs unsloth |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen2.5-32B | qwen2 | 2.0489±0.0091 | 0.7662 | 0.9086 | 2.67× | 2.25× | 10.95 GB | 34.72 GB | 20.82 GB | 68.4% | 47.4% |
| Qwen3-32B | qwen3 | 2.1832±0.0040 | 0.8157 | 0.9287 | 2.68× | 2.35× | 11.03 GB | 36.00 GB | 20.77 GB | 69.4% | 46.9% |
| Gemma3-27B | gemma3 | 1.8933±0.0048 | 0.7835 | 0.9365 | 2.42× | 2.02× | 9.95 GB | 33.17 GB | 22.06 GB | 70.0% | 54.9% |
| Gemma4-31B | gemma4 | 2.2537±0.0055 | 0.9889 | 1.0255 | 2.28× | 2.20× | 10.65 GB | 38.75 GB | 20.57 GB | 72.5% | 48.2% |

## Takeaways

- **Bitwise parity holds at scale.** `torch.equal(resident, blockwise)` is
  exact (0.00e+00) at every scale gated on this box — Qwen2.5 through 72B and
  Qwen3 at 32B. Gemma3/Gemma4 are gated on the RTX 5090, not here.
- **~70–73% VRAM cut vs resident, ~46–57% vs unsloth.** A 72B fine-tunes in
  **18 GB** instead of 68 GB resident / 42 GB unsloth.
- **No PyTorch-version regression.** poros speed is identical on torch 2.10 vs
  2.12 (Qwen2.5-72B: 1.868 vs 1.864 s/step; Qwen3-32B: 1.059 vs 1.048).
- **The slowdown ratio is hardware-dependent.** The B300's resident baseline is
  extraordinarily fast (~0.27 s/step at 32B), so a fixed streaming cost shows
  up as a large ratio (~3.5–4.2×). Same-path runs measure 2.28–2.68× on the
  A100 and ~1.6× at 32B on an RTX PRO 6000.

## Reproduce

```bash
# poros + resident (needs bf16 base weights on disk)
poros bench matrix configs/bench/official_b300_qwen25_72b_pr.yaml -o out/
# unsloth (pulls its own prequantized 4-bit repo)
poros bench matrix configs/bench/official_b300_qwen25_72b_un.yaml -o out/
# bitwise parity gate
POROS_GPU_TEST_MODEL=Qwen/Qwen2.5-72B pytest -m gpu tests/test_gpu_parity.py
```