Reference

Poros is designed around a specific tradeoff: memory for time. The limitations below are stated plainly because the project's credibility depends on honesty about what it does and does not do.

Current limitations

Throughput

Poros is slower per step than fully-resident NF4 QLoRA because streaming frozen blocks over PCIe costs wall-clock time — the fundamental memory-for-time tradeoff. The slowdown ratio is hardware-dependent: measured 1.7× at 7B on an RTX 3090, about 1.6× at 32B on an RTX PRO 6000, 2.28–2.68× on an A100, and ~3.5–4.2× on a B300 (measured — see official-b300-results.md). The ratio is larger on faster GPUs because the resident baseline gets faster while the PCIe transfer (the streaming cost) does not — a B300 does a 32B resident step in ~0.27 s, so the fixed transfer cost dominates the ratio.

Poros competes on feasibility (fits where others OOM), not raw speed. At 32B, fully-resident QLoRA needs ~37–47 GB depending on the GPU, so it OOMs on a 24 GB card. Poros trains the same model at ~11 GB:

Peak VRAM follows the model, block size, and sequence length rather than the card, which is why the first two agree to the reported precision.

What 0.00e+00 means — and what it does not

The bitwise claim compares two runs of Poros itself: the same model, config, seed, and kernels, once fully resident and once streamed blockwise. Those are torch.equal on the loss stream, adapter weights, and optimizer state — streaming adds zero numerical difference. That is the whole claim.

It is not a claim that Poros reproduces the numbers of a different implementation. An unpatched PEFT/HF training loop dispatches its own kernel sequence and reduction order, so its losses differ from a Poros run of the same recipe by normal cross-implementation floating-point noise — measured ~2.6×10⁻³ on a first-step loss at 0.5B (RTX 5090), the same band as the ≤4×10⁻⁴ cross-method final-loss spread in the benchmark table. Neither number is "wrong"; they are different valid FP execution orders. If you need your existing loop's exact trajectory, Poros cannot promise it — no alternative implementation can. What it promises instead is that choosing streaming over residency costs you nothing numerically, and that the guarantee is checked with torch.equal, never allclose.

The claim is also per-platform (see the parity matrix for which GPU/architecture pairs are gated) and NF4-only — bf16/fp16 are experimental and carry no bitwise claim.

bf16 LoRA (EXPERIMENTAL)

Dense-model bf16 streaming ships as quantization="bf16_bitwise":

• Parity is bitwise. The parity gate measures max_abs_diff = 0.0 across per-step losses, adapter gradients, adapter weights, and optimizer state — validated up to Qwen2.5-32B on A100 (receipts).

• Throughput is bandwidth-bound. bf16 is 4× the bytes of NF4, and the frozen base streams twice per step (forward + remat backward); at 32B the parity probe moved ~440 GB host→GPU per step. Expect several× slower than resident on PCIe — bf16 streaming benefits most from datacenter-class links (Grace-Blackwell NVLink-C2C). NF4 remains the recommended path.

bf16_bitwise emits an EXPERIMENTAL warning until the full official matrix runs at this precision. fp16 remains experimental and non-bitwise. Arches that are structurally non-bitwise in bf16 (MXFP4-native MoE dequantized to bf16, e.g. gpt-oss) run under a structural self-check tolerance instead — see poros/core/precision.py.

int8 (EXPERIMENTAL, not exposed)

int8 is bitwise (0.00e+00 parity) but bitsandbytes caches its compute state (state.CB, state.SCB, state.CxB) as module attributes on the GPU. The block manager offloads parameters but not these cached buffers, so the frozen base remains GPU-resident. Result: Poros int8 uses more VRAM than Unsloth int8 at the same scale (no committed measurement — the structural reason is above).

int8 is not exposed in the quantization field. The roadmap tracks either fixing the streaming or documenting int8 as non-streaming.

Single-GPU only

The paper validates single-GPU blockwise residency, and Poros ships single-GPU only. Multi-GPU support is deferred to a future release: it has not been validated against the blockwise engine, so the current release exposes no distributed surface (no launch command, no poros.distributed).

When multi-GPU lands it will be data-parallel — each GPU runs Poros locally and syncs adapter gradients — improving throughput and effective batch size but NOT reducing per-GPU VRAM. Sharded block residency (splitting the frozen base across GPUs), FSDP, ZeRO, tensor parallel, pipeline parallel, and multi-node remain future work.

Consumer-card caveats

VRAM savings are portable: they depend on the model and block size, not the GPU. But per-step latency depends on PCIe bandwidth. The measured numbers in the README are from an RTX 3090 and an RTX 5090; on narrower links (e.g. Gen4 x8), per-step time is higher.

Validated architectures only

Poros works on the validated dense families — Qwen2.5, Qwen3, Qwen3.5/Qwen3.6, Gemma3, and Gemma4 (see the registry via poros doctor, or poros check <model> for one model). Other architectures fail at startup with a clear error. This is by design: the architecture gate (blockwise_arch_check) refuses to train on an architecture that is neither validated nor registered by you as user_asserted, where parity is unknown.

See architecture-support.md for the validation protocol and how to add support for a new family.

No custom kernels

Poros ships no mandatory custom Triton kernels. It uses standard PyTorch operations. Kernel-level speed parity with tools like Unsloth is not a goal of the current release.

Pretraining is out of scope

Poros exploits a read-only frozen base. The offload is a near-free pointer release, and only tiny adapter optimizer state stays resident. Pretraining breaks these properties: mutable weights require bidirectional PCIe traffic (gradient sync), offload becomes a costly host memory sync, and full-model Adam state is large. Full-parameter blockwise training would effectively become ZeRO, not Poros.

Roadmap

Everything below ships only after its own 0.00e+00 parity gate passes — the validated path stays bitwise or it does not ship as validated. No dates.

Precision

• fp8 support. Streamed fp8 base weights on hardware with native fp8 compute; halves the transfer volume bf16 pays today.
• int8 streaming. Release the full bitsandbytes compute state on offload so int8 streams like NF4, or document it explicitly as non-streaming.

Throughput

• bf16 prefetch. Coalesce per-block transfers and deepen prefetch overlap for bf16_bitwise; bf16 stays a high-bandwidth-bus recommendation until then.
• Faster steps at NF4. Deeper prefetch and selective recompute.
• The Poros kernel. A custom-kernel program held to the same bitwise standard as the engine: only fusions that are bit-exact by construction qualify (pointwise chains — SwiGLU multiply, residual adds, rotary with precomputed tables — produce the same per-element values in the same order), and nothing ships without passing the 0.00e+00 parity gates. Reduction-reordering fusions are excluded by the spec, not deferred.

Architectures

• More dense families as their parity gates pass.
• MoE support. First-class mixture-of-experts streaming, including Gemma 4 MoE if routing under blockwise residency holds parity (today MXFP4-native MoE runs dequantized to bf16 under a structural, non-bitwise self-check — see the bf16 section above).
• Multimodal backbones.

Scale-out

• Multi-GPU support. FSDP / ZeRO / tensor parallel, multi-node training, and sharded block residency (per-GPU VRAM reduction).

Further out

• Custom Triton kernels.