Usage

PorosTrainer

The main training orchestrator. Handles model loading, adapter attachment, blockwise residency setup, and the training loop.

PorosTrainer.from_pretrained

@classmethod
def from_pretrained(
    cls,
    model_name_or_path: str,
    *,
    quantization: Literal["nf4", "bf16", "fp16", "bf16_bitwise"] = "nf4",
    lora_rank: int = 16,
    lora_alpha: int = 32,
    lora_dropout: float = 0.05,
    lora_target_modules: Optional[Union[List[str], str]] = None,
    block_size: Optional[int] = None,
    config: Optional[PorosConfig] = None,
    **kwargs,
) -> "PorosTrainer":

Load a model, attach LoRA adapters, and set up blockwise residency.

Parameters:

• model_name_or_path — HF model ID or local path.
• quantization — one of:
  • "nf4" (default, validated) — 4-bit NF4 QLoRA; elementwise-exact resident parity (0.00e+00, torch.equal).
  • "bf16" / "fp16" — EXPERIMENTAL dense 16-bit; parity is not bitwise and speed is ~10× slower than resident.
  • "bf16_bitwise" — EXPERIMENTAL dense-BF16 streaming runtime that does reach elementwise-exact (bitwise) resident parity under strict determinism (per-block RNG save/restore); it is slow and is not the supported NF4 path. Reported separately with its own gates (dense-BF16 appendix of the paper).
• lora_rank — LoRA rank (default 16).
• lora_alpha — LoRA alpha (default 32).
• lora_dropout — LoRA dropout (default 0.05).
• lora_target_modules — Modules to target. None = auto-detect per architecture.
• block_size — Number of layers resident per block. None = the validated default 4 (not guessed from VRAM). Memory<->speed knob: larger = fewer H2D transfers = faster but more resident VRAM; smaller = less VRAM but slower. Never affects parity.
• config — A PorosConfig instance. If provided, it is authoritative: its adapter and quantization fields override the individual keyword arguments.

PorosTrainer.train

def train(
    self,
    dataset: Union[str, "Dataset", "DataLoader"],
    *,
    steps: Optional[int] = None,
    seq_len: Optional[int] = None,
    eval_dataset: Optional[Union[str, "Dataset", "DataLoader"]] = None,
    resume_from: Optional[str] = None,
) -> Dict[str, Any]:

Run the training loop.

Parameters:

• dataset — HF dataset name, a Dataset object, or a DataLoader. String datasets accept instruction/output, plain text, and chat/messages formats. The special value "synthetic" (or "synthetic:<n_samples>") generates deterministic random tokens for benchmarking with no download or dataset license — token content does not affect VRAM / throughput / parity.
• steps — Number of optimizer steps. Defaults to config.max_steps or epoch-based.
• seq_len — Maximum sequence length (default from config).
• eval_dataset — Optional evaluation dataset.
• resume_from — Path to a checkpoint directory written by save(); restores adapter weights, optimizer state, LR scheduler, RNG state, and the step counter, then continues from the next step.

Returns: a metrics dictionary with keys including final_loss, total_steps, avg_step_time_ms, steps_per_sec, peak_vram_allocated_gb, and peak_vram_reserved_gb.

PorosTrainer.save

def save(
    self,
    output_dir: str,
    *,
    push_to_hub: bool = False,
    hub_model_id: Optional[str] = None,
) -> None:

Save adapter weights. Optionally push to the HF Hub.

PorosTrainer.evaluate

def evaluate(
    self,
    eval_dataset: Union[str, "Dataset", "DataLoader"],
) -> Dict[str, float]:

Evaluate and return {eval_loss, eval_perplexity}.

PorosModel

Lower-level model loading and adapter management.

PorosModel.from_pretrained

@classmethod
def from_pretrained(
    cls,
    model_name_or_path: str,
    *,
    quantization: Literal["nf4", "bf16", "fp16", "bf16_bitwise"] = "nf4",
    torch_dtype: str = "bfloat16",
    lora_rank: int = 16,
    lora_alpha: int = 32,
    lora_dropout: float = 0.05,
    lora_target_modules: Optional[Union[List[str], str]] = None,
    adapter_backend: Literal["auto", "peft", "native"] = "auto",
) -> "PorosModel":

Load an HF model with quantization and LoRA adapters.

Properties

• model — the underlying nn.Module.
• trainable_parameters — list of all trainable (adapter) parameters.

PorosModel.merge_and_unload

Merge LoRA weights into the base model and return the unloaded module.

HF drop-in patch

poros.patch_hf_model

def patch_hf_model(
    model: "nn.Module",
    *,
    block_size: Optional[int] = None,
    enable_prefetch: bool = True,
    preserve_rng_state: bool = True,
    arch_check: Literal["error", "warn", "off"] = "error",
    deterministic: bool = True,
    managed_parameter_filter: Optional[Callable[[str, "nn.Parameter"], bool]] = None,
) -> "nn.Module":

deterministic=True (default) enables process-global deterministic algorithms + math-only SDPA (install.md). managed_parameter_filter(name, param) -> bool: return False to keep a parameter resident and unmanaged (adapters always are); the adapters-resident path behind it is gated behind GPU validation — not a stable default.

Patch a Hugging Face model so its transformer blocks stream through the GPU blockwise. The model may already have PEFT LoRA applied.

Behavior:

1. Validates model.config.model_type against the architecture registry.
2. Locates the layer container via layer discovery.
3. Warns if trainable non-adapter parameters are found (Poros requires a frozen base).
4. Uses the validated default block_size=4 if not provided (not guessed from VRAM).
5. Regroups layers into blocks and routes forward/backward through the Poros autograd functions with per-block RNG save/restore.
6. Installs the block manager with cross-stream record_stream safety.

The patched model is still a compliant nn.Module.

PorosConfig

Central configuration model (Pydantic BaseModel). Can be instantiated without torch.

from poros import PorosConfig

config = PorosConfig(
    model_name_or_path="Qwen/Qwen2.5-32B",
    quantization="nf4",
    block_size=4,
    lora_rank=16,
    lora_alpha=32,
)

Fields

Model:

• model_name_or_path: str — HF model ID (default "Qwen/Qwen2.5-7B")
• quantization: "nf4" | "bf16" | "fp16" | "bf16_bitwise" — default "nf4"
• torch_dtype: "bfloat16" | "float16" | "float32" — default "bfloat16"

Blockwise residency:

• trust_remote_code: bool = False — allow the model repo to execute its own Python on load. Off by default; Poros's validated architectures never need it. CLI: poros train --trust-remote-code.
• block_size: Optional[int] — None = validated default 4 (not guessed from VRAM)
• freeze_embeddings: bool — default True
• freeze_lm_head: bool — default True
• preserve_rng_state: bool — default True (required for parity)
• blockwise_arch_check: "error" | "warn" | "off" — default "error"

LoRA:

• lora_rank: int — default 16
• lora_alpha: int — default 32
• lora_dropout: float — default 0.05
• lora_target_modules: Optional[List[str] | str] — None = auto per arch

Optimizer:

• learning_rate: float — default 2e-5
• weight_decay: float — default 0.01
• max_grad_norm: float — default 1.0
• warmup_steps: int — default 100
• lr_scheduler: "constant" | "cosine" | "linear" — default "cosine"

Training:

• num_epochs: int — default 1
• per_device_batch_size: int — default 1
• gradient_accumulation_steps: int — default 8
• max_seq_length: int — default 2048
• seed: int — default 42

Memory:

• nf4_load_strategy: str — default "auto": try the bounded-GPU streamed load (byte-identical on every validated family; see docs/validation/streamed-load/), falling back to the standard loader on any failure. "gpu" forces the standard loader; "block_streamed" forces streamed with no fallback.
• enable_prefetch: bool — default True

Output:

• output_dir: str — default "./poros_output"
• save_steps: int — default 500
• logging_steps: int — default 10

CLI reference

Entry point: poros

poros --help, poros --version, poros doctor, poros check, poros report, poros leaderboard, and poros schema all work without torch or a GPU.

For scripting/agents: check, leaderboard, and doctor accept --json for machine-readable output on stdout (human output is unchanged without the flag). Every command returns conventional exit codes (0 ok, 1 failure, 2 usage error).

Determinism

Poros centralizes determinism settings in poros.core.determinism:

from poros import enforce_determinism
enforce_determinism()

This sets:

• CUBLAS_WORKSPACE_CONFIG=":4096:8"
• torch.use_deterministic_algorithms(True)
• torch.backends.cudnn.deterministic = True
• torch.backends.cudnn.benchmark = False
• Flash SDPA disabled, mem-efficient SDPA disabled, math-only SDPA enabled

These are set automatically when using PorosTrainer or patch_hf_model.

Architecture registry

from poros import validated_arch_types, architectures, register_architecture

validated_arch_types()    # first-party only, backed by committed parity artifacts
user_asserted_arch_types()  # what YOU registered (runtime or env)
runnable_arch_types()     # validated + user_asserted -- what the gate admits
architectures()           # full {model_type: ArchSpec} incl. roadmap targets

# Extend without forking (runtime), or set POROS_USER_ASSERTED_ARCHS:
register_architecture("phi3", notes="validated locally against my own run")

register_architecture() and POROS_USER_ASSERTED_ARCHS create user_asserted entries. Poros will run them, and poros check reports them as user_asserted — but no Poros parity guarantee applies, and prepare() labels such a model probed_not_validated, never validated_bitwise. validated_arch_types() is deliberately NOT influenced by either mechanism: it is the set backed by committed Caistro parity artifacts, and it is what a bitwise guarantee may be derived from. Use runnable_arch_types() for "what will Poros agree to run". VALIDATED_BLOCKWISE_ARCH_TYPES remains importable as a back-compat snapshot of the built-in set. See architecture-support.md for the full validation protocol and the precision registry.

Errors

All errors inherit from PorosError and include structured fields: type, message, remediation, doc_url.

• PorosArchNotValidatedError — raised when attempting to use an architecture not in the validated registry.
• PorosOOMError — raised on GPU out-of-memory, with suggestions for smaller block_size or shorter seq_len.
• PorosConfigError — raised on invalid configuration.