ormas-monitor¶
Catch the layer that breaks while your model trains, and get proof.
ormas-monitor is an open-source (Apache-2.0) PyTorch library and command-line tool from Oxiedo.
- It watches each layer against its own history. A layer's gradient is compared with that layer's normal range, not with a global threshold, so the alarm names the layer that failed.
- It certifies when a channel can never recover. For Conv/Linear → BatchNorm → ReLU channels, an exact test on the weights, combined with run-time checks of the wiring, the gradient and the optimizer, proves that no gradient can ever reach the channel again. The monitor states the assumptions the proof needs, checks them before it issues a certificate, and records why when it refuses.
- It can repair those channels and writes a tamper-evident record. Repair, off by default, re-initialises stuck channels and their optimizer state. The record is a hash-chained, signable file of everything the monitor saw.
It runs entirely on your machine. The core package makes no network calls and never reads your training data; only the demo downloads CIFAR-10, and only sinks you add send anything.
Start here¶
I am an engineer¶
| You want to | Go to |
|---|---|
| See it work end to end, with real output | Tutorial: five minutes |
| Check a saved model file, with no code changes | Quickstart: check a file |
| Watch a training run (plain PyTorch, Lightning or Hugging Face) | Quickstart: two lines |
| Understand what "certified dead" means | Dead channels and the certificate |
| Know what it does not do | Limitations |
I am not technical (founder, manager, CFO, compliance)¶
| You want to | Go to |
|---|---|
| Understand it in plain words, in two minutes | For leaders |
| Know why a team would use it | Why use it |
| Get help, or shape what we build next | Work with us |
The supporting tools¶
These exist to support the monitor. They are not separate products.
ormas-monitor check: a weights-only health check of a saved model, and a diff between two checkpoints: what changed and what died.- The training record: a hash-chained, signable, diffable record of what happened in training. It is evidence for technical documentation, not a compliance certificate.
- Reports for two readers: an engineer report, and a plain-language executive summary for a CEO, CFO or compliance lead.
Verified so far¶
On one seed (0) and one Apple M-series Mac (MPS, torch 2.11), on CIFAR-10: after two conv layers of a trained CNN were zeroed, a watch-only run stayed at 10.00% (chance) for all 12 epochs, while the run with repair recovered to 76.77% (healthy: 78.84%). Monitoring cost +3.6% median epoch time at the default check_every=50. Removing the dead channels check found in torchvision's VGG-19-BN cut its compute by 18.6% with identical answers. See Results for every number and the command that reproduces it.
Not yet tested on NVIDIA GPUs, multi-GPU or at scale
The test suite runs automatically on Linux, macOS and Windows. All measured numbers come from one Apple-silicon Mac. A GPU stress test is planned; until it has run, treat performance on other hardware as unmeasured.
Work with us¶
The tool is free. Oxiedo is looking for its first design partners, teams who train their own models and want help getting more out of their runs. See Work with us or email raadh@oxiedo.com.