Distributed training as shared state.
PyTorch stayed PyTorch. The experiment replaced the distributed backend and asked which parts of the usual choreography were properties of the workload and which were properties of the programming surface.
PyTorch unchanged
The backend implemented all_reduce, barrier, broadcast, all_gather and reduce without requiring PyTorch itself to become a FrogNet application.
Contributions as state
Contributions can materialize independently. A New York contribution was observed before one of the Seattle contributions.
Heterogeneous paths
The experiment deliberately used machines and paths with large performance asymmetry instead of a homogeneous datacenter fabric.
A datacenter does not have to be the prerequisite for distributed compute.
FrogTorch is evidence for a broader idea: machines that were never bought, placed or matched to form a cluster can still become useful compute behind one shared-state programming surface. That can mean spare workstations, lab machines, older and newer systems, different CPU architectures, or machines in different buildings and cities.
This is the virtual-datacenter consequence of the experiment. Virtual datacenter describes the resource model: geographically separated and heterogeneous machines can participate behind one programming surface. It is not a claim that FrogNet reproduces a conventional datacenter scheduler, management plane, fault model or high-performance interconnect, and it is not a claim that the recorded FrogTorch runs establish parity with one. The architectural claim is narrower and more useful: the programming model can stop requiring a homogeneous datacenter before useful distributed work can begin.
PyTorch is an especially useful test because the framework above the backend stayed PyTorch. The experiment changed the distributed machinery underneath it rather than requiring the application ecosystem to be rewritten first.
See the complete compute and machine-independence inventory →Same-campaign comparisons
These are measurements from the recorded campaigns, not a claim that FrogNet is universally faster than Gloo.
| Campaign | Gloo elapsed time / FrogNet elapsed time |
|---|---|
| 1 | 1.95× |
| 2 | 1.64× |
| 3 | 1.03× |
| 4 | 1.75× |
Same result
Recorded campaigns produced bit-identical parameter hashes and the same held-out loss: 0.4940.
Stability mattered too
Recorded FrogNet shard medians stayed within a 24% band. Gloo medians on the same machines and workload varied by 127% when the 62.0 s campaign is included.
A separate control found that gradient_as_bucket_view materially changed Gloo performance. That result stays in the record because the control matters as much as the headline ratio.
Semantic variants are kept separate.
Preserve DDP semantics
shard and faithful keep ordinary DDP agreement semantics while changing how the backend uses the network.
Deliberately change consumption
fresh_only and every_k can skip work. They are experiments in application semantics and are not counted as ordinary DDP equivalents.
Why this matters
The useful result is not one speed ratio. It is that distributed numerical computation fit the same shared-state vocabulary used by applications, monitoring and network infrastructure while preserving the framework above it in the conservative arm.