July 4, 2026 · Norway
Rethinking a decentralized P2P LLM Swarm
My notes from trying to understand whether a decentralized LLM mesh can actually compete with centralized inference.
This is my attempt to think through a distributed LLM architecture using Hyperswarm.
The original idea was simple and exciting: build something like BitTorrent for AI. Take a huge model, split it into many small shards, spread those shards across consumer devices, and let the swarm collectively run inference.
It sounds beautiful.
But the more I thought through the actual mechanics, the more the design started to collapse under physics, latency, and the sequential nature of LLM inference.
The initial dream: BitTorrent for AI
The naive version of the system looked something like this:
- Start with a massive model, maybe a 2-trillion-parameter model.
- Chop it into millions of small shards.
- Store those shards across phones, laptops, and desktops.
- Use a peer-to-peer network to route inference through the swarm.
At first glance, this feels like the decentralized answer to centralized AI infrastructure.
But LLM inference is not like downloading a movie from many peers at once. It is autoregressive and sequential. Layer n cannot run until layer n - 1 is finished.
That creates a brutal networking problem.
Even if Hyperswarm can help discover nearby peers quickly, and even if the network somehow achieves around 10ms round-trip time per hop, a large model with something like 120 layers becomes painfully slow if each step depends on another consumer node across the internet.
In my rough math, this kind of setup could land around 0.8 tokens per second. At that speed, generating a paragraph would take minutes.
This is where the romantic decentralized idea meets the boring reality of hardware. Big centralized AI systems win because their GPUs are physically close together and connected with extremely fast fabrics like NVLink, quoted around 900 GB/s. That is not something we can recreate over home Wi-Fi, 5G, and residential routers.
So the main lesson for me was:
Live network sharding of a single LLM is probably the wrong battle.
The pivot: stop sharding live inference
The revised architecture is much more practical:
Do not split one giant model across the internet for live inference.
Instead:
- Run intact, optimized 3B or 8B specialist models locally on the edge device.
- Use the device's native NPU, GPU, or WebGPU-capable runtime where possible.
- Use Hyperswarm only as an asynchronous coordination layer.
- Push lightweight training updates, not live inference activations.
In other words, the swarm should not behave like a distributed GPU cluster. It should behave more like a decentralized federated learning network.
Here is the rough shape of the architecture:
+-------------------------------------------------------------+
| EDGE NODE (Phone/Laptop) |
| [User Prompt] ---> [Local JS Intent Router] |
| | |
| +---> [Local 8B Expert Model] |
| | |
| (Expert Correction) |
| | |
| v |
| [Local AirLLM LoRA Trainer] |
+-------------------------------------------------------------+
|
(Async Overnight Swarm Push)
|
v
+-------------------------------------------------------------+
| HYPERSWARM P2P LAYER |
| [UDP Hole-Punching] ---> [DHT Swarm Topic] ---> [Aggregator]|
+-------------------------------------------------------------+
This design feels less magical, but much more realistic.
The architectural pillars
1. Hyperswarm for peer discovery and NAT traversal
Hyperswarm's strength is not magically making slow networks fast. Its value is in peer discovery, DHT coordination, and helping devices establish direct encrypted peer-to-peer connections through consumer routers.
That makes it useful as the background fabric:
- find peers,
- join shared topics,
- push updates,
- coordinate aggregation,
- avoid relying entirely on a central server.
But I would not use it to move activations between model layers during live inference.
2. AirLLM-style layer streaming for low-memory devices
RAM is the enemy on thin edge devices.
The useful idea from AirLLM is layer streaming: instead of holding an entire model in memory, the device can load one layer from storage, process it, evict it, and move to the next.
For this design, the interesting use case is not necessarily live distributed inference. It is overnight local training or fine-tuning while the device is idle, charging, and thermally safe.
The phone does not need to hold everything in memory at once. It can slowly process updates layer by layer and eventually produce a compact LoRA patch.
3. The expert correction flywheel
The most valuable data would come from actual experts correcting hallucinations and bad outputs:
- doctors,
- auditors,
- senior developers,
- lawyers,
- researchers,
- domain specialists.
But getting experts to clean data is extremely hard. It is a logistical nightmare.
The privacy angle is what makes the local-first model interesting. If professionals can correct model behavior on their own devices, without sending raw private data to a central cloud, they may be more willing to participate.
The swarm does not need their raw data. It only needs carefully constrained update patches.
A rough code fabric
This is a sketch of how I think such a local node could be structured.
Local intent routing
Instead of sending every prompt to one giant general model, a lightweight local router decides whether a specialist model can handle the request.
// local-router.js
import { localInferenceEngine } from './edge-llm.js';
import { swarmClient } from './swarm-client.js';
export async function handleUserPrompt(prompt) {
const intent = await localInferenceEngine.classifyIntent(prompt);
if (intent.domain === 'javascript' && intent.confidence > 0.85) {
return localInferenceEngine.streamResponse('llama3-js-3b', prompt);
}
return swarmClient.requestRemoteInference(intent.domain, prompt);
}
The important part is that the first decision happens locally. The network is a fallback, not the default path.
Background LoRA training and swarm sync
This is the asynchronous part. It should only run when the device is idle, plugged in, and within safe thermal limits.
// background-worker.js
import crypto from 'node:crypto';
import Hyperswarm from 'hyperswarm';
import {
loadLayerFromDisk,
runBackprop,
initializeGradients,
compileDelta
} from './airllm-edge.js';
export async function executeOvernightTraining(expertDataset) {
const modelLayers = ['layer1.bin', 'layer2.bin', 'layer3.bin'];
let gradients = initializeGradients(expertDataset);
for (const layerFile of modelLayers) {
const currentLayer = await loadLayerFromDisk(layerFile);
gradients = await runBackprop(currentLayer, gradients);
currentLayer.free();
}
const loraPatch = compileDelta(gradients);
const swarm = new Hyperswarm();
const topic = crypto
.createHash('sha256')
.update('swarm-aggregation-v1')
.digest();
swarm.join(topic, { server: false, client: true });
swarm.on('connection', (socket) => {
socket.write(JSON.stringify({
type: 'LORA_UPDATE',
data: loraPatch
}));
socket.end();
});
}
The goal here is to produce a small update patch, not move the whole model around.
The hard problems
This is where the design gets uncomfortable. A decentralized learning swarm has problems that a closed data center can avoid or control more easily.
1. Sybil attacks and weight poisoning
If the network is open, what stops someone from spinning up thousands of emulator nodes and flooding the swarm with poisoned LoRA patches?
If those updates are aggregated blindly, the model becomes worse. Or worse than worse: it becomes subtly dangerous.
A possible countermeasure is a decentralized Web of Trust:
- cryptographic identity for nodes,
- historical validation scores,
- reputation-weighted updates,
- sandboxing for unknown contributors,
- validation sets before merging updates into trusted routes.
The key idea is that not every update should count equally.
2. Catastrophic model drift
If one expert trains on cardiology and another trains on tax fraud, simply averaging their LoRA matrices could cause interference. The updates may not be compatible.
Standard federated averaging feels too blunt for this.
A better direction might be a modular Mixture of Experts approach, where updates are routed by domain, region, or function instead of flattened into one shared parameter soup.
The aggregation layer would need to preserve specialization rather than erase it.
3. Thermal degradation and device safeguards
Training on consumer hardware is not free.
Backpropagation can keep the CPU, GPU, or NPU active for a long time. On a phone, that means heat, throttling, and battery wear.
So the client must be conservative:
- only train while charging,
- pause when unplugged,
- monitor battery temperature,
- stop around a safety threshold such as 38°C,
- respect OS-level power and thermal signals.
A swarm that burns people's phones is not sovereign. It is just rude.
Where I think the system stands
The pieces exist, but they are not yet one clean system.
Hyperswarm gives a useful peer-to-peer networking layer. AirLLM demonstrates a way to think about running large models under memory pressure by shifting the burden toward disk I/O.
The missing piece is orchestration:
- local inference,
- local expert correction,
- local LoRA generation,
- thermal-aware scheduling,
- P2P update distribution,
- trust-weighted aggregation,
- modular model merging.
The winning architecture may not be a decentralized version of a hyperscale GPU cluster. It may be a network of local specialist models that slowly improve through private, asynchronous, expert-guided updates.
That feels much more plausible to me.
The interface question I am left with
The big practical question is where the local interface should live.
Would I build this as a standalone native desktop/mobile client, with deeper access to hardware telemetry and local storage?
Or would I try to run as much as possible inside a browser sandbox using WebGPU, accepting stricter limits in exchange for easier distribution?
I am leaning toward native for serious training and browser/WebGPU for lightweight inference experiments. But I am not fully settled.
If you are thinking about this too, I would genuinely like to compare notes. Tag @methodinvalid on twitter.