AI Without Servers: When Your Devices Pool Their GPUs
A developer just ran a 27-billion-parameter language model — inside browser tabs, split between a MacBook and an iPhone. A lab curiosity? Perhaps not. This demo may sketch the future of AI: distributed, local, and reachable from a single link.
The demo that shifts the perspective
The starting point is simple: your phone cannot run a large language model like Qwen 27B on its own. It has neither the memory nor the compute.
The answer from the SwarmLLM project (swarmllm.ai) is elegant. If one device is not enough, why not teach the model to share itself?
In practice, the model is sliced layer by layer and distributed across the devices around you. Each device downloads and hosts only its own share. Tokens then travel peer to peer over WebRTC, and the answer appears on every screen at once — all in the browser, thanks to WebGPU, with nothing to install.
The result: roughly 2.5 tokens per second on a 27B model quantised to 4 bits, split across two consumer devices. That is not fast. But speed is not the point here. The point is that it works at all.
Why this is more than a technical stunt
Three building blocks make the approach possible, and all three are maturing at the same moment.
WebGPU gives web pages direct access to the device’s graphics processor. The same code runs on a Mac, a PC or an Android phone, with no driver and no installation. Inference engines like WebLLM already reach up to 80% of native performance in the browser.
WebRTC, the peer-to-peer protocol that already powers video calls in the browser, is reused here to move model activations between devices without any central server.
Quantisation compresses models until they fit, piece by piece, into a phone’s memory.
Taken separately, these technologies have existed for years. Combined, something new emerges: the browser becomes a universal, federatable AI runtime. Every open tab is a potential compute node.
What it could change
Inference without the cloud
Today, generative AI leans heavily on data centres: a cost per request, a dependency on connectivity, data passing through third-party servers. Local distributed inference inverts that logic. Computation happens where the data and the users already are. For sensitive uses — health, legal, education, personal data — that is a serious argument: nothing leaves the devices.
The home and the office as micro-clusters
An average household already holds five to ten GPU-equipped devices: phones, laptops, tablets, consoles, connected TVs. A meeting room holds just as many. Tomorrow, those devices could pool their power to run models locally that are cloud-only today. The family assistant or the team’s AI would no longer be a subscription, but an emergent property of the hardware already in the room.
Resilient, frugal AI
No server, no single point of failure, no bill that climbs with usage. In poorly connected areas, at events that saturate networks, or simply to reduce AI’s footprint, swarm computing offers a serious alternative: it uses hardware already manufactured, already bought, and unused about 95% of the time.
New multi-screen experiences
This may be the most unexpected implication. In the SwarmLLM demo, the answer appears on both screens at once. The AI is no longer in a device — it is between devices. Picture collective experiences where the phones of a group of visitors, an audience or a set of customers form a local, ephemeral intelligence together, one that vanishes when everyone closes their tab.
A reshuffling of the economics
If a single link can mobilise the public’s compute, the barrier to entry for AI collapses for creators, schools, SMEs and local authorities. The GPU-cloud business model will not disappear — training and intensive workloads will depend on it for a long time — but a growing share of everyday inference could migrate to the edge.
The obstacles, because there are some
Let us stay clear-eyed. 2.5 tokens per second is ten to forty times slower than a cloud API. Network latency between devices, hardware heterogeneity, battery drain, the security of peer-to-peer exchanges, and the question of trust between nodes — who guarantees that a device is honestly computing its slice? — are all open problems. Research is moving quickly on them: peer consensus, smart routing between small local models and large remote ones. But the road to mainstream use will take years.
The essential point: waking the world’s largest GPU fleet
Running an LLM in the browser is no longer novel in itself; engines like WebLLM already do it on a single device. What SwarmLLM adds — and this is where everything is decided — is the ability to aggregate GPU resources that are asleep.
Because the world’s largest data centre already exists. It is scattered across our pockets, our bags and our living rooms. Billions of phones, computers and tablets carry GPUs that sit idle the vast majority of the time: silicon already manufactured, already paid for, already powered, computing nothing. Meanwhile, the industry is investing hundreds of billions in new data centres to meet inference demand.
Swarm inference proposes a third path between “all cloud” and “all local”: pooling that latent capacity, on demand, with no installation, for the duration of a session. One device alone is too limited; ten devices federated by a single link become infrastructure. It is the same conceptual shift as ride-sharing against the car fleet — you do not add cars, you fill the empty seats.
For years the strategic question was:
“Who owns the biggest data centres?”
If this approach delivers, the question of the decade becomes:
“Who knows how to orchestrate the compute that already exists, everywhere, on everyone’s devices?”
That shift in framing is, we think, where its real significance lies. It is also familiar territory for us: everything ARGO builds runs in the browser, with no app to install, precisely because the device already in someone’s hand is the most available piece of hardware there is. Swarm inference simply extends that logic from rendering an experience to computing one.
Sources: SwarmLLM · WebGPU specification, W3C · WebLLM project · WebRTC, MDN