Betting on local AI (webnn)

Introduction

There are numerous AI services available today that offer powerful models at your fingertips. While this trend will certainly continue, what’s less obvious is the state of local AI.

Yes, you can run models on your own machine if you have the necessary skills, and tools like Ollama make running local LLMs more approachable. However, keep in mind that not all models are large language models—just as WebAssembly (WASM) is a feature built for specific functionality, machine learning models can also be tailored for particular purposes.

Significant progress has already been made to bring models directly to users’ local machines. One area of particular interest is running models as a native feature in the browser. The good news is that this has already started, and there is even a W3C user group dedicated to this effort.

WebNN API Specification (W3C draft)
https://www.w3.org/TR/webnn/

In recent years, the web platform has expanded in powerful new directions, from WebAssembly (WASM) to WebGPU. The next frontier? Machine learning in the browser. Enter WebNN and the broader Web Machine Learning initiative—exciting developments that aim to make the web an even richer application platform for ML-based experiences. Below is a snapshot of the current status of WebNN, where it’s going, and why it matters.

What is WebNN?

WebNN stands for the Web Neural Network API. It is a proposed low-level web API that provides hardware-accelerated machine learning inference in web applications, without the need for additional plugins or platform-specific frameworks.

Goals:
- Provide a unified interface for web developers to run machine learning models across CPUs, GPUs, and specialized accelerators.
- Enable high-performance, low-latency inference.
- Let developers target a single API without worrying about platform fragmentation.
Who’s behind it?
WebNN is part of the work overseen by the W3C Web Machine Learning Working Group (and previously by the Community Group). Representatives from major tech companies, including Microsoft, Google, and Intel, are actively contributing to its specification and prototype implementations.

Why Does WebNN Matter?

1. Performance on the Web

Traditional ML frameworks running in the browser today rely heavily on JavaScript and WebAssembly, or they leverage higher-level libraries like TensorFlow.js. While these solutions work, they sometimes struggle to achieve the same level of performance as native applications. WebNN aims to close that gap by exposing hardware capabilities directly in a standardized way—thus ensuring that neural network operations are optimized, parallelized, and accelerated.

2. Consistent Developer Experience

By introducing a consistent, standardized API, WebNN allows developers to write machine learning code once and trust that it will run efficiently across browsers that support the specification. No need for browser-specific workarounds or multiple code paths—just one common interface.

3. New Use Cases

When ML inference can happen quickly in the browser, new classes of applications become possible. Image recognition, speech processing, or even on-device personal assistants can run client-side without sending data to a server. This not only preserves privacy but also reduces latency and network dependencies.

Current Status of WebNN

1. Spec Progress at the W3C

The WebNN API is still in the process of specification within the W3C Web Machine Learning Working Group. While the core API surface is becoming more stable, there is ongoing discussion about how it will integrate with other emerging web standards, especially WebGPU.

You can track the current specification on the W3C Web Machine Learning GitHub repository.
Active issues and proposals focus on shaping the operator set (e.g., convolutions, pooling, activations) and ensuring a robust developer ergonomics and security/privacy model.

2. Browser Implementations

Chrome: Experimental support for WebNN is available behind a flag or through special builds of Chromium. Google’s involvement signals active exploration of hardware acceleration paths on different devices.
Edge: Microsoft, a significant contributor to the spec, has experimented with prototypes in Edge (which is Chromium-based). Any WebNN features in Edge would likely align with the same flags or experimental builds as Chromium.
Other Browsers: While other major browsers (Firefox, Safari) have not yet publicly signaled full-scale plans to implement WebNN, the progress of the Working Group and demonstration of real-world benefits may eventually encourage broader support.

3. Polyfills and Framework Integrations

For the time being, developers can use various machine learning libraries (like ONNX Runtime Web, TensorFlow.js, or community-driven polyfills) that simulate WebNN’s functionality via WebAssembly or WebGPU. These libraries aim to bridge the gap until native support is more widely available.

The Broader Web Machine Learning Ecosystem

WebGPU and ML

The WebGPU API—now available in some browsers in an experimental or emerging form—plays nicely with WebNN. WebGPU offers low-level GPU access to the web, which can be used to accelerate machine learning computations. Over time, one might see synergy between WebNN’s specialized ML operations and WebGPU’s flexible programming model.

Privacy and Security Considerations

Running machine learning models directly in the browser raises important privacy and security questions—particularly around fingerprinting or resource usage. The Working Group is actively discussing these issues, aiming to design a spec that allows developers to harness acceleration while preserving user privacy and security.

Developer Resources

W3C’s Web Machine Learning Working Group: Official site
WebNN Explainer: High-level overview of goals, motivations, and design principles.
GitHub Issues: Active discussions, proposals, and community input about the evolving spec.

What’s Next for WebNN?

Standard Maturation
The Working Group aims to bring WebNN to a Recommendation stage. That means reaching consensus on the API design, ensuring compatibility, and building a solid set of test suites.
Broader Implementation
As WebNN matures, expect to see more robust, stable implementations in Chromium-based browsers—and possibly interest from others once the specification is stable and the performance benefits are clearer.
Integration with Existing ML Tools
Watch for deeper integrations with popular frameworks. As soon as WebNN is stable enough and present in a critical mass of browsers, libraries like ONNX Runtime, TensorFlow.js, and PyTorch.js could add native WebNN backends, improving performance further.
More Examples & Demos
To drive adoption, we’ll likely see more demos of how WebNN can accelerate common tasks: real-time video segmentation, image classification, audio processing, and more. If you’re keen to experiment, keep an eye on GitHub repos that showcase proof-of-concept demos, often reliant on cutting-edge browser builds.

Conclusion

The WebNN API is an exciting step forward for bringing high-performance machine learning to the web platform. While it’s still early days—requiring experimental flags and not yet supported across all browsers—momentum is steadily building. As standards solidify, we can look forward to a future where cutting-edge ML models run swiftly and securely in the browser, without external dependencies or performance bottlenecks.

Stay tuned—the next few years promise to be transformative for web-based machine learning. If you’re a developer, consider tinkering with experimental builds to see firsthand what the future of on-device web ML might look like. And if you’re a curious user, prepare for a new generation of rich, ML-driven experiences that load instantly right in your browser.

Further Reading & Resources

By keeping an eye on these developments, you’ll stay at the forefront of the next major leap in web technology—an era where powerful machine learning models become first-class citizens of the open web.

UI needs to change

caperaven