Neuromorphic Computing Boosts Efficient Large Language Models

Large language models have reshaped the way machines understand and generate human language. These powerful systems excel at tasks like translation, summarization, and conversation, but they come with a significant downside. That is they consume vast amounts of energy and computational resources. This has sparked a push toward more efficient designs that can maintain performance while minimizing costs, both in terms of power and processing time. A recent study from researchers at the University of Groningen, Intel Labs, and the University of California, Santa Cruz offers a promising solution by adapting a highly efficient language model for Intel’s neuromorphic processor, Loihi 2. This approach could redefine how we deploy artificial intelligence, making it faster, cheaper, and more sustainable for everyone from casual users to developers and investors.
The research focuses on a specific type of large language model called a MatMul-free model. Different from traditional models that rely heavily on matrix multiplications, a computationally intensive process, this model eliminates them entirely. Matrix multiplications, or MatMuls, are mathematical operations that multiply large grids of numbers, often requiring significant power and time. Instead, the MatMul-free model uses simpler operations like additions and bit shifts, which adjust numbers by moving their binary digits. This shift reduces the computational burden, making it an ideal candidate for Intel’s Loihi 2, a chip designed to replicate how the human brain processes information. Loihi 2 excels at handling low precision calculations by using fewer bits to represent numbers and event-driven tasks, where computations happen only when needed, rather than constantly running in the background.
The general population may benefit from AI tools that operate on smaller, more energy efficient devices, including laptops and smartphones, without sacrificing functionality. Consider an AI assistant that delivers instant answers without draining battery or requiring extensive data center support. Researchers demonstrated a 370 million parameter model, adjustable values that control model function, maintained accuracy when subjected to quantization, a method that reduces numerical precision. The quantized model showed no performance degradation on a standard graphics processing unit when compared to the original, highlighting that efficiency does not compromise quality
Developers stand to gain significantly from this advancement. The MatMul-free model aligns with Loihi 2’s strengths, such as its ability to process sequences of data, like sentences or time-based signals using stateful neurons. These neurons retain information from previous steps, much like memory in a human brain, which suits the model’s reliance on state space models. State space models are an alternative to the attention mechanisms found in many conventional language models, offering linear scaling, meaning their computational needs grow steadily rather than exploding as input size increases. This makes them easier to implement and optimize on specialized hardware like Loihi 2. For developers working on AI applications, this compatibility opens the door to building faster, more energy efficient systems, potentially reducing the time and cost of deploying new software.
Investors, particularly those eyeing Intel’s roadmap, will find the connection to Loihi 2 compelling. Intel has positioned Loihi 2 as a cornerstone of its neuromorphic computing efforts, targeting applications where efficiency and speed are critical. Preliminary results from the study suggest that the MatMul-free model on Loihi 2 could achieve up to three times higher throughput, processing speed, and use half the energy of comparable transformer based models running on an edge GPU, a type of processor designed for compact devices. These gains come from the chip’s ability to skip unnecessary calculations and its support for sparse data, where many values are zero and can be ignored. While these numbers are estimates based on early tests, further refinements could push efficiency even higher. For Intel, this positions Loihi 2 as a potential leader in the growing market for AI hardware, with a total addressable market that spans edge devices, data centers, and beyond.
The model itself is built around a clever architecture. It alternates between two types of processing layers. Token mixers, which handle relationships between words, and channel mixers, which refine the data flowing through the model. Instead of heavy matrix operations, it uses ternary weights—values restricted to -1, 0, or 1—that simplify calculations into basic additions. This design not only cuts energy use but also leverages Loihi 2’s ability to process sparse data efficiently. The researchers adapted the model by quantizing its weights and activations to fit the chip’s 8 bit and 24 bit limits, ensuring it runs smoothly without losing its language abilities. They also replaced complex mathematical functions with lookup tables and fast approximations, tailoring the software to the hardware’s strengths.
For those unfamiliar with neuromorphic computing, it’s worth noting that Loihi 2 operates differently from traditional processors. As opposed to executing instructions in a rigid sequence, it uses a network of artificial neurons that communicate asynchronously. That is only firing when triggered by new data. This imitates the brain’s event-driven nature, where neurons activate in response to stimuli rather than constantly ticking like a clock. The researchers mapped the MatMul-free model onto this system, distributing its workload across Loihi 2’s 120 processing cores. Early tests of a single model block on one chip showed promising alignment with simulations, hinting at the potential for full scale deployment.
The promise of the MatMul-free language model on Intel’s Loihi 2 becomes even clearer when looking at its performance compared to traditional setups. Researchers pitted this model against two transformer based language models running on an NVIDIA Jetson Orin Nano, a compact processor often used in edge devices like robots or smart cameras. These transformers, one with 400 million parameters and another with 500 million, represent the kind of AI systems widely used today. The comparison focused on two key tasks. Prefill, where the model processes a long input all at once, and generation, where it produces text one word at a time. Across various input sizes, from 500 to 16,000 tokens, where tokens are chunks of text like words or punctuation, the Loihi 2 setup consistently outperformed its rivals.
In the prefill phase, the MatMul-free model on Loihi 2 processed over 6,600 tokens per second, regardless of input size, while using just 3.7 millijoules of energy per token. By contrast, the transformer models on the Jetson peaked at around 3,861 tokens per second for the larger model and 3,153 for the smaller one, but only at specific input lengths—beyond that, their performance dropped. Energy use for these transformers ranged from 4.4 to 17.9 millijoules per token, meaning Loihi 2 delivered at least twice the processing speed with about half the energy cost. During generation, the advantage grew. Loihi 2 maintained a steady 41.5 tokens per second with 405 millijoules per token, while the transformers managed only 15.4 and 14.9 tokens per second at their best, with energy costs climbing as high as 1,200 millijoules per token. This translates to nearly three times the speed and half the energy use, a gap that widens as text length increases.
For the average person, this efficiency could mean AI tools that work faster and last longer on a single charge, whether it’s a virtual assistant drafting emails or a chatbot answering questions. The secret lies in the MatMul-free model’s design, which scales linearly—its resource demands grow predictably with input size, unlike the transformers’ more erratic scaling. Loihi 2 enhances this by keeping data close to where it’s processed, reducing the energy spent shuffling information around, a common bottleneck in GPUs. These early results, though based on a single chip and not yet fully optimized, suggest even greater potential with further tweaks.
Developers will appreciate how this stability simplifies building reliable AI systems. The Loihi 2 implementation spreads the workload across 120 processing cores, each acting like a mini-brain that handles its own piece of the puzzle. This parallelism, running many tasks at once, mirrors how the model processes language, making it a natural fit. Unlike traditional GPUs, which struggle with underuse at smaller scales and inefficiency at larger ones, Loihi 2’s performance holds steady. This consistency could streamline development for applications needing real-time text generation, like interactive games or customer service bots, all while keeping costs down thanks to lower energy needs.
Investors, especially those following Intel’s moves, should note Loihi 2’s edge over high end hardware too. Compared to an NVIDIA H100 GPU, a top tier processor for AI workloads, the MatMul-free model on Loihi 2 still shines in generation, offering three times the throughput and over 14 times better energy efficiency. In prefill, the H100 pulls ahead in raw speed for large inputs, but Loihi 2 remains competitive on energy, especially for smaller tasks. This balance makes Loihi 2 a strong contender in the expanding market for AI inference, the process of using trained models to make prediction, which spans edge devices to cloud servers. Intel’s neuromorphic push could tap into a total addressable market worth billions, as businesses and consumers alike demand greener, faster AI solutions.
The broader takeaway is that pairing smart software with tailored hardware can reshape what’s possible. The researchers conclude that neuromorphic principles, mimicking brain-like processing, unlock substantial gains in efficiency and speed. By stripping out heavy matrix operations, compressing data into low precision formats, and streamlining calculations, they’ve crafted a 370 million-parameter model that rivals bigger, thirstier systems. This matters as AI evolves toward reasoning tasks, like solving complex problems step by step, which demand fast, efficient text generation over long stretches. Loihi 2’s ability to handle this workload sustainably positions it as a platform for future innovation.
For everyday users, this could lead to AI that’s not just powerful but practical, running on devices you already own without a hitch. Developers get a toolset that’s both cutting-edge and cost effective, ready to power the next wave of intelligent software. And for investors, Intel’s Loihi 2 signals a strategic play in a competitive field, with the potential to carve out a significant slice of the AI hardware market. As refinements continue, this blend of neuromorphic computing and efficient language models offers a glimpse of an AI future that’s faster, leaner, and accessible to all.
Citation: Abreu, S., Shrestha, S. B., Zhu, R.-J., & Eshraghian, J. (2025). Neuromorphic Principles for Efficient Large Language Models on Intel Loihi 2. arXiv:2503.18002v1 [cs.NE]. https://doi.org/10.48550/arXiv.2503.18002
NeuromorphicCore

