A Brain-Inspired Chip Can Run AI With Far Less Energy

Artificial intelligence can't grow at their current pace. Every year deep neural networks get bigger because they are inspired by the brain. Hardware improvements aren't keeping up with the amount of memory and processing capacity required to run these large programs. Artificial intelligence may hit a wall in the near future.

Even if we were able to keep scaling up hardware, it would still take a lot of energy to run them on a traditional computer. The high carbon emissions generated from running large artificial intelligence programs are already harmful for the environment and will only get worse as the program grows.

One solution uses inspiration from the brain to create energy efficient designs. The chips lack the computational power needed to run a large deep neural network. That makes it easy for them to be overlooked.

In August, Weier Wan, H.-S. Philip Wong and their colleagues revealed a new chip called NeuRRAM that has 3 million memory cells and thousands of neurons. The type of memory used is called RRAM. NeuRRAM is reprogrammable to save more energy and space. Digital memory can store either a 1 or a 0, but the NeuRRAM chip can store multiple values at a time. In the same amount of chip space, the chip can store more information.

As a result, the new chip can perform as well as digital computers on complex artificial intelligence tasks like image and speech recognition, and the authors claim it is up to 1,000 times more energy efficient.

The results have been very good for researchers not involved in the work. A long time RRAM researcher at the University of Hong Kong said that the paper was unique. At the device level, at the circuit architecture level, and at the algorithm level, it makes contributions.

Creating New Memories

Every single computation in a digital computer is inefficient because of a simple design flaw. The computer's memory is placed on the board away from the processor where computing takes place.

The amount of time spent on the commute is similar to the amount of time spent on the processor, said Wan, a computer scientist at Aizip.

It seems straightforward to fix this problem with all-in-one chips. It is closer to how our brains process information due to the belief that computation happens within populations of neurons. Current forms of memory are incompatible with the technology in the processor, making it hard to create such devices.

A technology known as compute-in-memory was created by computer scientists decades ago. These ideas were overlooked for a long time due to the performance of traditional digital computers.

Wong said that the work was forgotten because it was similar to most scientific work.

The first such device dates back to 1964, when electrical engineers discovered they could manipulate metal oxides to turn them on and off. The ability of a material to switch between two states is significant. Digital memory has a state of high and low voltages.

If you want an RRAM device to switch states, you need to apply a voltage to the metal oxide. Metal oxides are not conductors of electricity. The current builds up as it pushes through the material's weak spots and leads to the other side. The current can flow freely once it breaks through.

When enough charge builds up inside a cloud, it can find a low-resistance path and strike lightning. The path through the metal oxide is the same as with lightning. The path can be erased by applying another voltage to the material. Researchers can use RRAMs to store digital memory.

Researchers didn't realize the potential for energy efficient computing and didn't need it yet. Researchers realized the possibilities after the discovery of new metal oxides.

A colleague working on RRAM admitted that he didn't fully understand the physics involved, recalls Wong. Wong thinks he should not try to understand it if he doesn't understand it.

In 2004, researchers at SAMSUNG ELECTRONICS announced that they had successfully integrated RRAM memory on top of a traditional computing chip, suggesting that a compute-in-memory chip might be possible. Wong wanted to at least attempt.

Compute-in-Memory Chips for AI

Wong worked for more than a decade to build up RRAM technology to the point where it could reliably handle high-powered computing tasks. Around 2015, computer scientists began to realize the enormous potential of these energy efficient devices for large artificial intelligence programs. Scientists at the University of California, Santa Barbara demonstrated that RRAM devices could do more than just store memory. The majority of computations within a neural network are simple matrix multiplication tasks.

The NeuRRAM chip stores the weights of the connections between the chips in the RRAM memory cells. The weights that the NeuRRAM memory cells hold represent the full range of resistance states that occur when the device switches between a low-resistance to a high-resistance state. The chip can run many matrix computations in parallel, instead of in lockstep one after another, as in the digital processing versions.

Since digital processing is decades behind analog, there are many issues that need to be worked out. Since the physical chip can introduce variability and noise, it's important that the chips are very precise. These flaws don't matter as much for traditional chips. If the RRAM device isn't exactly the same every time, the accuracy of the images will suffer.

Wong said that the lighting path was different every time they looked at it. Every time you program them, they are slightly different. The NeuRRAM chip was created by Wong and his colleagues after they proved that RRAM devices can store continuous artificial intelligence weights and still be as accurate as digital computers.

Flexibility was one of the major issues they had to solve. It used to be that chip designers had to line up the tiny RRAM devices in one place. The computation could only be done in one direction because of the hard-wired devices. Extra wires and circuits were needed in order to support neural networks.

A new chip architecture was created by Wong's team. The change to the design saved energy.

Melika Payvand is a researcher at the Swiss Federal Institute of Technology. I think it's a really great work.

For several years, Wong's team worked with partners to design, manufacture, test, calibrate and run artificial intelligence software. They considered using other emerging types of memory that could be used in a compute-in-memory chip, but RRAM had an edge because of its advantages in analog programming, and because it was relatively easy to integrate with traditional computing materials.

Their recent results represent the first RRAM chip that can run large and complex artificial intelligence programs, a feat that has only been possible in theoretical simulations. Anup Das is a computer scientist at Drexel University. It's the first demonstration.

Digital artificial intelligence systems are more precise but less efficient. The chip has bridged the gap for the first time.

Scaling Up

The NeuRRAM chip is just the size of a fingernail and can serve as an analogue processor. While it can run neural networks at least as well as digital computers do, the chip also has the ability to perform computations in different directions. Their chip can input a voltage to the rows of the RRAM array and read outputs from the columns as is standard for RRAM chips, but it can also do it backwards from the columns to the rows, so it can be used in neural networks that operate in different directions.

This has been possible for a long time, but nobody thought to do it. "Why didn't we think about this before?" Payvand wanted to know. I don't know

There are a lot of other opportunities that can be opened up by this. He mentioned the ability of a simple system to run the large amount of computations required for multidimensional physics simulations or self-driving cars.

Size is a problem. Billions of weights are contained in the largest neural networks. Wong wants to stack NeuRRAM chips on top of each other.

Keeping the energy costs low will be just as important as scaling them down further. One way to get there is by copying the brain and using the electrical spike to communicate. A signal is sent from one neuron to another when the difference between the inside and outside of the cell is critical.

Tony Kenyon is a researcher at University College London. It is possible that you will have more energy efficiency if you use very sparse spikes. The current NeuRRAM chip would likely need a completely different architecture to run spike-based programs.

New hope has been created by the energy efficiency the team achieved while running large artificial intelligence programs on the NeuRRAM chip. One day, we might be able to match the human brain's 86 billion neurons and trillions of synaptic connections without running out of power.

The quantum-safe future will be worked on.