Neuchips is focusing on inferencing chips and introducing technologies that boost efficiency in the semiconductor industry.
In 2023, the global semiconductor market faced significant challenges in 2023, with a noticeable decline in sales. According to the Semiconductor Industry Association (SIA), the market saw a decrease in sales to US$526.8 billion, marking an 8.2% drop compared to the previous year. This downturn was heavily influenced by a 37% decline in the memory sector, identified by Gartner Inc. as the segment with the largest decrease across the industry.
AI’s emerging role in market resurgence
Despite the overall downturn, the AI sector emerged as a bright spot, particularly in the second half of the year. Counterpoint Technology Market Research highlighted AI as a crucial driver of content and revenue within the semiconductor industry. This growth was propelled by the increasing use of AI-based applications across various domains, including data centers, edge infrastructure and endpoint devices.
Gartner forecasts a robust recovery led by AI technologies, with AI chips generating US$53.4 billion in revenue in 2023—an increase of about 21% year-over-year. The sector is expected to continue its rapid growth, reaching US$67.1 billion in 2024 and potentially doubling in size to US$119.4 billion by 2027.
Ken Lau, CEO of Neuchips, highlighted the potential for AI, particularly generative AI, which could evolve into a trillion-dollar market by the 2030s, with significant investments transitioning from training to inferencing technologies. “After training data, inferencing can significantly enhance operational efficiencies,” Lau said. He further explained that AI could transform customer interactions and digital marketing strategies, leading to direct consumer engagement and sales online.
“I think there are ways that we can’t even imagine going forward. The opportunities are limitless for AI. That’s how I see it. And a big part of that is going to be inferencing, not just training,” he added.
Inferencing as a strategic focus at Neuchips
Founded in 2019, Neuchips has strategically emphasized inferencing, particularly through its recommendation engine, which is extensively utilized in data centers to enhance online shopping experiences.
Lau stated that its initial prototype used field programmable gate arrays (FPGAs) (a type of semiconductor device) to validate the design, leading to the development of the N3000 inference chip. The chip outperforms competitors by 1.7 times per watt according to MLPerf 3.0 Benchmarking.
Lau explained that when Neuchips designed this chip, it was with the recommendation engine in mind. However, when GenAI gained momentum, the AI company tested it with its chip and achieved compatibility due to optimized memory subsystems, applicable to both recommendation engines and GenAI.
Recognition and future plans
The N3000 chip won the “Best AI Chip” award at the EE Awards Asia 2023, reflecting the company’s high execution standards. Lau emphasized the efficiency of their development process, noting that the chip was successfully completed with a single production run—a significant achievement for a startup operating with limited resources.
Industry challenges and future outlook
Despite the optimistic growth projections for AI chips, the semiconductor industry faces several challenges, particularly in software integration and power consumption. Lau remarked on the integration challenges, citing examples from healthcare where data privacy and internal training are crucial. Neuchips addresses these challenges by offering low-power chips suitable for compact spaces and providing comprehensive software support, including software development kits (SDKs) and training services.
Looking ahead, Neuchips announced advancements in their Evo series, including the Raptor Gen AI inference chip, which supports Large Language Models (LLMs) and is set to launch in Q2 2024. As the AI applications ecosystem expands, Neuchips plans to innovate continuously in chip design, focusing on various form factors to accommodate diverse uses, including PCs and low-end workstations.
Also read:
- Top 7 Companies in the Semiconductor Chip Manufacturing Industry
- Elon Musk’s Neuralink Debuts Brain Chip Implant: A Bold Future with Ethical Questions
- The World Is Facing A Semiconductor Chip Shortage, Here’s Why
Header Image from EETimes Taiwan via PR Newswire
Press release link: https://www.prnewswire.com/news-releases/neuchips-driving-ai-innovations-in-inferencing-302118337.html
