Recently, a company called d-Matrix launched out of stealth mode with a $44m Series A round. The co-founders are proven, long-time SV tech innovators. They've developed the first 'baked' in-memory AI model which, for a change, actually is different than what's been out there.

To find out more, VMblog spoke with Sid Sheth, founder and CEO at d-Matrix.

VMblog:  What is the challenge facing the current ‘near-memory’ AI compute methodology?

Sid Sheth:  The dominant AI compute model has a problem. The laws of physics show near-memory AI compute cannot scale with the explosion of transformer-led demand for AI. Memory and energy limits have been reached and money and resources cannot overcome what is physically impossible. Early generation AI has reached the memory wall.

In-memory AI compute is the answer and d-Matrix is aiming to be the first to bring digital in-memory AI compute to market. The emergence of transformers (developed by Google engineers), represent major progress in AI but, transformers consume unsustainable volumes of energy and memory. The figures bear this out – Microsoft sees 86 billion inferences per day; Facebook 200 trillion inferences per day cannot be supported this way. If the model doesn’t change, in 10 years, we could be spending the entire GDP of the US to run AI compute. 

VMblog:  Why do you believe d-Matrix has the solution to this problem?

Sheth:  d-Matrix has been on a three-year journey to build the world’s most efficient computing platform for AI inference at scale. We’ve developed a path-breaking, truly digital in-memory compute (DIMC) architecture and NighthawkTM silicon chiplet that are practical to implement and significantly advance AI compute efficiency far past the memory wall it has hit today.

We’re bringing the first DIMC-based inference compute platform to market, as transformer-led demand for AI explodes and current memory and energy limits are hitting a threshold. We’ve built a novel AI compute platform that uses a combination of intelligent ML tools and frictionless software approaches coupled with chiplets in a Lego block formation enabling integration of multiple programming engines on a common package. d-Matrix has proven its thesis with multiple chiplet developments — the Nighthawk platform we announced last week, and the soon to be released Jayhawk platform. Using this first-of-a-kind compute architecture and DIMC, d-Matrix will be able to deliver an increase in compute efficiency, several times over, and ensure their clients receive massive gains in performance levels, without compromising on energy costs. 

VMblog:  Does the industry agree? What kind of backing have you attracted and what’s your valuation?

Sheth:  For a four-year-old business, we’ve seen a huge amount of interest from industry. At Series A, a slew of heavyweight industry investors have pledged $44 million in Series A funding to bring the vision to life. The investment has been led by US VC Playground Global. M12 (Microsoft Venture Fund), and SK Hynix have joined our existing investors Nautilus Venture Partners, Marvell Technology (Nasdaq: MRVL) and Entrada Ventures in this Series A round. The company’s valuation as indicated by Pitchbook is $102m.

VMblog:  There’s some serious competition in this space, what makes you confident you’ll be the first to market?

Sheth:  The serious competition is a great validation of the size of the opportunity and the scope of what’s at stake. D-Matrix is the first company in the world to develop and commercialize a chiplet based digital in-memory compute platform for transformers. We don’t see any other company taking this approach and we have a strong lead with all our Nighthawk and Jayhawk chiplet developments. So, we remain quietly confident that we can be the first to bring our solution to market and solve some critical problems for the industry.

VMblog:  What does the innovation roadmap look like?

Sheth:  We expect to launch our first product, code-named Corsair, in the second half of 2023. Meanwhile, we continue work on its two foundation prototypes, nicknamed Nighthawk^TM and Jayhawk, which provide the basic in-memory computing platform as well as a custom interconnect for d-Matrix chiplets. We expect to be moving towards a second funding round later in 2022.

VMblog:  Who is d-Matrix? Give us a brief history.

Sheth:  The company was founded in 2019 by both myself, Sid Sheth, and Sudeep Bhoja, and our backgrounds include building power-efficient compute and interconnect solutions for datacenters over the last 20+ years. d-Matrix was founded with a clear focus on building the world’s most efficient hardware platform for datacenter inference.  

Previously, we successfully built a $500m business at Inphi (now Marvell Technology) from the ground up, to service the largest datacenter operators in the world. Prior successes have seen the d-Matrix team deploying their solutions at large hyper-scale cloud operators such as Google, Meta, AWS and Microsoft. This gave them credibility with, and early access to, these strategic customers. d-Matrix, in conjunction with some of these customer organizations, picked in-memory compute as the approach that would give maximum gains in compute efficiency – targeting it at Transformer workloads and Large Language Models for NLP and other multi-modal (video, text, speech) applications.  

We’re based in Santa Clara, California and currently have over 50 employees (although we’re growing quickly), 40% of whom are PhDs.

VMblog:  Who's involved? Anyone we know?

Sheth:  We are fortunate to have received the support of:

• Sasha Ostojic (ex-VP Software at Nvidia, Cruise) and Peter Barrett (Lead Investor in PsiQuantum, Universal Hydrogen), both from Playground Global
• Microsoft through their M12 venture fund.
• Sasha Ostojic (Playground Global) and Michael Stewart (M12) have joined d-Matrix's Board of Directors, too.