Infrastructure control is key for sovereign AI stack, says IBM

India’s AI debate is often framed around what the country lacks – whether it’s frontier foundation models or leading edge indigenous semiconductor capacity. But according to IBM, infrastructure development is a key – and often underappreciated – part of the story.

“From an infrastructure perspective, are we building systems and hardware in India that are capable of supporting world-class AI? My answer would be yes. We are playing a significant role in making that happen,” says Rahul Rao, Distinguished Engineer, Processor Design, IBM India Systems Development Lab (or ISDL).

And Indian enterprises are already on the journey, even if they are at different stages of maturity, notes Subhathra Srinivasaraghavan, Vice President, IBM India Systems Development Lab. “Every one of them is talking about AI. Every one of them has started the journey.” 

IBM ISDL’s teams across India in Bengaluru, Pune and Hyderabad work on everything from processor design, firmware, operating systems, AI libraries, storage and system engineering – pretty much a full stack approach. According to Subhathra, ISDL designs and develops complex processors end to end from India, while Rahul’s team focuses on optimizing on-chip data movement, AI engines, and inferencing efficiency, among several other things.

That work also connects sovereign AI to a much longer semiconductor roadmap. “AI and quantum computing are advancing at breakneck speed, making us ask new questions of the devices we use. At IBM, our semiconductor research is focused on solving these fundamental engineering challenges through advances such as our sub-1 nanometer chip technology, enabled by our nanostack architecture,” says Rahul Rao.

Both of them believe that ISDL’s work goes beyond just building a national chatbot and planting a flag on the sovereign AI front. It is to develop talent and capability across essential components of the AI stack that makes it sovereign – everything from chips to systems, software to security, governance and deployment.

Achieving sovereignty over AI

The easiest part of sovereignty is often knowing where data is stored, and that’s why for years “data sovereignty” was treated as a real-estate problem. But AI has made this approach ineffective as well as outdated.

Sovereign AI is less about geography and more about control, as Rahul separates the issue into three layers: data residency, the controls governing the data layer, and the models that act on it to deliver intelligence.

“To truly achieve sovereignty, it’s important to have control over all of these layers,” says Rahul. “The data layer, where the data resides, is probably the easiest part because physically the data is either on-premises or hosted in a specific environment. The second part is the control mechanisms, and those have to be built in from the beginning. Just as you build security controls into the chip, you also have to build those controls into the system software.”

Also read: Not thinking about storage in AI is a mistake: Dell’s Venkat Sitaram

That distinction matters, because the same data that was earlier sitting in a passive database is now being actively interpreted inside AI systems – which have an insatiable appetite for data.

“When data was simply sitting in systems of record for bookkeeping purposes, it was largely passive. A query would retrieve it when needed,” Subhathra notes. “Now that same data is actively being used to influence decisions, generate outputs, and create insights. That is where concerns around personal information and data governance become much more significant.”

While most of the AI boom was trained by moving workloads towards specialised compute, that doesn’t make sense for inference-related workloads. And this is where IBM’s infrastructure argument becomes more interesting than a standard “buy our box” pitch. 

IBM isn’t envisioning every AI workload on a mainframe or Power server. It is arguing that enterprise AI needs a more dynamic strategy – which means large-scale accelerators whenever training runs are needed, smart inferencing based on where data resides, and the right AI models keeping latency, energy and control needs in mind.

“Training is extremely important for model creation, but AI is not just about the models you use. AI is about which model you choose. AI is about how you build the infrastructure around those models. AI is about how you implement governance based on your business requirements and applications. It’s highly customized,” Rahul emphasizes.

Sovereignty starts lower than the cloud

IBM’s view of infrastructure is deliberately broad. It involves processors, servers, and storage. Then comes firmware and operating systems, which leads to virtualisation, containers and AI libraries – all key layers below any AI-enabled business application running on top.

Also read: Yotta to Adani: India building sovereign, frontier AI with Global South relevance

That full-stack perspective is central to IBM’s sovereign AI story. Securing only the application layer, or just the silicon-level, or just the firmware, all in isolation isn’t the same as securing the entire tech stack. “You need to control the firmware. You need to control the system software. You need to have confidence in the protection mechanisms that sit across the stack,” explains Subhathra.

Asked to explain how silicon-level security lapses can still make absolute control impossible, Rahul is refreshingly candid about the trust problem facing any multinational technology provider. How can, for example, a customer be absolutely sure that there’s no inadvertent backdoor on IBM-designed chips?

“To some extent, it’s very difficult to prove conclusively that there is no backdoor,” says Rahul. “We can explain how we design systems and what controls we have in place, but if somebody asks for a document that verifies every possible entry point and exit point, that’s almost impossible, not just for us but for any technology provider.”

Another myth about sovereign AI is how it’s often confused with encryption or confidential computing. They are related, but not interchangeable, according to Subhathra, who describes them as components within a larger design.

“To achieve sovereignty, you may use confidential computing. You may use encryption. You may use quantum-safe algorithms. You may need a variety of technologies. But sovereignty is a broader concept that sits above all of those,” says Subhathra.

While encryption protects data, sovereignty governs its entire life. Because an encrypted file can still be copied to an unauthorised location, a protected AI model can still be operated under policies with improper control.  “Hence, sovereignty is about having control over what happens to the data, not just protecting the data itself,” reiterates Rahul.

“The entire framework needs to be thoughtfully designed rather than simply pursuing a random use case,” emphasizes Subhathra, pointing out why IBM sees so many AI journeys stall at the pilot stage. “They don’t progress beyond that because it becomes very difficult to retrospectively address all of these considerations.”

Sovereign AI doesn’t always mean new hardware

So how will sovereign AI percolate through big industries still extracting ROI from legacy tech? Sectors such as banking, government, power and insurance, do they just throw out all the old tech infrastructure and spend more for modern AI? Subhathra rejects the idea wholeheartedly, suggesting that modernisation doesn’t automatically mean abandoning enterprise platforms.

“Modernization isn’t about moving to a different platform. It’s about ensuring that, whichever platform you’re running on, your architecture can scale, remain agile, and support the diverse business requirements that continue to emerge.”

A decades-old system isn;t obsolete if it remains resilient, secure and capable of running modern software. IBM points to Linux on Z, open-source support, and AI-assisted upgrades as ways to modernise architecture without having to trash old stack that still works – and which can still enable sovereign AI going forward.

The essence of what IBM’s Subhathra and Rahul are saying is that sovereign AI demands enterprises to make uncomfortable choices they might have conveniently avoided until now. Choices about data and control, open standards and operational responsibility. It will also require organisations to resist the temptation of simply buying an AI model and calling it a sovereign strategy.

Also read: RBI’s draft AI model ignores agentic AI risk, experts warn