RBI’s draft AI model ignores agentic AI risk, experts warn

“It covers models. One model, one decision, one explanation. That logic made sense until very recently,” according to Vikram Raichura, Founder and MD of Helo.ai by VIVAConnect. He was commenting on the Reserve Bank of India’s draft AI model-risk framework that’s open for comments until July 24.

The thing you have to understand about RBI’s draft AI framework is that whatever they finalize will be adopted by India’s banking system. Right now, as things stand, RBI’s draft AI framework covers AI models very well. Experts suggest while the draft’s instincts are good, it hasn’t accounted for agentic AI use cases. 

Raichura describes the gap clearly. “Agentic AI is not one model making one call. It’s a chain. Each model feeds the next. The risk doesn’t sit inside any single model. It sits in between them, in how they interact. The current draft doesn’t look at that layer. That’s the gap.”

If our banking workflows have to be AI-enabled, their tasks will lean not only on frontier models but also agentic AI. Whether it’s a credit score or new loan application, customers may only see one number or decision, but that doesn’t mean it didn’t take several AI agents for banking systems to get there. That bigger picture needs protection as well.

Praveer Kochhar, Co-Founder and CPO of KOGO Tech Labs, explains why a model-by-model test can miss the larger danger: “A loan approval, for example, might pass through five or six agents in a row. One checks eligibility, another pulls credit data, another runs a fraud check, another approves disbursal and one more sends the confirmation to the customer. Each one hands off to the next and trusts what it received. There’s no single model that made the decision but it was a chain of decisions that led to the outcome.”

Also read: Sam Altman says letting AI automate everything will be dangerous and unfulfilling, here is why

Individually, every component of a task completed by an AI model might receive a go ahead in terms of risk. But what about AI agents that orchestrate different models in multiple steps to arrive at a decision? Approving the sum of the parts isn’t the same as approving the final result, in this case.

Bikash Barai, Founder and CEO of FireCompass, makes that distinction explicit. “A model can be perfectly fine in isolation while the orchestrated system still does the wrong thing, because the risk lives in how agents interact, what they pass to each other, and what tools they can call. All of those aspects need to be addressed explicitly, and the current draft does not yet do that,” says Barai.

Experts also highlighted the RBI draft AI risk framework’s point on who decides and carries the maximum authority of any AI model-based workflow. And more importantly when human approval becomes compulsory.

Raichura’s warning is especially relevant for banks buying agentic platforms from vendors: “Can you see every step the system took and why? Can you set limits the vendor cannot override? If a transaction goes wrong mid-flow, can you stop it and reverse it? If the answer is no, you own the accountability but not the control. That’s a position no bank should be comfortable in.”

The RBI correctly keeps accountability with the regulated entity when technology is outsourced. But it also needs to account for future changes – what if the orchestration layer changes? Scope of permissions expand unduly? Who takes the onus in these scenarios and more?

Kochhar argues that other controls must sit beside it. “First, bounded autonomy means deciding before any of this goes live. It tells you exactly what each agent is allowed to do on its own and when it has to check with a human or another system first. Without this, one agent’s authority can quietly expand as tasks get passed along the chain. Second, audit trails need to record the whole journey.”

For any AI model or agentic AI workflow, it should explain how it reached the final decision for any given task. Because reconstructing the chain is essential for governance and auditing purposes. Because agents act rather than merely recommend, banks need reversibility of action in case something goes wrong.

Barai places the priorities in the correct order. “Visibility alone is not enough. You must be able to respond. If something goes wrong, can I shut it down cleanly? That is harder than it sounds, because agents typically spin up other agents. Cleaning everything up and shutting it all down without leaving anything running is a genuine engineering problem. Alongside that: can a human stay in the loop, and can I influence an agent during its actions, not just review it afterwards?”

The RBI has built a strong foundation. Banks need to use the consultation window to demand workflow-level risk classification, agent-level authority matrix, view of hand-off logs and practical reversibility. Otherwise, India may end up with excellent safeguards for the model that explained the decision but inadequate governance for agents that orchestrated the decision.

Also read: If AI agents replace human workforce, should AI be taxed?