No black-box AI: RBI proposes strict governance framework for banks' AI systems

The RBI has proposed a comprehensive regulatory framework to govern the use of artificial intelligence and other decision-making models across banks and financial institutions, requiring entities to build "kill switch" mechanisms for AI systems, ensure human oversight over automated decisions, and independently validate even third-party AI models before deployment.

The draft Guidance on Regulatory Principles for Model Risk Management, 2026 marks the Reserve Bank of India's most detailed attempt yet to regulate the rapidly expanding use of AI, machine learning (ML), predictive analytics and other models in India's financial system. The framework will apply to commercial banks, small finance banks, payments banks, cooperative banks, regional rural banks, NBFCs, All India Financial Institutions, asset reconstruction companies and credit information companies.

The draft comes at a time when financial institutions are increasingly deploying AI-powered credit underwriting, fraud detection, customer service chatbots, treasury management systems and risk models. RBI noted that while such technologies improve efficiency and customer service, they also introduce significant model risks that could lead to flawed decisions, operational disruptions, compliance failures and financial losses if not adequately governed.

Mandatory 'kill switch' for AI models

One of the most notable provisions requires regulated entities to establish mechanisms to override, suspend or completely deactivate AI models whenever necessary.

The RBI has proposed that institutions implement robust human oversight mechanisms for AI-driven decision-making, including human-in-the-loop arrangements, override capabilities, suspension features and explicit "kill-switch arrangements." Banks would also be required to periodically review AI-generated decisions to detect anomalies and strengthen oversight based on operational experience.

The central bank has also cautioned institutions against automation bias and excessive dependence on AI-generated recommendations. It said employees responsible for supervising AI systems should possess sufficient expertise to challenge or override model outputs whenever required.

Human alternative for customer-facing AI

For AI models interacting directly with customers, including generative AI systems, RBI has proposed several additional safeguards.

Financial institutions would have to clearly disclose to customers whenever they are interacting with an AI-powered system and communicate the limitations of such technologies. More significantly, customers must be given an option to switch from AI assistance to a human representative upon request.

The draft also mandates additional cybersecurity protections for customer-facing AI systems, including safeguards against prompt injection attacks, adversarial inputs, limits on session persistence and mechanisms to detect abnormal usage patterns.

AI-specific risk controls

Recognising the unique characteristics of AI models, RBI has prescribed a separate set of governance principles beyond traditional model risk management.

Banks will have to establish explainability thresholds for AI models, particularly those involved in material decision-making such as lending or customer servicing. Where complete explainability cannot be achieved, institutions must compensate through enhanced validation, more frequent monitoring, corroboration of outputs and restrictions on model usage.

The regulator has also directed financial institutions to specifically address AI risks such as hallucinations, discriminatory outcomes, data drift, concept drift and excessive output variability.

For generative AI models, institutions will need to implement system-level controls to minimise hallucinations, particularly where AI-generated content influences customer interactions or business decisions. They must also identify and mitigate algorithmic bias to prevent unfair treatment of specific customer groups and ensure models are not overfitted to training data.

Additionally, RBI wants institutions to conduct structured "red teaming" or equivalent stress testing for AI models, especially those capable of interacting with customers or generating content autonomously.

Third-party AI models under tighter scrutiny

The framework places significant responsibility on regulated entities using AI systems sourced from external technology providers.

The RBI has made it clear that banks and financial institutions will remain accountable for outcomes generated by third-party AI models regardless of vendor assurances or certifications. Every external model must undergo independent validation by the regulated entity itself before deployment.

Institutions must also conduct due diligence covering the technology provider's credibility, model methodology, data quality and limitations. Contracts with AI vendors should provide access to sufficient technical documentation, permit audit rights for both the institution and RBI, and include business continuity and exit arrangements.

Where technology providers fail to disclose adequate information about AI systems, institutions would be expected to identify resulting risks and impose appropriate usage restrictions.

Board-level oversight

Beyond AI, the draft introduces a comprehensive model risk management architecture covering every model used by financial institutions, whether internally developed or procured from third parties.

Every regulated entity will have to establish a Board-approved Model Risk Management Framework (MRMF) covering governance, model classification, validation, deployment, monitoring, business continuity and decommissioning. The board will oversee the institution's overall model risk appetite, while the Risk Management Committee of the Board will review high-risk models before deployment and monitor AI models and third-party systems.

Institutions must classify all models according to risk, maintain detailed inventories of every active and retired model, and ensure no model is deployed unless formally recorded within the inventory. Decommissioned models must remain documented for at least 10 years.

Independent validation mandatory

The RBI has proposed that every model—including AI systems and third-party models—must undergo independent validation before deployment and periodically thereafter.

Validation must assess model inputs, assumptions, conceptual soundness, performance and alignment with intended use. Validation reports are required to be submitted to the institution's board-level risk committee or delegated authority within three months of completion.

The regulator has also required institutions to maintain structured change management processes so that any material modification to a model automatically triggers fresh validation and approval before being put into production.

The draft guidance, released for public consultation, will eventually replace the RBI's existing 2002 guidance relating to credit risk models once finalised.