Economic Survey 2025: Indian banks bet big on AI, but “Black-Box” risks loom

AI

Indian banks are aggressively integrating artificial intelligence (AI) across operations, from credit underwriting to fraud detection, as they embrace the digital era. However, as AI transforms banking, concerns over transparency, accountability, and cyber threats are mounting.

The latest Economic Survey notes that “the rapid pace of technological evolution in India, particularly in areas like AI, blockchain, and data analytics, has created new opportunities to reimagine traditional financial services and processes.” Banks are leveraging AI for key functions such as regulatory capital planning, liquidity management, portfolio optimisation, and personalised customer service.

"AI and large language models (LLMs) have enhanced customer service through interactive
chatbots and personalised experiences, while blockchain offers secure, transparent,
and efficient transactions," it notes.

While AI promises efficiency, its “black-box” nature raises critical challenges.

"The black-box nature of AI systems can make it difficult to assess the system's reliability or
contest its decisions. This lack of transparency can lead to trust concerns and challenges
in validating the fairness and accuracy of AI decisions, making it challenging to audit or
interpret the algorithms that drive the decisions,' it said.

In AI, a "black box" means the system is not transparent with inputs and operations not visible to the user or another interested party. A black box, in a general sense, is an impenetrable system, making it difficult to see or understand how decisions are made.

RBI extends MuleHunter.AI to more banks, tightens noose around mule accounts

Other risks outlined in the survey include:

Human oversight concerns – “inadequate human oversight, over-reliance on AI, and loss of human expertise.”

Cyber threats – as AI-driven systems expand, so do vulnerabilities to sophisticated cyberattacks.

Malicious AI usage – “synthetic identity frauds, rogue trading, and market manipulation” are flagged as serious concerns.

Third-party dependencies – banks increasingly rely on external AI service providers, leading to risks of “service provider concentration.”

"Establishing robust AI governance is the first and crucial step in addressing the challenges that come with the implementation of AI systems. Without an appropriate governance framework, AI systems may operate without clear guidelines or oversight, leading to potential abuse or misuse of technology," it added.

RBI sets up expert panel to craft ethical AI framework for financial sector

RBI steps in

Recognising these risks, the RBI has been actively engaging with banks and experts to establish oversight mechanisms. It has launched a regulatory sandbox for testing AI-driven innovations and announced the formation of a Framework for Responsible and Ethical Enablement of Artificial Intelligence (FREE-AI).

The central bank also cautions that regulatory and supervisory effectiveness may take a backseat if financial regulators’ AI-related skills and knowledge do not keep pace with developments in this space.