AI Bubble or Real Value? How Banks Measure What Their AI Delivers for Customers

The AI boom is hotly debated: some see a bubble, others the start of a long upswing. For a bank, a different question matters more, and it can actually be answered: what has its own AI changed for its own customers? This is where the biggest gap lies. Many banks use AI, but few can prove its value. This article shows how large that gap is, why usage figures are not enough and how banks can measure the value of AI in their customer business properly.

Illustration: AI-generated, hypothetical example.

Hypothetical example: The board of a savings bank asks the head of retail customers what the AI assistant in customer service has delivered in its first year. She can show how often it was used and how satisfied the advisers are. Whether fewer customers left or more bought products that suit them, she cannot say. So she proposes using the assistant in half of the branches next quarter and keeping the other half as a comparison group. The board agrees. The bank, the people and the process are invented; the board's question is one many banks are asking.

What the a16z report says about an AI bubble

The report describes a strong boom but still shallow adoption inside companies. In its market report from September 2026, the investment firm Andreessen Horowitz (a16z) shows that technology stocks contribute around 76 per cent of S&P 500 earnings growth in 2026 (a16z, State of Markets II). The same report describes corporate AI adoption as broad but mostly shallow: nearly 30 per cent of S&P 500 companies report some "quantifiable impact" of AI, but only about 2 per cent report any tracked metric.

This shows why the AI bubble debate is so hard to settle. The investment is visible and large. The value on the user side, by contrast, is often not measured. As long as that remains the case, every assessment is more opinion than evidence. That applies to the market as a whole and to every individual bank.

Banks: many use cases, few proven results

In banking, too, the use of AI is growing faster than the evidence of its value. The Evident AI Index compares 50 of the largest banks in North America, Europe and Asia-Pacific. Its October 2026 edition finds that only 12 per cent of published use cases report impact on operational KPIs and barely 1 per cent disclose concrete financial returns (Evident AI Index for Banks 2026). Still, 12 of the 50 banks now report a realised or projected return across their AI activities, up from 8 a year earlier.

The banks that do publish figures show that it can be done. According to CEO Brian Moynihan, Bank of America has implemented around 140 AI use cases at a cost of about USD 400 million, generating around USD 800 million in benefits. JPMorgan Chase sets roughly USD 2 billion of investment against benefits of a similar size. Lloyds expects more than GBP 100 million in cost and revenue benefits from generative AI in 2026, BNP Paribas around EUR 750 million (S&P Global Market Intelligence, October 2026).

The same source also points to the limits of these figures. Analysts at Morningstar DBRS note that AI benefits are often embedded in broader transformation programmes, which makes attribution difficult. Banks also calculate differently: some include avoided costs, some subtract the investment, others do not. Comparisons between banks are therefore of limited use. For internal management, however, this is not an obstacle. A bank does not need to be comparable with others, only with itself.

Broader surveys confirm the picture. In McKinsey's "State of AI 2026", 37 per cent of respondents attribute any EBIT impact to AI, barely more than a year earlier. Only about 6 per cent are "high performers" who attribute at least 5 per cent of their EBIT to AI (McKinsey, State of AI 2026). According to McKinsey, this group is twice as likely as others to have defined processes for measuring the impact of its AI initiatives. Measurement is not a side issue here; it is a feature of the companies that actually create value.

Why usage figures are not enough

How often an assistant is used says nothing yet about whether customers benefit. Many reports show figures such as requests per day, number of users or minutes saved. That is a start, but it measures activity, not impact. An assistant can be used a lot and still change nothing about churn, complaints or sales.

A three-step ladder helps. The first step is usage: is the AI being used at all? The second step is operations: are requests resolved faster or more often at first contact, are there fewer follow-up questions? The third step is the customer outcome: do more customers stay, do they buy products that suit them and that they keep, are there fewer complaints? Only the third step answers the board's question.

At the third step, it also matters what should not happen. More sales are not a success if cancellations, complaints or marketing objections rise at the same time. Sound measurement therefore pairs every target metric with a guardrail metric. This prevents a system from optimising for short-term sales while damaging the customer relationship. We have described how to set target metrics across the whole of customer value in our article on customer value management in banking.

How banks measure the value of AI in customer business

The most reliable way is a comparison with a group that does not get the AI. A simple before-and-after comparison also measures everything else that changed in the meantime: interest rates, seasonality, a new campaign. A comparison group holds these influences constant. The principle is well known from medicine and marketing and can be applied to AI in customer business.

In practice, this means five steps:

1. Set the target metric in advance. Before the AI goes live, decide what it will be measured against, for example the churn rate for current accounts or the share of requests resolved at first contact. Anyone who looks only afterwards will always find a number that looks good.

2. Add guardrail metrics. Every target metric needs one or two metrics that must not get worse, such as complaints, cancellations or objections to marketing.

3. Form a comparison group. A randomly selected share of customers, branches or advisory teams works without the AI for a fixed period. The group must be large enough that differences are not just chance.

4. Log every recommendation. For later attribution, it must be clear which recommendation was shown when, on what basis and with which consent, and what came of it.

5. Translate the result into euros, but carefully. Lower churn and more suitable sales can be converted into a value using cautious assumptions. Those assumptions belong on the table so that the board and controlling can check them.

The comparison group has a cost: for a while, some customers do not get the new support. In return, it delivers a figure that stands up to scrutiny. For services that give customers a clear advantage, the comparison period can be kept short; after that, everyone gets the solution. Which use cases lend themselves to such a success measure is shown in our article on ten next best action use cases.

Traceability and data protection are the prerequisite

Without complete logs and clean consent, value can neither be measured nor justified. Attributing a result to AI requires every decision to be documented: trigger, data used, model version and rationale. A bank needs the same records for internal audit and for supervisory questions. Measuring and proving are two sides of the same task.

The comparison group must also comply with data protection rules. A customer who has objected to processing for marketing belongs in neither group of a sales measure. Purpose and consent must therefore be checked before an AI generates any recommendation at all. How this works in the context of an AI agent is described in our article on the context layer in banking.

Acceleraid is built for exactly these requirements. The AI Context Layer checks consent and purpose before every request and gives agents only the smallest context the purpose allows. Every access is logged in an audit-proof way, and a decision record captures trigger, sources, model version, excluded options and rationale. Processing takes place in Frankfurt, and customer data does not train third-party models. The basis for reliable measurement is therefore already in the data instead of having to be pieced together afterwards. The legal assessment of each individual use case remains the responsibility of the bank's compliance function.

What this means for the AI bubble question

A single bank cannot decide whether the market as a whole is overheated, but it can determine the value of its own AI. For the market, the question remains open, and the figures from a16z, Evident and McKinsey show one thing above all: evidence is lagging behind deployment. For a bank, this does not mean betting for or against the boom, but adopting a way of working. Every new AI application gets a target metric, a guardrail metric and a comparison group before it starts. What works is expanded. What does not work is stopped before it ties up budget.

This turns the question "bubble or not?" into one that can be answered anew every quarter. The head of retail customers in the example can then tell the board more than how often the assistant was used. She can show what it changed for customers.

Five takeaways

1. According to a16z, nearly 30 per cent of S&P 500 companies report a quantifiable impact of AI, but only about 2 per cent report a tracked metric.

2. Among the 50 largest banks, Evident finds that only 12 per cent of use cases report impact on operational KPIs and barely 1 per cent disclose concrete financial returns; 12 banks report an overall return.

3. Usage figures measure activity, not impact; what matters is the customer outcome, such as churn, suitable sales and complaints.

4. The most reliable evidence is a comparison group without AI, with a target metric set in advance and guardrail metrics that must not get worse.

5. Complete logs and checked consent are the prerequisite for measuring value and proving it to internal audit and supervisors.

Illustrations: AI-generated. AI-assisted content: We use AI technologies and automated agents in the creation of our articles, including from Microsoft, Google, OpenAI, Anthropic and other providers. Topics, editorial direction and final approval remain with our team.