At CIO100 in Frisco, Texas, roughly 100 rising technology leaders sat down for our “Next CIO” session. The group was asked to reflect on a single question: Are you ready to take on the role of CIO? Using the CIO Readiness Framework that we have developed and refined over years of advisory work, we asked each person in the room to score themselves across the five dimensions of the framework. The results point to a gap that should worry any organization building its next generation of technology leaders.

The CIO Readiness Framework

The CIO Readiness Framework organizes the CIO job into five dimensions. We asked each rising leader to score themselves on the same 1-to-5 scale, from “Emerging” to “CIO-Ready.” The five dimensions of the framework are:

• Enterprise leadership: the ability to lead beyond your own function, anticipate where the business is headed and mobilize people through change.
• Business value and financial acumen: understanding how the enterprise makes money well enough to connect technology decisions to growth, margin and risk.
• Influence, narrative and enterprise selling: building belief and support before a decision is ever formally proposed, not just presenting sound logic once it is. 
• Relationships, talent and operating leverage: building trusted executive relationships, developing successors and creating an organization that delivers beyond your own personal reach.
• Technology stewardship and digital judgment: the technical fluency and architectural judgment needed to make durable enterprise technology decisions.

Where the room stands

Across the five dimensions, the average self-assessment landed at 3.4 out of 5, squarely in ‘Proficient’ territory. Consider who was in the room: people already selected by their own organizations as ready to be developed for the next level. Even so, not one of the five dimensions averaged ‘Advanced’ or higher across the entire group. Technology Stewardship and Digital Judgment (the ability to make sound decisions on platforms, architecture and risk) came in as the most mature dimension in the room. At the bottom sat two dimensions in a near tie: Influence, Narrative and Enterprise Selling; and Relationships, Talent and Operating Leverage.

Much more interesting, however, is the spread between the highest- and lowest-rated dimensions. On these bottom two dimensions, ~65% of attendees rated themselves Proficient or below. Compare that to Technology Stewardship, where the number was only 36%. Put plainly, the people in that room are confident in their technical judgment. They are far less confident in the parts of the job that have nothing to do with technology at all.

Why the human dimensions lag, and what to do about it

This tracks with what we hear constantly in our advisory work. Most people who reach the doorstep of the CIO role got there by being excellent at the technical and operational core of IT. Few of them spent their first fifteen years being evaluated on stakeholder mapping, coalition-building, or developing a successor. Those muscles simply were not required until now.

The good news is that these are learnable skills. We recommend a simple approach to close these capability gaps: for the dimensions where you rated yourself lowest, identify a goal that targets your weaknesses, then attach a tactic (a concrete action or behavior) that moves you toward achieving your goal. Lastly, give the whole thing a timeframe. Six months is often a good starting point, as it is long enough to make real progress and short enough that you’ll actually check.

In this activity, the goal represents the destination – for example, to develop a brand of “enterprise leader,” rather than just “strong IT operator.” The tactic is how you get there, something specific enough that you’ll know in six months whether you did it or not. “Get better at influence” is a goal with no tactic attached, which is exactly why it rarely changes anything. “Hold pre-alignment conversations with three sponsors before my next major proposal” is a tactic, and it’s either done or it isn’t.

Here’s what that pairing looks like applied to the two lowest-scoring dimensions from the CIO100 room:

• For Influence, narrative and enterprise selling, a reasonable goal is building support for ideas before they ever reach a formal decision point. Tactics in service of that goal include identifying the informal decision-makers behind a priority and earning their support early, or taking on an external opportunity (e.g., industry panels, published point of views) to build credibility beyond the building.
• For Relationships, talent and operating leverage, a reasonable goal is creating executive capacity instead of personally absorbing more of the work. Tactics in service of that goal may include adding standing one-on-ones with two peers on the executive team, and delegating two recurring items off your own plate with clear decision rights attached.

The takeaway for CIOs building their bench

If you’re a sitting CIO developing your own successors, this data serves as a useful gut check. The people you’re grooming may already operate at an advanced level technically while carrying real gaps in the skills that determine whether they succeed once they have the title. Executive presence, coalition-building and delegation take years to build, so the earlier you start, the better.

The future leaders we worked with at CIO100 had no shortage of ability. What most of them lacked were the specific, practiced habits that turn a strong technology leader into an enterprise one, and the self-assessment data shows they already know it. Acknowledging that gap is the first step toward closing it.

Concerns about the risks of AI systems are certain to be met with four words: human in the loop. The discussion may broaden, but the assurance is inevitable. It’s an AI governance phrase that’s become so rote you hear it in every direction and likely have said it yourself.

But IT leaders should be wary of vendor or team claims that they’ve built human-in-the-loop systems into AI tools because some of these supposed guardrails are no more than rubber stamps.

Some so-called human-in-the-loop systems don’t give employees overseeing the AI tools either the control or the time necessary to fix any problems, some IT experts point out.

For human-in-the-loop systems to actually work, employees overseeing AI tools need to have the domain knowledge and context to take the action the AI tool is addressing when the AI isn’t involved, and they need to have the authority to override the AI decision, says Doug Shepherd, head of offensive security at internet services provider Cloudflare.

Promises of human-in-the-loop systems give IT leaders comfort, but the underlying process often doesn’t work as advertised, he adds.

“If your human in the loop can flag something but can’t actually stop it, that’s not human in the loop, that’s a human adjacent to the loop,” Shepherd says. “That’s performative governance.”

Shepherd, speaking at the recent CIO 100 Awards and Conference in Frisco, Texas, encouraged attendees to embrace AI and focus on projects that drive adoption and impact. Organizations that fail to push AI initiatives will be left behind, he suggested, but he also warned that blind adoption, without focusing on meaningful outcomes and guardrails, can lead to huge setbacks.

Many organizations reach for human in the loop as an important control, but no one stress tests it, he adds. “It gets projects approved, and too often, it does the political work, but not the risk work,” he says.

Darren Kimura, CEO and president at AI integration platform vendor AISquared, agrees that many organizations are deceiving themselves with so-called human-in-the-loop systems.

“Most companies that say they have a human in the loop actually have a human watching the loop,” he says. “The person can see the decision and flag a concern, but they cannot stop it, change it, reject it, or escalate it.”

IT leaders should ask themselves a handful of questions: Can reviewers halt the actions before they take effect? Can they change the output? Are their overrides recorded and enforced downstream? “If the answer to any of those is no, the human is just monitoring AI,” Kimura says.

Too many decisions

Another problem with human-in-the-loop systems is the decision fatigue that can set in when employees are asked to review too many AI decisions and end up button mashing instead of thinking about the consequences.

The AI reviewer needs the expertise and context to evaluate the recommendation, enough time to do so, and both the authority and technical ability to reject or reverse it, says Eric Billingsley, COO and CTO of AI assurance company TrustScale.

But even a qualified and empowered reviewer may gradually stop exercising independent judgment when the AI is consistently right, he notes.

“If the system is right 95% of the time, the person’s job becomes waiting for the rare case when it is wrong,” he says. “Humans are not particularly good at sustained vigilance of a highly reliable automated system. Eventually, review becomes confirmation.”

A good AI system can create bad human controls, he adds. “When the exceptional case arrives, the reviewer may approve it because the system has trained them, through hundreds of correct recommendations, to trust it,” he says.

Billingsley advises IT leaders to evaluate human-in-the-loop systems the same way they monitor other security controls. A control must be monitored, tested, and produce evidence that it is operating as intended, he says.

“A log showing that someone clicked ‘approve’ is not enough,” Billingsley adds. “You need evidence that the person had the necessary context, applied independent judgment, and had the authority to override the AI.”

Robert Blumofe, EVP and CTO at cloud computing and security vendor Akamai, sees the same problems Billingsley does. Some type of human oversight is preferable to fully autonomous AI, he says, but human in the loop can turn into a mind-numbing exercise.

“LLMs produce the correct output just often enough to lull us into a complacent belief that they are more reliable than they really are,” he notes. “After diligently checking the AI output each time and finding no errors, diligence wanes, and human in the loop turns into rote approval.”

IT leaders should take the time to figure out what they’re getting into when vendors or their internal teams pitch a human-in-the-loop system, Blumofe says.

“It’s incredibly important to understand exactly how the system is designed and when and how the human will interact with the AI,” he adds.

Organizations should also explore ways to deploy other technologies as guardrails for AI, instead of turning to unreliable human oversight, Blumofe suggests.

“You need non-AI systems in the guardrail role,” he explains. “These technology tools would help to automate testing and validation of AI outputs, flag issues, and have the capability to pause the AI work. This keeps humans out of approval loops, while also helping to reduce risk.”

When humans aren’t the right choice

Other IT leaders suggest that human-in-the-loop systems aren’t the right solution in every AI use case. When AI is used to flag and mitigate cybersecurity incidents, for example, waiting for a human to approve an action may be too late.

“If an endpoint is compromised, you may want the system to isolate it immediately,” says AISquared’s Kimura. “Waiting 20 or 30 minutes for someone to approve that action could allow the attack to spread.”

The objective is not to put a human into every AI decision, he adds. “It is to put the right human, with the right context and authority, at the right point in the workflow.”

CIOs are increasingly expected to drive IT adoption in their organizations, with change management becoming a huge — and more challenging — imperative in the age of AI.

Evangelism of the latest technologies has long been part of the job, but many CIOs now say resistance to AI adoption and the fast-paced evolution of IT tools have raised the stakes.

Change fatigue has become a major challenge as Andrea Ballinger, CIO of Rensselaer Polytechnic Institute, tries to update the IT systems and provide a tech-driven ultra-personalized student experience at the university, she says.

“It’s not even inside of our institutions or our private companies, but the world is throwing so much at us,” she adds. “What you heard today, you’re being told something else tomorrow.”

For CIOs, change management means recognizing that some employees are on a slower journey and, at the same time, encouraging staff to embrace progress, Ballinger says. Good leaders will recognize that some employees will resist, but it’s their responsibility to help employees navigate the changes, she adds.

“Change management is understanding where people are at,” she says. “It’s having that sense of urgency, but a sense of urgency does not mean running without a parachute or without a plan. It means you act today.”

Change management was a big topic of conversation at the CIO 100 Awards and Conference in Frisco, Texas, in mid-August. Several speakers mentioned the challenge, with Ravi Malick, global CIO at cloud-based content sharing service Box, saying change management now represents about 80% of the job, far outpacing pure IT issues.

The change management aspects of a major digital transformation are often what makes or breaks the effort, he says.

AI in particular has forced CIOs to pay more attention to change management because it fundamentally changes the way employees work, he adds. Some past technologies, like the internet and mobile computing, largely started in the consumer space, then leaked over into the enterprise, giving employees time to get comfortable, he notes.

“AI is something that’s reshaping both the consumer space and the enterprise at the same time,” Malick says. “Both the enterprise and individual people are trying to figure out how to get the most value out of it.”

Some revolution, some evolution

As a company, Box is moving forward quickly on some AI initiatives while taking a wait-and-see approach on others, in part to manage the changes required, notes Malick, who sees adoption of AI and other new technologies as a major challenge.

“There are parts of this that are revolutionary, and there are parts that need to be evolutionary,” he explains. “The best way to get somebody pointed in a different direction is to make them realize they haven’t done an 180-degree turn. Get them to realize, ‘I turned on my own, and I actually like the direction that I’m pointed in.’”

To encourage adoption, Box has pitched AI to employees as an enabler and amplifier, not as a technology that will replace their jobs, Malick says.

“We’re asking, What are the things that we can do now that we weren’t able to do before?” he says. “How can we apply your years of the experience and intellectual power toward other areas that we just couldn’t get to before?”

Box isn’t closely tracking how employees are using the time saved through AI tools, he adds. If employees are using the extra time to improve their quality of life, that’s ok, he says.

“Maybe they’re not working on the weekends at the end of the month closing the books,” he says. “Maybe they actually have weekends now and can spend more time with their families.”

Change across the organization

Other CIOs say the change management piece of the job has increased significantly in the past two to three years.

In recent years, CIOs have been pulled into change management roles within other parts of the business as teams identify AI opportunities, says Orla Daly, CIO at skills management company Skillsoft.

“As AI blurs the lines between technology, operations, and people strategy, the CIO role is becoming closer to that of a COO,” she adds. “Workforce strategy is folding in alongside technology strategy, so leading change now sits at the center of the role rather than being one piece of it.”

The rapidly changing technology landscape has also thrust change management to the forefront of the CIO role, she says. “The pace at which decisions need to be made has increased so dramatically that you can’t lead at a distance and expect strategy to translate cleanly into action,” Daly says.

Daly also notes that slow adopters aren’t always active resisters. Skillsoft’s 2026 Workforce Readiness Report found that while 86% of employees use AI tools at work only 24% feel fully equipped to use them effectively, and just 16% receive training before a new tool is introduced.

“That gap suggests an over rotation on tooling without understanding how it changes how work is executed,” she says. “In most cases, it’s uncertainty and a lack of confidence to take the first step, not a lack of interest.”

Daly and other CIOs suggest that mandating the use of a new tool is rarely the right approach.

“Requiring it can create activity, but activity isn’t the same as adoption,” she explains. “If you hand people tools without clear use cases, guardrails, and training, a mandate just accelerates inconsistent use, and you mistake activity for progress.”

NTT DATA focuses on employee AI fluency instead of mandated activity, and the CIO has a huge role to play, says Barry Shurkey, CIO at the company. The CIO role increasingly sits at the intersection of technology, business strategy, and people, he says.

“AI success is not just about moving quickly; it is about helping people understand the change, embrace it, and move forward with confidence,” he adds.

NTT DATA’s own research suggests that AI front-runners use AI to amplify the impact of experienced, highly skilled employees rather than to replace them, Shurkey says.

“As AI accelerates transformation, CIOs are doing more than implementing technology,” he adds. “They are redefining how people work, make decisions, create value, and just as importantly, managing the intensified resistance that’s driven by fear of job loss or control.”

CIOs have talked about enabling the business for years, but IT exec Monica Caldas expects the role will soon be about orchestrating how the business performs.

“Today, CIOs are helping organizations navigate technological, operational, and cultural transformation simultaneously,” says Caldas, global CIO for Liberty Mutual Insurance and a 2026 inductee into CIO.com’s CIO Hall of Fame. “By 2029, much of that foundation will be in place. The role will increasingly focus on orchestrating an intelligence-enabled enterprise, where AI is embedded into workflows, decision-making, and business operations. As intelligent systems take on more routine work, CIOs will spend more time shaping business strategy, workforce evolution, and new sources of competitive advantage.”

In the upcoming years, Caldas predicts, “the role becomes less about implementing technology and more about helping organizations reimagine how humans and intelligent systems work together to create value in the Intelligence Era.”

She adds, “We’re entering a period where AI is reshaping how decisions are made, how work gets done, and how organizations operate. Just as previous waves of technology changed how enterprises functioned, AI is creating new opportunities for CIOs to act as strategic business leaders and enterprise shapers — not simply technology operators.”

Anthony Moisant, CIO and CSO for Indeed, has a similar vision for the role’s future.

“The CIO is becoming the architect of the company’s operating system itself. The CIO is becoming the architect of how a business runs,” says Moisant, also a 2026 Hall of Fame inductee.

Kathy Kay, executive vice president and CIO for Principal Financial Group, describes the future of the CIO role in much the same way.

“Already I’m having to be even closer to the business and talking about how the business should run. I now have way more of those conversations than conversations about technology,” she says.

Longtime IT leaders are unlikely to be surprised by all this. Anyone who has been watching the profession for the past decade or so has seen the CIO role evolve to one focused more on business strategy than it had been. And those with 20-plus years in the profession have watched it truly transform, from a senior-level operations manager position focused on technical decisions to the influential C-suite executive role it is today.

Now, as organizations devise their three-year strategic plans, those same leaders expect more changes for the role, saying that CIOs in 2029 will not just enable how organizations do business, they will devise what they offer, what they do, and how they produce value.

“They will be business-value creators,” says Craig Stephenson, senior client partner and CIO/CTO practice leader at Korn Ferry, an executive search and organizational consulting firm. “CIOs will own not just tech transformation but business transformation, and they will be enterprise leaders driving that transformation at scale.”

‘This is a game changer’

Dani Brown, who retired July 31 after nearly six years as SVP and CIO of Whirlpool, sees that future for the CIO role, too.

“The CIO of the future is different,” says Brown, also a 2026 Hall of Fame inductee. “This is not just an incremental change; it’s a shift. This is a game changer.”

AI is driving much, if not all, of the shift in the CIO’s position, Brown says, because, more than any other technology in the past, AI is changing what business can offer and how they deliver those offerings.

“AI is reshaping business models and quite frankly entire industries,” she says. “So today, it’s not just about how you enable solutions to problems but how do you use AI to shift a business model or industry.”

That task of engineering a shift of the organization or the industry itself is a monumentally different task than reengineering a process and, as such, speaks to the shift, that “game changer,” that Brown predicts happening in the CIO role.

“CIOs will have to determine how they leverage technology to revolutionize how they engage with consumers, how they transform marketing and differentiate products and the company, how they deliver services, and how they use AI to change internal operations to deliver better margins for the company. With the implications of technology on business today being like it has never been before, CEOs want a business partner beside them who understands that,” Brown says.

She adds that many CIOs are already doing such work.

But Brown doesn’t expect these new CIO responsibilities will displace the responsibilities that traditionally fell under the CIO’s remit. They’ll still have to be technologists to understand how best to implement technologies for business advantage. CIOs will still be accountable for deploying and maintaining the IT environment. And, as is the case already, they’ll be measured on creating and running an IT department that enables the business, can respond to changing business needs, and can do so efficiently, effectively, and securely.

AI shifts how the CIO sits in the C-suite

Kay stresses that CIOs, too, must adopt AI for their IT operations to ensure success in the future.

That, though, speaks to other ways the CIO role is changing.

As CIOs advise their colleagues on the use of AI and reengineer their organization’s services, products, and workflows, they’re also going to have to help the organization adjust to working side by side with autonomous AI agents, Kay says.

CIOs will also have to leverage their understanding of AI as technologists to share how AI reshapes the market, she says.

Kay says she’s already doing that. For example, she has brought AI’s implications on medicine to the attention of her colleagues, explaining how AI is expected to bring better medicines to market, which will likely mean longer life expectancy that in turn could impact her company’s products and services.

“As a CIO, I’m now asking, ‘If this happens, does our business strategy still hold?’ We haven’t seen the CIO play this role in the past. Now we’re the ones to say, ‘We need to pay attention to this,’” says Kay, another 2026 CIO Hall of Fame inductee.

That requires someone who has the courage to challenge existing strategies and colleagues on their stances, she says. And it requires someone who is “OK pushing them to have those conversations.”

For some, the future is already here

As Moisant sees it, leading-edge CIOs are already living that future.

“More and more today it’s the expectation for CIOs to be thinking about the total system, how the organization runs end to end, and becoming more of an architect of that total system,” he says. “That has already become an expectation for some in the field.”

And it’s going to become more common in the upcoming years, he adds, with the majority of CIOs having to meet those expectations in the future.

Caldas’ vision is similar. She sees the CIO’s responsibilities centering on three areas as AI becomes embedded in how work gets done.

To start, CIOs will have to ensure “the organization has the right foundations, including trusted data, resilient platforms, cybersecurity, governance, and responsible AI practices to operate at scale. Those fundamentals won’t go away; if anything, they become more important,” she says.

They’ll have to help “the organization rethink how intelligent systems, people, and business processes work together. The opportunity is no longer just automation; it’s unlocking human potential and enabling employees to focus their energy on higher-value work while intelligent systems take on more routine tasks.”

And third, they will help shape business strategy and competitive advantage. “As technology becomes increasingly inseparable from the business, CIOs will play a larger role in identifying new opportunities, accelerating decision-making, and helping their organizations continuously adapt and reinvent how work gets done.”

The skills necessary to succeed

If all that sounds exceptionally challenging, that’s because it is, says David Ulicne, executive director of executive education at Carnegie Mellon University’s Heinz College of Information Systems and Public Policy.

“It’s a tough job to be a CIO, especially now that we are in the agentic era. The expectations are overwhelming,” he says. To meet the demands of the role now and in the future, “CIOs have to in some ways reinvent themselves again.”

Technical, strategic thinking, leadership, influence, financial management, and communication skills all need to be top-notch in CIOs if they want to succeed, he explains, as do people management and change management skills to help employees adjust to a workplace that will be staffed with both agents and humans.

Caldas likewise says future CIOs will need a different mix of skills, some familiar and others new to the position.

“I believe the most successful CIOs will combine technical fluency with business leadership and human-centered change management,” she says.

She lists as key skills:

• Continual curiosity: CIOs will need to be continuous learners, with the ability to experiment, learn, and iterate quickly.
• Value-informed decision-making: CIOs will need to be able to distinguish between opportunities that create meaningful business value and those that simply create noise — and in many cases, do so quickly. “Knowing when to double down, when to pivot, and when to stop investing will become a critical skill.”
• Business vision and fluency: CIOs will need to readily translate technology into business value — an ability already in demand today.
• Human and organizational leadership: “As intelligent systems become more embedded in everyday work, the differentiator will be the ability to help people adapt, develop new skills, and work effectively alongside new technologies,” she says. “Success will depend as much on leadership, culture, and organizational change as it does on technology itself.”

With all that taken to be the CIO’s evolving remit, Caldas says she already finds herself acting as part technologist, part economist, and part communicator.

“As we move from the Digital Era into the Intelligence Era, the role is becoming less about technology itself and more about helping organizations understand what technological change means for strategy, investments, talent, operations, and competitive advantage,” Caldas says. “Creating clarity in moments of change becomes just as important as delivering technology itself.”

When Sastry Durvasula joined TIAA in early 2022, he saw an organization fighting against outdated legacy technologies and in need of a major IT refresh.

Since then, the financial services organization has completed two phases of a comprehensive transformation initiative called Technology Ecosystem Transformation, or TETRIS, leading to a huge reduction in tech debt and a major expansion of functionality for customers.

The ongoing project, anchored in cloud and AI technologies, started in 2023 with phase one that modernized the core technology stack with 10 new enterprise platforms. Phase two, launched in late 2024, went further by enabling 87 use cases across all major lines of the business.

The project, for example, allowed TIAA to launch its MyChoice Multi-Year Guaranteed Annuity product, and helped create the TIAA Gateway portal, an API-based suite that integrates with partners in retirement and wealth planning using industry standards.

TIAA Gateway took home a CIO 100 Award in 2025, and phase two received a CIO 100 Award in 2026.

Durvasula, TIAA’s chief operating officer, pitched the multimillion-dollar TETRIS project to the board as a three-pronged strategy, with empowering business growth, fueling innovation, and transforming the IT core as its key goals.

Not only did TETRIS need to modernize the company’s IT systems, decommission legacy processes, and automate other processes, but Durvasula pitched it as the way to expand the reach of TIAA’s products and move the company into the future.

“As you expect in a company of our size, we have problems of yesterday, today, and tomorrow being solved at the same time,” he says.

Focus on business use cases

As TETRIS moved into phase two, project leaders shifted their goals from pure technology modernization to business outcome-driven prioritization. So once phase one delivered needed IT platforms like a data cloud and design studio, TIAA pivoted toward enabling business use cases.

This business-first approach ensured continuing executive support and clear ROI at every key milestone, TIAA says.

In 2022, just before the project launched, more than 80% of TIAA’s IT workloads resided in fragmented, end-of-life platforms, which created operational risk, compromised security and resilience, and constrained its ability to innovate. Through TETRIS phase two, however, the organization has cut that tech debt nearly in half.

And consolidating 17 design systems also led to digital products looking and behaving differently, depending on the team that designed them, and accelerated product launches by 35%, enabled multi-lingual capabilities, and increased accessibility to more than 185,000 customers who don’t speak English.

In addition, TETRIS allowed TIAA to combine multiple middleware systems and data lakes, Durvasula says, and the organization moved mainframe applications and data center infrastructure to the cloud.

A giant leap forward

TETRIS has been a huge project, with the company saying it empowered TIAA to have one of the largest leapfrog moments in company history in its submission for the 2026 CIO 100 Award.

Despite the reported failure rates of large transformation projects — some estimates suggest up to 95% fail to meet their goals — TETRIS was essential to keep TIAA competitive and move it forward in the market, Durvasula says.

A big part of the project has been workflow modernization, he says, because TIAA were using some technologies and workflows that were decades old.

“There’s your classical platform and application rationalization, and then there’s your end-of-support, end-of-life stuff that should’ve been remediated long ago,” he says. “Some of the processes we have, because we’re such a large, old company, were designed when the internet just came along.”

Stick to the metrics

Two keys to pulling off such a large project are establishing metrics for success and transparency with leadership, Durvasula says. Project leaders set milestones to indicate when things went well, and they planned for bumps in the road so the TIAA board knew when setbacks happened.

“Not everything is as pretty as it sounds in an awards application, but the success measures we established with our board were based on both phases,” he says. “For the first one, we said we’d deliver enterprise-grade platforms and accomplish migration objectives, but not tied to any specific business objectives.”

Phase two metrics focused more on business objectives, and the project team kept the TIAA board updated as TETRIS moved forward. Setting realistic goals was important, he says, with the team determined not to overpromise results.

“Large programs have a range of objectives, and if you publish the outcomes you’re looking for, people start looking for them, especially stakeholders, the C-suite, and board,” he says. “You have to be honest about which metrics or KPIs you can deliver in the first and second year, and when you’ll start seeing real business scale and impact, which definitely won’t be that soon in a large program like this.”

Goals also need to be flexible, Durvasula says, so transparency with leadership sometimes means telling them the project needs to reset. “If something doesn’t go well, what’s the level of fungibility you have?” he says. “We pick this tool, but what if it doesn’t work? You need to have a plan B.”

So TIAA’s IT team is heavily focused on flexible systems, and what was contemporary three years ago is probably legacy now, especially thanks to AI.

The power of change management

Another big lesson from a project of this size is the need to focus on change management. Retiring old IT systems requires the organization to bring employees along on the journey and convince them the changes are for the better.

TIAA established a multi-disciplinary team to implement a change management program focusing on breaking down silos and setting common adoption goals across the organization and its lines of business. Stakeholder forms and a huge focus on continuous collaboration helped employees understand the need for the changes.

“It’s a big organizational change,” Durvasula says. “If you’re working on a legacy system, and you think at some point it’s going to be modernized, then you become a legacy talent, and won’t have a job.” But the right change management program can convince these employees they can upskill and bring value to the new systems.

“You can bring your functional knowledge of the business and learn new technical skills,” he says. “It’s a massive culture- and people-change initiative as much as tech initiative.”

TIAA’s change management efforts were also made easier because TETRIS happened at the same time as the recent AI boom and involved AI elements. So it wasn’t hard to convince employees they needed to improve their AI skills.

“Because of AI, everybody woke up to this new reality,” he says. “We rode that wave when transformation drove from a cultural and organizational change management point.”

With vision comes perspective, and over a distinguished career, IT and digital transformation leader Niraj Bhatt has held may titles, and earned three consecutive CIO 100 awards since 2023.

As a storied advisor for startups and Fortune 500 companies, helping them navigate the unpredictability and fluidity of AI, Bhatt knows how emerging tech is rapidly reshaping the way organizations build products and deliver value, and how challenges shift as companies move from experimentation to real-world deployment.

AI, of course means a lot of different things to different people, and also for frictionless startups and large enterprises. For the former, speed is a huge asset, allowing them to punch above their weight. But it also means they need lightning fast reactions when landscapes shift. “The same speed can also hurt them when larger AI companies release new offerings that disrupt what startups are building,” he says, referencing recent moves by Anthropic and Google.

On the enterprise side, the conversation is more about scale and risk. Many large organizations have moved past the POC stage and now wrestle with the realities of putting AI into production.

Cost for both is naturally a recurring theme as organizations scale up AI efforts, and true expenses become clear only after the initial excitement fades. “Every input and output token, and the model you’re selecting, add up,” he says. Some customers like Open AI, he adds, get throttled because their usage, volumes, and costs are growing so fast, making planning, observability, and monitoring critical for any team moving beyond experimentation.

So understanding the full software development lifecycle is also vital. Therefore, before committing to production, he helps clients see the big picture, and make sure they understand technical requirements as well as operational and financial implications. “The cost picture isn’t just about usage, but scale and the model choices teams make,” he says.

Bhatt also discusses effective approaches to AI and enterprise IT, technology leadership, and the evolving role of today’s CIOs. Watch the full video below for more insights, and be sure to subscribe to the monthly Center Stage newsletter by clicking here.

On AI hype: If you can’t explain something to someone who’s eight or 80, you don’t really understand it. It’s gone from LLMs, to RAG, to agentic AI, and now the essence is all about tokens. It’s predicting that next token and understanding that is key. So when LLMs came out, they were good at doing that on the data on which they were trained. When the enterprises looked at it, they wanted to make those LLMs work for their data. And the question became how to provide our data and context. It’s about building the right context for the LLM. Agentic AI is similar and that’s where the RAG evolution came in, in that I’ve got my data because every LLM has limitations in terms of how much context it can carry.

There are ranges of LLMs, where Google has the highest in regard to the context window size and what they support. Agentic AI is more action oriented, though. LLMs rely on the metadata you provide for the tools. Then they’re doing token prediction in that whatever I’m looking for, I should use a specific tool. Then it’s the infrastructure underlying which LLM it relies on to invoke the agent. So if you try to explain the microservices to a person, you’re going to struggle. But it’s very important to understand the evolution and that’s where you can cut through the hype. Understanding in this context is key.

On navigating challenges around talent: What I’m seeing on the IT side is there’s so much cognitive load, so how do we empower people to build solutions with the right mix of products and platforms? I think it’s about democratizing AI for the entire organization. Your talent strategy is everyone, all inclusive, starting from interns, the business and tech sides, CEO, everybody.Like your customer success or revenue officers, you need a talent strategy because in the end, IT alone isn’t going to be in a position to deliver for everyone in the organization.

AI has the potential to make everyone in the organization more productive. You have to plan that and facilitate broad innovation across the organization.That’s where the talent strategy, and working with HR and the people officer becomes very important providing those tools. One part of it is training, but how do I build an agent for a receptionist receiving calls, for instance?I’m not going to rely on vibe coding or things of that nature. But what are the tools? Where do I go, where do I host this? I think through that entire ecosystem beyond copilots. That’s where innovation can kick in, and that broader talent strategy is something I’m working with my customers on.

On collaboration: I heard a panel discussion recently, and a question was asked about what’s the number-one trait CIO needs to be successful at in the world of AI, and the answer was collaboration. You need to bring everybody together, move forward together, and make sure everybody’s on board. And in my mind, simplifying that is more like systems thinking when you operate, just bringing everybody along and ensuring they’re meeting outcomes.

But maybe what’s more important is managing expectations. Because if you’re a CIO, there’s a tremendous amount of pressure to deliver and have a rock solid AI strategy. So what I’m doing with my customers is get the board, CEO, and CFO into a room and help them understand what I’m talking about, the evolution, and what’s the art of possible. You don’t want to be a CIO who thinks I have a hammer and everything is a nail. Having buy in from the senior leaders is essential to know you’re headed in the right direction. You’re not reacting to pressure from top leadership, but driving and becoming the change agent for good for the company.

On navigating AI: It’s interesting times. I’m covering a spectrum of startups, non-technical and technical founders, and advising Fortune 500 companies. What I’m seeing is they love the velocity and momentum because that’s what they’ve always wanted, and AI is providing that. They’re able to bring their products to markets very quickly, so something that would’ve taken three years a couple of years ago is probably now taking them three months. There’s a lot of excitement there. But on the flip side, the same velocity is also hurting them. There are so many frontier AI companies getting disrupted. OpenAI, for instance, has offerings in sales and marketing, and Google has an interactive video model. So a lot of startups working in the marketing space are getting stuck. A lot of what I’m focused on is working with founders, helping them pivot in the gen AI space, ensuring their systems and products are built and structured in the right manner.

And on the enterprise space, what I’m seeing is the POC wave, and people have seen the value. There’s some excitement but now the struggle is getting them to production. That’s where you run into cost, latency, legal compliance, privacy issues, and customer concerns that if we get tickets to production, how’s it going to look and how are we going to scale. So engineering and product teams have to be ably supported by the enterprise architecture and R&D teams. I then help them get up to speed and build that internal platform product for the production workloads. It’s exciting times on both sides.

Each year the CIO 100 Awards showcase outstanding IT initiatives, and every year they illustrate the power and potential of technology to transform how people work, how organizations perform, and the value they offer to customers.

The 2026 cohort of winners is no different. Each one demonstrates how IT executives and their teams successfully move from ideation to deployment to scaling a solution for the future, overcoming challenges and driving adoption along the way to ensure their organization gets a return on its investment.

[ Interested in meeting and learning from all CIO 100 winners? Join us next week at CIO 100 Awards & Conference in Frisco, TX. Limited seats remain! Register here ]

The winning initiatives come from a range of industries and utilize a host of technologies to achieve their goals, as is the case annually. A growing proportion of these stand-out projects leverage artificial intelligence, raising the bar on the art of the possible for all IT departments.

The following 10 award-winning projects serve as representatives for the outstanding work done by all the 2026 honorees.

ABB democratizes AI agent creation and deployment

Organization: ABB

Project: ABBY — AI Agentic Platform for Workforce Transformation

IT leader: Vikke Kandell, CIO

IT leaders at ABB, a manufacturer, had some big hurdles to clear when it came to building an AI strategy.

They had to overcome employee fears that AI would take away jobs, the potentially high cost of AI vendor licenses, and pressure from investors, customers, and executives to advance the use of AI in the enterprise.

“We looked at this and asked, ‘How do we address all this?’ and build something that the company is proud of,” says Babu Kuttala, vice president of data analytics and AI.

The answer is ABBY, an AI agentic platform that enables employees to create and deploy specialized AI agents for specific business tasks.

To build ABBY, Kuttala and his team used best-of-breed LLMs (about 25 in total). They built a centralized orchestration layer using generative AI that integrates internal knowledge bases with external ecosystems, creating a unified platform where agents can access enterprise data, understand required actions, and execute tasks across multiple systems. And they created preconfigured skills so that employees could build agents tailored to their workflows without having to code.

ABBY was rolled out in 2025 to 100 users but is now used by 63,000 (more than 75% of the company’s workforce, Kuttala notes) with an average of 10,000-plus workers using it daily. IT continues to add LLMs and capabilities to expand use of ABBY even further, Kuttala says.

Belcorp modernizes manufacturing with Smart Factory

Organization: Belcorp

Project: QPlant — Smart Factory

IT leader: Venkat Gopalan, Chief Digital, Data, and Technology Officer

Legacy processes were limiting Belcorp’s ability to scale and compete. Its manufacturing relied on ERP-driven processes with limited shop-floor automation and weak connectivity across production, packaging, quality, and maintenance. The company depended heavily on manual records and post-process reconciliation, resulting in fragmented data, limited real-time insight, inefficiencies, and higher risks for errors.

Smart Factory changed all that. The IT initiative reimagined how manufacturing teams work “by creating a connected, data-driven environment where production, quality, maintenance, and operations are aligned around real-time information and standardized execution,” says Venkat Gopalan, chief digital, data, and technology officer.

At Smart Factory’s core is a manufacturing execution system that orchestrates production workflows, quality processes, and operational execution, he explains. IoT-enabled equipment integration and a centralized SCADA platform provide real-time visibility into shop-floor operations, while electronic batch records digitize production execution, strengthen traceability, and reinforce compliance by design.

Integrating those operational technologies with the company’s enterprise platforms was another critical component of success, Gopalan says, creating a trusted flow of real-time data across manufacturing, quality, maintenance, and business systems. “This connected architecture transformed isolated data into actionable insights, enabling faster decision-making, greater operational visibility, and continuous improvement across the manufacturing lifecycle,” he adds.

The initiative generated more than $1 million in financial benefits in its first year alone.

“Most importantly, Smart Factory established the digital foundation for the future of manufacturing at Belcorp,” Gopalan says. “With real-time operational data and connected systems now in place, we’re well positioned to accelerate advanced analytics, AI-driven optimization, predictive maintenance, and other Industry 4.0 capabilities that will continue delivering value for years to come.”

Cohesity replatforms post-acquisition for commercial growth

Organization: Cohesity

Project: Lead to Cash Replatforming Program (Veritas Integration)

IT leader: Brian Spanswick, CIO

Cohesity set an ambitious objective: Complete an enterprise-scale lead-to-cash replatform in under six months.

That’s a tight timeline for any replatforming initiative, but Cohesity’s project had another layer of complexity. It followed Cohesity’s December 2024 acquisition of Veritas, a company twice its size in revenue, leaving Cohesity to integrate the majority of a global enterprise revenue engine into its own operating model without disrupting customers, partners, or sellers.

“We had to bring the two companies together, merge the workforces together, and create an overall harmonized organization and operating infrastructure platform,” says Eric Brown, who as CFO and COO led the project.

The program migrated heavily customized CRM, CPQ, PRM, ERP, and subscription platforms (some of which were “very brittle, very bespoke,” Brown says) to a unified SaaS CRM, CPQ, and ERP environment with uninterrupted selling, billing, and partner operations.

This was no lift-and shift, Brown stresses. “It was a business process optimization project as well. We want to run very efficiently, so we questioned everything and used the migration process to simplify and streamline the business in every possible respect.”

The initiative enabled continuity for 13,000-plus customers, protected revenue during integration, and established a scalable commercial foundation for future growth.

Brown cites several factors that contributed to success. First, leadership was upfront about what it would take to meet the deadline, a process that involved carefully prioritizing the capabilities that would appear in the first iteration. Leadership also streamlined decision-making, establishing office hours that “ran with military precision” to handle issues. And the company selected a specialized partner, requiring its top talent be assigned to Cohesity.

Dairyland Power goes agentic to protect field crews

Organization: Dairyland Power Cooperative

Project: ODIN — Organizational Effectiveness Agentic AI

IT leader: Nate Melby, VP and CIO

Dairyland Power Cooperative had amassed a large collection of field observations, incident reports, near-misses, safety rules, and work methods that could yield insights into processes and practices that could help protect its workers.

But the insights were essentially out of reach, trapped in siloes.

Dairyland’s organizational effectiveness team turned to CIO Nate Melby for help unlocking those insights. Melby then turned to agentic AI, recognizing that the technology could address the team’s need to make better use of its data.

“This was about finding insights on how to work more safely,” Melby says. “It’s about preventing incidents.”

The collaboration between the two teams created ODIN, the first agentic AI implementation of its kind in the electric utility industry.

Focused on worker safety, ODIN autonomously connects the collective safety knowledge of the organization and delivers actionable insights directly to field crews at the moment work is planned.

ODIN was developed through a hybrid approach that combined an agentic AI platform and Dairyland’s internal private generative AI platform called VoltWrite. ODIN leverages LLMs, retrieval-augmented generation, and a coordinated swarm of autonomous agents.

Agents work together to analyze internal safety data, performance history, work practices, and safety rules and then synthesize the information into clear guidance on the safest way to perform specific tasks.

ODIN has produced results, including a reduction in OSHA recordable injuries and improvements in the quality and consistency of pre-job safety briefings.

ODIN was deployed in early 2025 for use by Dairyland’s workers in transmission construction and electrical maintenance, which are the highest-risk work areas. Dairyland is looking to expand ODIN’s use to other teams.

Dow’s digital sustainability ledger drives low-carbon sales

Organization: Dow

Project: Carbon Footprint Ledger

IT leader: Deb Bauler, Chief Information and Digital Officer

Executives at Dow consider the Carbon Footprint Ledger (CFL) as more than a technology or innovative carbon accounting methodology. According to Senior Global IT Director Jeremy Preston, CFL is “a digital business capability that enables Dow to translate sustainability investments into customer value.”

CFL transformed how Dow uses greenhouse gas emissions data. It combines a methodology aligned to international standards with an enterprise-scale digital platform. It also integrates manufacturing, supply chain, commercial, and sustainability data to generate product carbon footprints under enterprise-level governance and management at scale.

In doing so, Preston says it creates “a trusted, traceable link between low-carbon processes and raw materials implemented across its manufacturing network and the lower-carbon products customers seek.”

The technology team worked closely with sustainability and business teams, collaboratively developing the capabilities needed to reconstruct product genealogy, maintain end-to-end data lineage, track low-carbon attributes across interconnected manufacturing processes, and generate product carbon footprints that can support customer offerings and commercial transactions.

CFL was built on Dow’s Integrated Data Hub and in partnership with Boston Consulting Group and Databricks.

The core CFL platform is fully deployed and supports commercial transactions today.

Preston says CFL “enables Dow to turn sustainability investments into customer value, commercial differentiation, and new growth opportunities.” Dow reports that it has driven hundreds of millions of dollars in low-carbon product sales in 2025 and 2026.

The company is now expanding its use. “We are extending adoption across additional products, manufacturing networks, business segments, and customer use cases while continuing to enhance automation, analytics, and integration with commercial processes,” Preston says.

J&J transforms quality management with AI

Organization: Johnson & Johnson

Project: Q&C Strategy

IT leader: Michael Comprelli, Vice President, Head of Technology, Technical Operations, and Risk; Joel O’Connor, Head of Technology, Medtech Quality, and Compliance

Johnson & Johnson is using AI to transform quality management through its Q&C Strategy.

QuIn is an AI-powered digital assistant that fuses human expertise with machine learning, automation, and data-driven insights to boost efficiency, reliability, and worker impact. By embedding gen AI into core quality management systems processes, QuIn proactively gathers actionable insights, increases operational efficiency, and allows teams to focus on high-value, patient-centric work.

Cora is an innovative generative AI platform that provides regulatory intelligence monitoring, impact analysis, and augmented content revision. Cora assists with document analysis, compliance comparison, stakeholder analysis, policy/standard creation, procedural/document updates, and document comparison. Cora is purpose-built for regulated environments, validating outputs against source material and offering a user experience that instills trust in the outcome.

QuIn and Cora, which automate time-intensive tasks and democratize information access, are on track to deliver significant value, with J&J reporting more than $62 million in documented true cost savings by 2028 from QuIn alone. Cora delivered $2 million in cost efficiency in 2025 and will deliver a documented cost savings of $25 million by 2028.

“Our teams proved responsible AI can be applied meaningfully in a highly regulated environment without compromising the rigor, accountability, or human judgment that quality requires,” says Michael Comprelli, vice president, head of technology, technical operations, and risk.

He continues, saying that J&J “moved these ideas beyond experimentation and into products that employees use in their daily work. We did that by bringing together Quality expertise, product management, data engineering, architecture, cybersecurity, user-experience design and AI engineering around a common purpose.”

JLL brings intelligent automation to business services

Organization: JLL

Project: Business Service Digitization

IT leader: Pinak Dash, Global Head of JLL Business Services and Legal Technologies

JLL launched its digitization initiative to drive process redesign as well as systematic AI and RPA deployment across JLL Business Services (JBS).

The initiative was designed to address inefficiencies that hampered scalability and competitive positioning. It was also designed to eliminate manual processes that consumed thousands of hours across finance, HR, legal, procurement, marketing, research, IT, and lease administration.

Pinak Dash, global head of JBS and legal technologies, says the digitization initiative had a dual-strategy combining traditional digitization with generative AI innovation to hundreds of processes.

JLL lists three innovations critical to the program’s success.

First is a hybrid platform that integrates RPA with JLL’s proprietary AI platform called Falcon, which created intelligent automation that adapts and learns. It enables real-time process automation, intelligent document processing with automated extraction/validation, and smart decision-making for continuously optimizing workflows.

The second innovation is its use of ProHance for real-time process monitoring and enabling of data-driven optimization. Sensors capture granular productivity metrics, identify bottlenecks, and provide actionable insights for continuous improvement across automated and manual processes.

Third is its custom AI assistants and transaction agents. Falcon-powered assistants provide intelligent knowledge search while specialized agents execute complex transactions across enterprise SaaS platforms. These handle multisystem workflows, reducing human touchpoints while maintaining accuracy and compliance.

Dash says the initiative has delivered quantifiable benefits through improved efficiency, accuracy, and quality of services provided to clients.

“The initiative delivers on our business goals, makes us more efficient, provides customers better service, and it opens up the capabilities and bandwidth of our people to do what they like to do and to find innovative ways to serve our business,” he adds.

Nationwide partnership platform delivers efficiencies, business growth

Organization: Nationwide

Project: Enterprise Digital Platform (EDP)

IT leader: Michael Carrel, EVP and CTO

Nationwide’s new Enterprise Digital Platform (EDP) gives the company “a scalable way to connect with external partners quickly, securely, and consistently across all areas of our business,” says company EVP and CTO Michael Carrel.

He explains that “instead of treating every integration as a custom effort, EDP creates a common front door for digital products, documentation, onboarding and governance.”

That innovation has produced better experiences for the company’s partners. It saves time for Nationwide teams, partners, and customers. And it supports faster launch times for new products and enables growth across the business.

“EDP changed the model from fragmented, point-to-point integrations into an enterprise platform built around reusable digital products. That shift lets us support a range of integration options in one governed environment, meet partners at different stages of technical maturity, and add new capabilities over time without redesigning every relationship from scratch,” Carrel explains.

EDP uses cloud-native microservices, role-based access control, and advanced analytics. Nationwide IT created modular microservices to make EDP more scalable, resilient, and adaptable. And IT decoupled it from infrastructure-specific dependencies so that it would be a platform-agnostic developer portal. That, Carrel says, reduced operational constraints across environments.

Additionally, IT shifted from a user-specific model to role-based access, which improved security, simplified administration, and better served the needs of different audiences.

Meanwhile, robust analytics delivers visibility into platform usage and performance, which Carrel says helps ensure Nationwide continuously evolves the platform based on measurable outcomes.

The core platform is fully deployed, with Nationwide planning to expand it.

“Our Enterprise Digital Platform is more than a piece of technology,” Carrel notes, “it represents a strategic enabler to support growth objectives across Nationwide’s businesses.”

PITT Ohio fast-tracks shipment requests with AI assist

Organization: PITT Ohio

Project: No Touch Email (N@TE AI)

IT leader: Scott Sullivan, President and CEO (formerly CIO)

As PITT Ohio started its AI journey in 2024, the mandate was clear: Use the technology to solve “real problems,” says Ryan Carner, director of enterprise IT solutions.

“We wanted to hit the ground running and find a problem that was solvable,” Carner says, noting that the company also wanted to use the experience to build in-house AI skills. “The idea was to find a business case for AI that would be our first but not the only one.”

PITT Ohio leaders decided to tackle what Carner describes as a “mundane but very important task for how our business operates”: handling emails to the customer service team.

The need was significant. Customer service representatives were manually processing hundreds of pickup request emails daily, each requiring five to 15 minutes to interpret and re-enter shipment details into the company’s transportation management system (TMS). The emails were complicated, containing a lot of information submitted in nonstandardized ways and varying formats. This repetitive task consumed valuable time, introduced errors, and delayed customer response.

N@TE uses generative AI and natural language processing to transform unstructured email content into structured pickup orders automatically and in real-time. N@TE scans incoming emails, extracts key shipment data, and creates orders directly in the TMS via API integration. It operates seamlessly within existing workflows, requiring no change in customer behavior or retraining of staff.

PITT Ohio deployed N@TE in 2025, and the company also secured a patent for the product that year. N@TE has produced a 30-60X increase in processing speed, 99% accuracy in extracting and populating order data, and a 70% reduction in handling costs per pickup order.

SMU builds AI adoption through grassroots ambassador program

Organization: Southern Methodist University

Project: Scaling AI Without Scaling AI: Organizational AI Scaling Through Willingness

IT leader: Jason Warner, Associate CIO

Like executives in most organizations, leaders at Southern Methodist University encountered mixed attitudes about AI. Some workers had little interest in using the tech, others were afraid it would take jobs, still others were curious about what it could do.

Associate CIO Jason Warner and other leaders decided to leverage that last group, believing the best way to get SMU faculty and staff to embrace AI was to use enthusiasts to help smooth the way.

So, instead of treating AI as a conventional technology rollout, Warner and his colleagues built opt-in communities of practice known as the AI Coalition of the Willing and Operation Copilot.

The goal, Warner says, was to build institutional capability, reduce risk, and generate momentum.

“We knew the fastest way to scale AI was to scale the willingness of people to use the technology, and not talking to people about cost savings and the like,” Warner says, adding that willing users as great ambassadors and evangelists who showcase in formal and informal ways the technology’s potential for hesitant or skeptical colleagues.

Participating faculty members have access to a licensed ChatGPT account as long as they use it. Staff members have access to Copilot accounts after taking a self-paced training course and likewise must use it to keep that access.

Warner says these willing workers are demonstrating the benefits of AI (significant time reclamation, reduced cognitive load, improved quality of outputs, expanded professional capacity).

SMU is now moving to a single solution and scaling AI, confident that its use will deliver returns following in the footsteps of the early adopters.

Interested in meeting and learning from all CIO 100 winners? Join us next week at CIO 100 Awards & Conference in Frisco, TX. Limited seats remain! Register here 

One of the biggest challenges companies like Aetna face every year is an annual HEDIS review of its records to identify gaps in care. For large national payors, the scale of the challenge is immense. So Aetna has deployed a gen AI-driven document intelligence platform that has reduced the need for manual review by 65%.

“We have a large group of amazing trained medical coders who do this every day,” says Nathan Frank, chief digital and technology officer at Aetna. “This is about making it easier for them by speeding up the process. Something that might have taken weeks or months we can now do in days.”

The Healthcare Effectiveness Data and Information Set (HEDIS) is a range of performance measures for the managed care industry. Developed and maintained by the nonprofit National Committee for Quality Assurance (NCQA), the first version of HEDIS was released in 1991.

Under the HEDIS measures, large managed care providers like Aetna review more than 10 million medical records annually to identify gaps in care. These gaps are missed or overdue preventative care or chronic disease management tests including missed cancer screenings, blood sugar tests for diabetics, eye exams, and immunizations. Closing these gaps improves patient outcomes, and health plans are measured in how well they perform. But processing medical records is no easy task.

“We’re talking about medical charts that have white space filled with handwritten notes,” Frank explains. It’s not just structured data, it’s lots of physical clinical documentation.”

Adding up the numbers

Frank says industry benchmarks for large providers indicate an annual review process that requires about 50,000 work weeks, equivalent to nearly 1,000 dedicated full-time employees. It would take a team of 50 reviewers more than 20 years to complete a single annual review using fully manual processes.

Enter AI Medical Chart Review, a platform developed by Aetna that leverages cloud services and gen AI to automatically extract clinically relevant data from records, and prioritize records based on the likelihood of measure closure and evidence strength.

“Large language models and gen AI give us the ability to train a model to decipher the charts, identify the high value codes, and build correlations,” Frank says.

In the space of about six months, Frank’s team ideated the platform, and designed and trained a PoC that was able to process millions of records in just two weeks. As a result, AI Medical Chart Review has earned Aetna a CIO 100 Award in IT innovation.

“Now we’ve gone through 14 million documents,” Franks says. “We’re seeing a reduction of manual effort, which is now being transitioned into other areas like quality control and making sure the automated chart review is working as expected.”

Behind the curtain

Using gen AI, the platform automatically ingests and analyzes unstructured medical records and clinical documents. And as part of that process, it identifies and extracts clinically relevant information for specific HEDIS measures like diagnosis codes, medication records, lab results, and visit documentation. With this data, the platform generates a prioritized set of records based on the likelihood of measure closure and strength of clinical evidence, which is then passed to human employees for review and validation.

Frank says the platform has increased gap closure rates (leading to improved Star Ratings and higher reimbursement), streamlined workflows, and enabled teams once dedicated to manual record review to shift focus to higher-value activities.

Frank says much of the speed and success in building the platform comes down to a shift in the way it approached the design and build process. Rather than exhaustively writing specifications and requirements, Aetna created a team that included engineers and subject matter experts who worked together to build out capabilities iteratively.

“It allowed us to move much faster, and having a business subject matter expert sitting in the same virtual or physical room with us got us a much better outcome,” Frank says. “The product model, our cloud compute model, and our AI governance model allow for quick reviews to make sure we’re using AI responsibly with the right guardrails. It’s increased the speed to get from product launch to go-live.”

He adds that small teams that don’t have to deal with a lot of bureaucracy are key to moving quickly.

“You need to design with security, compliance, and a responsible use of AI as core principles from day one,” he says. “Everything we do from a new build standpoint starts with thinking about how we make it cloud native, how we build with the right elasticity and speed, and how we optimize the cost.”

The most important element of all, he says, is a good relationship with your subject matter experts.

“You can have a great product manager and engineer, but you really need that business subject matter expert who’s excited about it, and who has a passion for transforming the process,” Frank says. “Once you put those three together, you’ll see amazing things like this happen all the time.”