As generative AI rolls out across firms of all sizes, CIOs are refining vendor approaches, balancing delivery models with cost and strategic goals. 

Growing enterprise adoption of generative AI is adding pressure on CIOs to act quickly. By early 2024, some 87% of companies surveyed by Bain & Company said they were developing, piloting or had deployed generative AI in some capacity. 

Companies are also rushing to bring generative AI under the protective umbrella of their internal IT organizations as employees experiment with external, off-the-shelf tools.

CIOs need to convince C-Suite teams that AI will yield a tangible return on investment, while boards debate how much AI deployment should be led internally, CIO Professional Network participants said.

“They’ve done things the same way for a long time, and they’re having challenges trying to figure out how AI will deliver value for the costs,” said Stephen Greco, a consulting CIO for a media analytics company.

In-House Capacity Versus Outsourcing

Small and medium-sized firms are taking varying approaches to AI implementation oversight. Some companies employ internal subject-matter experts to scope out requirements and vendor plans, while others rely on external vendors to build AI implementation strategies.

“One camp is maintaining that on-site subject matter expertise, more or less like the traffic cop… and then you’re seeing this other model develop where ‘I’m just going to outsource it,’” said Rusty Everhart, Chief Information and Technology Officer at consulting and professional services firm ReedRook.

Viable Use Cases

CIOs need to avoid turning to AI as a solution for any problem and focus on viable use cases, said Dave Hatz, Vice President of Machine Intelligence at CTI. He warned against “throwing AI” at any problem. 

“AI is a tool, but a lot of times we’re really just talking about process automation much more than we are actual AI,” he said. 

CIOs ought to be mindful of enterprise software providers’ rush to incorporate AI into all offerings, and consider which uses will deliver a tangible return, suggested Greco, who noted that agentic AI may be one of the promising use cases.

“Agentic AI has some autonomous power to actually do things and make some business decisions or take care of things,” he said. “Agentic AI can probably be deployed internally and might yield some competitive advantage…especially with robotic process automation and decision making.”

Companies need research use cases that suit their business needs and avoid being swayed by software providers’ aggressive marketing of add-on AI capabilities. 

“Enterprise software providers have their marketing techniques to push [AI offerings] aggressively,” said Kiran Palla, senior advisor at the Internal Revenue Service. “Most use cases we are solving for are more for operational efficiency — these kinds of quick wins, or low-hanging fruit.”

Protecting Corporate Data

Companies need to ensure that their data won’t be leaked to an external partner. This means they need to work closely with service providers and understand how corporate data will interact with the toolset of an external service provider, suggested a CIO and consultant in the sports industry.

“There’s tons of firms out there that may not even be using an enterprise-grade model,” like OpenAI, Anthropic, or other large AI service providers, he said. “There’s some due diligence that absolutely needs to be done.”

Uncertain Pricing Models

Participants compared the evolution of pricing for generative AI to other types of enterprise software, where prices rose as adoption grew. A ‘race to zero’ pricing model may draw companies into vendor ecosystems, but it can’t continue over time, especially given the compute power required to run AI systems.

“AI costs a lot of money to offer, so that race to zero that happened before [with enterprise software] can’t happen now,” said Greco.

Small language models — AI models trained on a restrictive set of data and focused on specific tasks — may help companies save compute power and ultimately save money in some cases.

“You can build the small language model into your PC, your mobile device. You get the opportunity to offer lower cost solutions that are alternatives to pure cloud connected, pure large language models,” said Hatz.

Small language models may also be appropriate for agentic AI use cases, said Greco.

“It’s a much easier business case to make [than large language models],” owing to lower costs, as well as security and compliance benefits, he said.

“Large language models can be like using a battleship to go across the Hudson [River], when a speedboat would do.“