The CIO Professional Network held its second CION Vendor Showcase, continuing a series designed to give members a closer look at enterprise innovations without the traditional sales pitch. The session featured Cogniware and SuperPenguin, exploring how technology leaders can keep AI spending aligned with the value it delivers.
Srinath Godavarthi, Chief AI Officer at Cogniware, examined what organizations need to account for when putting AI into production, while SuperPenguin co-founders Chris Acker and Yuta Baba discussed how to trace AI costs to the work they support and help teams manage consumption.
From their presentations, attendees examined how greater cost visibility could affect engineering autonomy and whether spending metrics could explain the quality of the finished product.
Why It Matters: AI adoption creates a need to understand what the organization is paying for and what it receives in return. A pilot can demonstrate potential while leaving the cost of production unaccounted for. Usage-based billing also makes decisions by employees a factor in how much the organization spends. The discussion emphasized evaluating costs alongside results so leaders can determine which investments deserve additional funding. Giving teams access to that information allows them to address waste during the work and understand the financial consequences of their decisions.
- Include the Full Cost of Production: Srinath described an initiative at a financial services organization where 2 of 120 AI pilots reached production after 8 months. The applications were intended for internal use. He used the example to question how thoroughly organizations account for the cost of deploying AI. Infrastructure spending captures only part of that investment. Preparing employees to work differently can add expense, and governance issues can prevent deployment. Srinath encouraged leaders to present the board with a total cost of ownership that accounts for these requirements and to address governance during development.
- Evaluate Costs for Each Workflow: Estimating the cost of an AI request starts with understanding what the system must do to fulfill it. An agent coordinating activity across several systems can consume more resources than a request to summarize a document. An attendee questioned the cost multipliers presented for agentic AI, noting that spending varies with how a workflow is designed. Srinath agreed that cost estimates need to account for how each application operates and the resources it requires, emphasizing that helps leaders evaluate model selection and token efficiency in financial terms.
- Tie Productivity Gains to Business Results: Increased usage indicates that employees are engaging with the tools, while return on investment requires evidence of what that activity produces. Srinath offered a hypothetical example in which developers save time generating code, then spend more time correcting defects. In a contact center, hours saved also need to translate into work that benefits the organization. He encouraged leaders to use those outcomes when reviewing investments, asking whether CIOs could tell the CFO which AI projects deserved twice the investment and which should be stopped.
- Trace Spending to the Work It Supports: SuperPenguin focused on the gap between knowing the size of the AI bill and understanding what the organization received for that spending. Chris explained how metadata can connect AI requests to the work they support, allowing teams to examine the cost of serving a customer or developing a feature. In response to a question about shadow AI handling, Chris described how comparing tracked usage with billing records could identify discrepancies that indicate untracked activity. He clarified that the attribution tool can track AI requests made outside coding workflows, provided those interactions are connected and tagged.
- Set Budgets for Projects: Chris described a disconnect between finance teams reviewing AI bills and engineers making the decisions that generate them. To help teams manage costs before a month-end review, he recommended setting project budgets and giving engineers a way to track spending against those expectations. He explained that SuperPenguin’s desktop application supports this approach by showing engineers where their AI spending is going and sharing that information with leadership. With that visibility, teams can identify waste and adjust consumption before reaching a budget limit, reducing the need for cutoffs that interrupt productive work.
- Preserve Autonomy Through Cost Visibility: Attendees questioned whether spending dashboards would help engineers understand the business impact of their decisions or make them anxious about being monitored. Chris acknowledged that introducing this level of accountability requires a cultural adjustment. He argued that clear budgets and access to spending information give engineers room to make their own choices about how they work. Chris shared an example in which agents at his company repeatedly checked whether code had finished deploying. He said those checks accounted for approximately half of the company’s AI costs. Identifying waste of that kind gives teams an opportunity to reduce spending while retaining useful tools.
- Evaluate Product Quality Alongside Cost: One member asked whether the metrics could establish the quality of the finished product. In response, Yuta described SuperPenguin’s interest in tracking how long generated code lasts before developing bugs or requiring a rewrite, though this capability is currently unavailable. Chris added that organizations need evaluations tailored to their products, which they can repeat to understand how changes affect the customer experience. He explained that SuperPenguin focuses on measuring costs, while assessing the value of the output requires measures specific to each business. Connecting those evaluations with spending data would help organizations determine whether an investment is delivering results that justify the expense.
Go Deeper – AI Cost Optimization and Governance (VIDEO) – CION Vendor Showcase
