Robert Kugel's Analyst Perspectives

As we celebrate the first half of what seems to be the year of generative artificial intelligence, with an apparently unlimited discussion of use cases and bogeymen, my attention is turning to the very mundane question of costs. Specifically, how costs incurred – through investment and operation – will be distributed along the value chain and how this will affect the demand for AI ‒ by whom and for what purpose. It’s a question that needs asking even though, at this stage in the market’s evolution, I believe it is a classic underdetermined linear algebra problem, the kind with fewer equations than unknowns and an infinite number of solutions.

Perhaps one reason so few are asking the question is that we’ve grown accustomed to these sorts of costs being generally socialized and more than made up for by gains in productivity or other sources of value. Since this may not be true for generative AI in all cases, it’s worth raising the question.

The issue of generative AI costs is important because the technology will become ubiquitous in business computing. Ventana Research asserts that by 2027, almost all vendors of business applications will use generative AI to enhance capabilities and functionality to remain competitive. Until now, almost all the attention paid to generative AI has been on dreaming up (and sometimes) delivering use cases, creating and understanding caveats about the technology’s shortcomings and allowing self-interested individuals and organizations to induce mass-media freak-outs about generative AI robots taking over the world. All of this is amusing, but there’s been little regard for the cost of developing and applying the technology.

That’s probably justified because so many future scenarios are highly speculative. There will likely be generative AI-driven capabilities that prove to be uneconomic or simply nice to have relative to their cost. For software vendors, it will be necessary to craft, review and revise generative AI product-service road maps with an eye to estimating costs and how that might affect the services offered and pricing.

Costs are important because they potentially have a meaningful impact on the competitive dynamics of business software markets. In the case of generative AI applied to business computing, larger established software companies may have an inherent advantage over smaller organizations and new entrants owing to a large and possibly more diverse user base on which to train systems and spread fixed costs. Larger organizations may be able to innovate faster because of superior resources, or may be tripped up by internal management issues that slow down development cycles. There are likely to be technologies that rapidly become commodities, allowing smaller vendors and upstarts to apply them either for niche, best-in-class applications or as less expensive, easy-to-use software with sufficient but less capable functionality compared to what larger, established vendors offer. The permutations of outcomes, especially in the context of dynamic timescales, make it impossible for me to produce definitive scenarios with any useful level of confidence.

It’s also possible to take the opposing view – that in general, costs will not be important. Perhaps the cost of generative AI will be within the normal bounds of IT-spend or development budgets. Another reason is that model training costs are likely to decline sharply over the next five or so years. For example, training an image classifier (e.g., ResNet-50) on a public cloud initially cost about $1,000, but two years later declined to $10. According to ARK Invest, an asset manager focused on innovative technology, generative AI training for a GPT-3 level model cost $4.6 million in 2020 and dropped tenfold to $450,000 by the end of 2022. For what it's worth, ARK forecasts it to drop to $30 by 2030.

There is also the issue of who will bear the developmental and operational costs of generative AI. There are, for example, costs incurred at the infrastructure, application and end-user levels. The allocation of those costs along a business system will vary: either fully absorbed (reducing the profitability of the entity incurring the cost), shared (some absorbed and some passed along in pricing) or fully passed along with or without a markup. Competitive pressures will affect where costs are assessed: An entity will fully absorb costs to gain share, for example. It’s possible that, in some cases, development costs will be fully absorbed, using venture capital to create new technology or new applications with AI technology. Costs borne by venture investors may be recouped in many ways (such as a public offering or sale to a strategic buyer). The costs may be incurred by hyperscalers with fat margins that find it necessary to subsidize services to remain competitive or encourage adoption. These sorts of technology infrastructure entities may create generative AI-driven utilities licensed by others, as has been the case with voice recognition, which over decades, became a licensed commodity.

Although some degree of skepticism around the specific what and when of generative AI is certainly in order, the technology will have a profound impact on business computing through the rest of the decade. I strongly recommend that organizations adopt a fast-follower mindset to incorporate generative AI in operations, especially in finance and accounting. Business software vendors must make generative AI road maps transparent, reliable and up-to-date. To be credible, these plans should recognize the impact of costs and their effect on timetables and feasibility.

Regards,

Robert Kugel