The gut-check questions every leader needs to ask before building with AI

The growing trend in enterprise technology is for companies to use AI to build their own software, rather than purchasing a platform like SAP, Workday, or HubSpot. As someone who has been in the CIO role, I understand the appeal. However, before embarking on such a project, there are several important questions to consider.

Firstly, is building software your company's core competency? While the engineering talent may be available, is it worth diverting resources from your core business? In a global technology company with hundreds of thousands of employees, the focus was shifted to innovation rather than building in-house software for billing and HR.

Secondly, are you truly saving money by building instead of buying? The lure of reduced recurring subscription costs can be significant, but this doesn't account for the costs of development, running, and maintaining the custom software. Companies building their own LLM-based software solutions typically spend five to 10 times more than using an automation platform. Initial development costs are just the beginning, as ongoing maintenance, security, and keeping the software secure and up-to-date can be costly.

Thirdly, have you considered the security and governance implications? Building software comes with its own set of challenges, especially when it comes to cybersecurity. AI agents can easily exceed their permissions, leading to potential data breaches or other issues. It's crucial to evaluate the capabilities of your team to ensure the software remains secure, auditable, and compliant with regulations.

Fourthly, what happens when your CIO leaves? The average tenure of a CIO is around 4.5 years, and building a custom platform can take two or more years to develop and mature. The knowledge and expertise essential to the system's success may be lost when the CIO moves on, leaving the organization with no documentation, support team, or vendor to call upon.

Lastly, can you leverage proprietary data and institutional knowledge to your advantage? Building software with AI can make sense in certain contexts, such as a CDIO at a shipping and logistics company prioritizing projects that differentiate the business and utilize decades of internal data and institutional knowledge. When appropriate, a combination of building with AI and purchasing software from vendors can be the most effective approach, optimizing time and resources.