AMD and Cloudera look beyond hardware in right-sizing infrastructure

Right-sizing infrastructure for businesses embarking on AI deployments is a critical consideration, according to Jeremy Girven, AWS technical enablement manager at Advanced Micro Devices Inc. Performance metrics alone cannot adequately convey the implications of an investment for customers, Girven emphasized. The acceptable response time for users plays a significant role in determining the appropriate hardware for a particular use case.

For instance, if waiting 30 seconds for a response is acceptable, opting for cheaper hardware might be a more cost-effective solution. Conversely, if immediate response is necessary, customers may need to invest in more advanced hardware.

During an exclusive broadcast at EVOLVE26, Girven discussed AMD's partnership with Cloudera, focusing on infrastructure right-sizing and the economics of AI deployment in enterprises. Budgetary concerns are a significant aspect of these discussions, with Girven suggesting that organizations with limited technical expertise should collaborate with partners such as Cloudera or AMD to design tailored solutions meeting their specific requirements.

Currently, infrastructure right-sizing solutions for AI are not standardized, as each customer's needs are unique.

Girven stressed the importance of ongoing monitoring and adjustment of infrastructure utilization and capacity after deployment. Enterprises should not solely concentrate on the technological aspects of AI but also consider ancillary costs, such as network fees, which can significantly impact overall expenses. He highlighted instances where customers were caught off guard by unexpected network egress fees, underscoring the need for comprehensive cost analysis in AI deployments.