AI inference

AI is beginning to automate the development of software like Nvidia's CUDA, challenging its dominance in AI infrastructure. Startups and cloud giants are using AI coding agents to create alternatives, while Nvidia claims it also leverages AI to enhance CUDA development. The shift toward AI inference may further impact CUDA's long-standing industry lock-in.

SambaNova secured $1 billion in funding at an $11 billion valuation as investors support its challenge to Nvidia. General Atlantic led the investment round, and JPMorgan Chase became a customer to use SambaNova's chips for on-premises AI inference.

OpenAI is unveiling its first custom-built AI chip designed for AI inference in collaboration with Broadcom. The chip offers cost savings of approximately 50% compared to typical AI GPUs, according to Broadcom's CEO.

Inworld, an AI voice startup, is cutting its AI model prices by over 50% to help consumer AI startups combat rising inference costs. Consumer startups face profitability challenges as usage grows due to high costs, while larger AI companies leverage market power to control expenses.

Cerebras CEO discusses delivering AI inference at scale. The article focuses on advancements in AI technology and organizational strategies for large-scale implementation.

Nvidia unveiled a new 'superchip' designed for AI-driven home computing, enabling local AI model execution and shifting user interaction from traditional interfaces to AI agents. CEO Jensen Huang envisions AI supercomputers in homes, while Intel plans a cheaper AI chip focused on inference rather than training.

Chipmaker Groq is seeking to raise $650 million in internal funding as it shifts focus to AI inference, following Nvidia's $20B not-aqui-hire. The pivot emphasizes refining AI models' responses to prompted requests.

Intel is investing heavily in AI inference and agentic workloads to revive the CPU's relevance in computing, despite ongoing challenges in chip manufacturing. The company aims to leverage AI-driven applications like robots and edge devices to reposition CPUs as central to next-generation compute.