MLCommons, a group dedicated to benchmarking AI hardware, released new tests that gauge how quickly AI systems can handle user requests. These benchmarks focus on response speed for tasks like generating creative text formats or answering complex questions.

Imagine asking a large language model, like ChatGPT, a question. One of the new benchmarks, called Llama 2, simulates this scenario. This benchmark, developed by Meta, measures how fast the AI can process the question and deliver a response. Llama 2 uses a massive model with 70 billion parameters, pushing the boundaries of AI speed and complexity.

Another new benchmark focuses on image generation. This benchmark uses a model based on Stable Diffusion XL from Stability AI. Servers powered by Nvidia's powerful H100 chips dominated the raw performance tests for both benchmarks. However, the race for efficiency is also heating up. Server builder Krai showcased a design using a Qualcomm AI chip that excels in image generation while consuming significantly less power than Nvidia's top-of-the-line offering. While Nvidia and established players like Intel are still in the game, these new benchmarks highlight the growing importance of power efficiency alongside raw performance for real-world AI applications.