This is an excerpt of Sources by Alex Heath, a newsletter about AI and the tech industry, syndicated just for The Verge subscribers once a week.
Amazon’s AI chief has a message for the model benchmark obsessives: Stop looking at the leaderboards.
“I want real-world utility. None of these benchmarks are real,” Rohit Prasad, Amazon’s SVP of AGI, told me ahead of today’s announcements at AWS re:Invent in Las Vegas. “The only way to do real benchmarking is if everyone conforms to the same training data and the evals are completely held out. That’s not what’s happening. The evals are frankly getting noisy, and they’re not showing the real power …
Read the full story at The Verge.
Original Source: https://www.theverge.com/column/836902/amazons-ai-benchmarks-dont-matter
Original Source: https://www.theverge.com/column/836902/amazons-ai-benchmarks-dont-matter
Disclaimer: This article is a reblogged/syndicated piece from a third-party news source. Content is provided for informational purposes only. For the most up-to-date and complete information, please visit the original source. Digital Ground Media does not claim ownership of third-party content and is not responsible for its accuracy or completeness.
