Three remarkable things that happened in AI this week
AI finds a better way of taking corner kicks in soccer
What happened: In an article released in Nature, a team from Google DeepMind announced TacticAI, a model that analyzes corner kicks in soccer. By reviewing thousands of corner kicks from the Premier League, the team built a machine-readable representation of each situation, and then trained a model to identify variations that might improve the likelihood of success for each team, attacking and defending. The resulting suggested adjustments were preferred by experts (who didn’t know which recommendations were human or AI) more than 90% of the time, suggesting real-world value.
Why it matters: First, I mean, sports. Second, soccer tactics are a deep and rich problem, with a set of multiple independent agents (the players) cooperating and competing in complex ways in physical space, and where small differences in location and performance can lead to dramatically divergent outcomes. It’s much harder to solve than previous multi-agent problems such as keeping autonomous vehicles from crashing, so if Google has advanced the state of the art here, it opens up the possibility of simulating and managing more complex real-world multi-agent problems. Third, it’s nice to see a meaningful advance in AI that isn’t based on an LLM.
Converting existing LLMs into 1-bit models
What happened: The smart people over at Mobius Labs have found a way to make 1- and 2-bit versions of existing LLMs more effective. A 1-bit model is one where each parameter weight is only 0 or 1, instead of the 32- or 64-bit weights common in most LLMs today, and a 2-bit model obviously has four possible weights. If you can reduce the weights to 1 or 2 bits, the model becomes much smaller and much easier to run (think about how much simpler the multiplication at each step becomes), enabling models to be run cheaply and locally.
The problem has been getting 1- and 2-bit models to be effective, especially smaller models like Llama2–7B. The Mobius team has found a way around this, reducing pre-existing models to 1- or 2-bits (“quantization”), and then fine-tuning a subset of the parameters. It doesn’t seem like this should work, but it does, and the resulting quantized models perform well–in fact, the 2-bit version of Llama2–7B actually outperformed the full model on some tasks!
Why it matters: First, one of the biggest problems with LLMs today is how costly they are to run, given all the compute required to respond to each prompt. If we can dramatically reduce that cost through quantization, then a whole new range of use cases may become economically viable, allowing the use of large models for tasks that aren’t directly swapping out even more expensive human labor. Second, the astonishing performance of fine-tuned quantized models may provide more insight into how these models actually work and perhaps even point the way towards a more efficient way of building and training them. A 2-bit model outperforming a 32-bit model runs contrary to our common sense, and when combined with the growing body of work on sparsification, may shift the course of future research and development.
A voice model that claims to have emotional intelligence
What happened: Hume, a company focused on emotional intelligence in AI, has released an “empathic” voice chatbot. Hume says that the chatbot can understand vocal inflections to detect and measure emotions, building on the company’s research into the other ways humans show emotion, such as facial expressions. Does it work? I mean, it said I was primarily feeling doubt, anxiety and confusion, so it’s off to a good start.
Why it matters: Call-center software that purports to measure caller emotion has been around for years, but it’s mostly been focused on extremes of caller frustration or anger, to know when to escalate to a supervisor. If products like Hume’s can deliver a more nuanced, accurate version of this, there will be direct benefits for AI applications, such as knowing when a customer needs a deeper explanation, or testing how people feel about their experiences with a product or with advertising. But the bigger benefit will be helping us build AI systems that feel more natural and human, reducing the sense of unease many of us get with AI systems that can’t pick up on how we’re responding to them the way that another human could. This kind of work will be an important long-term element in integrating AI into society in a way that people accept.
