An AI speed test shows clever coders can still beat tech giants like Google and Intel | 7wData

• Read original article here

There is a common narrative in the world of AI that bigger is better. To train the fastest algorithms, they say, you need the most expansive datasets and the beefiest processors. Just look at Facebook’s announcement last week that it created one of the most accurate object recognition systems in the world using a dataset of 3.5 billion images. (All taken from Instagram, naturally.) This narrative benefits tech giants, helping them attract talent and investment, but a recent AI competition organized by Stanford University shows the conventional wisdom isn’t always true. Fittingly enough for the field of artificial intelligence, it turns out brains can still beat brawn.

The proof comes from the DAWNBench challenge, which was announced by Stanford researchers last November and the winners declared last week. Think of DAWNBench as an athletics meet for AI engineers, with hurdles and long jump replaced by tasks like object recognition and reading comprehension. Teams and individuals from universities, government departments, and industry competed to design the best algorithms, with Stanford’s researchers acting as adjudicators. Each entry had to meet basic accuracy standards (for example, recognizing 93 percent of dogs in a given dataset) and was judged on metrics like how long it took to train an algorithm and how much it cost.

These metrics were chosen to reflect the real-world demands of AI, explain Stanford’s Matei Zaharia and Cody Coleman. “By measuring the cost [...] you can find out if you’re a smaller group if you need Google-level infrastructure to compete,” Zaharia tells The Verge. And by measuring training speed, you know how long it takes to implement an AI solution. In other words, these metrics help us judge whether small teams can take on the tech giants.

The results don’t give a straightforward answer, but they suggest that raw computing power isn’t the be-all and end-all for AI success. Ingenuity in how you design your algorithms counts for at least as much. While big tech companies like Google and Intel had predictably strong showings in a number of tasks, smaller teams (and even individuals) ranked highly by using unusual and little-known techniques.

Take, for example, one of DAWNBench’s object recognition challenges, which required teams to train an algorithm that could identify items in a picture database called CIFAR-10. Databases like this are common in AI, and are used for research and experimentation. CIFAR-10 is a relatively old example, but mirrors the sort of data a real company might expect to deal with. It contains 60,000 small images, just 32 pixels by 32 pixels in size, with each picture falling into one of ten categories such as “dog,” “frog,” “ship,” or “truck.”

In DAWNBench’s league tables, the top three spots for fastest and cheapest algorithms to train were all taken by researchers affiliated with one group: Fast.AI. Fast.AI isn’t a big research lab, but a non-profit group that creates learning resources and is dedicated to making deep learning “accessible to all.” The institute’s co-founder, entrepreneur and data scientist Jeremy Howard, tells The Verge that his students’ victory was down to thinking creatively, and that this shows that anyone can “get world class results using basic resources.”

Howard explains that in order to create an algorithm for solving CIFAR, Fast.AI’s group turned to a relatively unknown technique known as “super convergence.” This wasn’t developed by a well-funded tech company or published in a big journal, but was created and self-published by a single engineer named Leslie Smith working at the Naval Research Laboratory.

Essentially, super convergence works by slowly increasing the flow of data used to train an algorithm. Think of it like this, if you were teaching someone to identify trees, you wouldn’t start by showing them a forest. Instead, you’d introduce information to them slowly, starting by teaching them what individual species and leaves look like.