Running AI on your device is cheaper and more private than relying on the cloud.
Generative AI is slowly worming its way into every facet of technology, and next up is your smartphone. Qualcomm says that its next flagship mobile chips are going to be built specifically to run generative AI on the silicon, reducing reliance on the cloud. Running an AI on your phone could be a boon for privacy, but is generative AI ready for that kind of integration?
The generative AI race kicked off in early 2023 when Microsoft showed off what ChatGPT could do as part of Bing. Google was caught flat-footed by the announcement, rolling out its buggy Bard AI just weeks later. And from there, AI has exploded in popularity as every firm that could justify it has trained new generative models, including Adobe, Grammarly, Canva, and so, so many more.
One thing all these tools have in common is that they live in the cloud, backed by massive server farms bristling with GPUs to crunch data. Qualcomm’s pitch, as described by Cnet, is to power generative AI with the silicon in your hand. It’s not going to be as powerful as the models you’ve seen online. OpenAI’s GPT-3 model that started that AI frenzy operated with 175 billion parameters, but Qualcomm expects its next-gen chips to run just 7 billion. It should also be noted that GPT-3 has since been superseded by GPT-4.
Regardless of the raw power available, Qualcomm believes on-device AI hardware can do things more powerful cloud-based models cannot. Your phone is a repository of personal data, like your location, app usage, web searches, driving patterns, photos, and more. That could give a generative AI more data to customize the experience and generate content. Instead of manually editing a photo, for example, you could just tell the phone how you want it changed. Qualcomm’s plans could also help preserve your privacy. Moving the aforementioned personal data into the cloud for AI to chew on is a privacy risk. Keeping it on your phone for the built-in AI is safer, and you’d still have access to AI features even if you don’t have internet connectivity.
This won’t be the first hint of AI on your phone. Qualcomm and others have been pushing AI processing for several years, though in a more limited fashion. The Hexagon digital signal processor (DSP) in Snapdragon chips has gained some AI capabilities aimed mostly at image processing and analysis. Google also builds AI features into its Tensor chips, which power on-device Pixel features like voice transcription and cinematic background creation.
Qualcomm will have more to say about on-device generative AI in October at its annual Snapdragon Summit. At that time, we’ll get the first official details on the chip that will eventually be released as the Snapdragon 8 Gen 3.
https://www.extremetech.com/mobile/qualcomms-next-smartphone-chip-will-be-built-for-generative-ai
