Scaling AI: Why It’s So Difficult

• Read original article here

SwissCognitive Guest Blogger: Zachary Amos – “How to Improve Customer Perception of AI Chatbots”

Staying aware of potential scaling hurdles can help developers and organizations prepare for any situation. While every AI model is unique, these five scaling challenges tend to be the most common culprits for problems in the industry.

One of the most common scaling challenges developers run into with AI is scaling technical requirements. This often concerns the processing resources and data access the model needs to operate. In training and testing, models function on a smaller scale. In testing, the AI may deliver training-data results in one minute. That might seem pretty reasonable. However, when developers scale up the model to do more processing, the time it requires will also increase.

Sometimes businesses and developers aren’t prepared for a model’s increased technical requirements when they scale it up. They may not have even designed the AI with these needs in mind. Likewise, a business may not have the budget to sustain a model’s scaled-up processing infrastructure. As a result, a scaling attempt can end up crippled by poor performance from a model without the necessary technical resources.

Data cleansing is among the most tedious tasks in developing and operating an AI model. It can become a significant issue when trying to scale. It is a crucial part of successfully training these models, though. Data cleansing involves having human analysts manually inspect, correct and label the data they feed to the AI. This ensures the model is learning from and accurately interpreting the proper information.

However, when a model gets scaled up, it needs to work with greater volumes of data. Manual data cleansing can become highly time-consuming when the AI needs to analyze these larger datasets, but accuracy is still critical for success. Businesses and developers may need other data preparation methods to keep up with more extensive data volumes.

By 2030, researchers expect an estimated 70% of companies to adopt some form of AI. That represents a virtually endless variety of different models and applications. It also means organizations are going to have plenty of competition when it comes to who can implement the best AI. Overcoming errors and reliance on humans will be a major hurdle in the race to scale these new models.

Like any other piece of code, AI models experience errors or behave unusually from time to time. The model might come across some information it isn’t familiar with or a situation no one trained it for. These incidents often occur during the scaling process when businesses throw more at the model and it suddenly experiences many new things. If developers don’t design the model to navigate through unfamiliar situations, it will inevitably end up stumped by something during the scaling process.

Similarly, some organizations get by keeping a human in the loop somewhere. This person might check the model’s conclusions or help it reach them. Whatever the case, a lack of complete autonomy can hinder a model’s ability to perform on a larger scale.

Unfortunately, some AI fails to scale because it falls victim to cyberattacks. Even if the model doesn’t suffer an attack or breach, security personnel might flag security vulnerabilities in the model that prohibit it from going into widespread use. The development team then has to go back and resolve those issues before attempting to scale again. Industry experts recommend focusing on five key priorities when handling security risks in AI models.

AI is more difficult to scale than a non-intelligent software program in this sense simply because it is so complicated. There are many loose ends developers need to keep track of and secure. The matter of the AI black box only makes things more complex since security vulnerabilities can sit in hiding and go undetected for disturbingly long periods.

Organizations often run into trouble with the sheer amount of data a model might need when scaling up. Even if developers know how much data the AI requires to process and access, it can be overwhelming in practice.

Even the type of database or data storage an organization uses can make a difference. Some database architectures lend themselves better to different AI models and large volumes of data. Choosing the wrong one can lead to bottlenecks, poor performance and trouble dealing with the necessary data volume.

Carnegie Mellon University has outlined three ways developers and companies can better scale their AI.

While this is still very new, machine learning operations could help businesses extend their AI models faster. An extension of DevOps, MLOps consists of:

Development workforces need more education and training to develop AI for production, but increased productivity in operations may mean companies could quicken the pace on scaling.

Developers may find a limit to how much they can teach AI, so researchers are finding algorithms and infrastructure to combat this problem. They’re returning to Systolic Arrays and Coarse-Grained Reconfigurable Architectures in hopes these early foundations will improve the data-processing capabilities of AI models.

Naturally, information is vital to AI. However, some industries might find collecting enough to teach their models challenging. Crowdsourcing and transfer learning could help provide solutions. Businesses can ask for valuable data to train their AI and then begin using it for different problems. This educates the model in two ways — on what it needs to know and how it can implement that in multiple scenarios.

Scaling AI is a challenging process for any team. Problem-solving will always be part of the process. However, those who know the frequent challenges to look out for can be better prepared. Developers and organizations should keep these five hurdles in mind to plan for scaling up — starting from the beginning of the development process.

Zachary Amos is the Features Editor at ReHack where he writes about artificial intelligence, cybersecurity and other tech topics.