Responsible AI practices – Google AI

• Read original article here

Fairness Interpretability Privacy Security
Fairness
AI systems are enabling new experiences and abilities for people around the globe. Beyond recommending books and television shows, AI systems can be used for more critical tasks, such as predicting the presence and severity of a medical condition, matching people to jobs and partners, or identifying if a person is crossing the street. Such computerized assistive or decision-making systems have the potential to be fairer and more inclusive at a broader scale than decision-making processes based on ad hoc rules or human judgments. The risk is that any unfairness in such systems can also have a wide-scale impact. Thus, as the impact of AI increases across sectors and societies, it is critical to work towards systems that are fair and inclusive for all.
This is a hard task. First, ML models learn from existing data collected from the real world, and so an accurate model may learn or even amplify problematic pre-existing biases in the data based on race, gender, religion or other characteristics. For example, a job-matching system might learn to favor male candidates for CEO interviews, or assume female pronouns when translating words like “nurse” or “babysitter” into Spanish, because that matches historical data.
Second, even with the most rigorous and cross-functional training and testing, it is a challenge to ensure that a system will be fair across all situations. For example, a speech recognition system that was trained on US adults may be fair and inclusive in that context. When used by teenagers, however, the system may fail to recognize evolving slang words or phrases. If the system is deployed in the United Kingdom, it may have a harder time with certain regional British accents than others. And even when the system is applied to US adults, we might discover unexpected segments of the population whose speech it handles poorly, for example people speaking with a stutter. Use of the system after launch can reveal unintentional, unfair blind spots that were difficult to predict.
Third, there is no standard definition of fairness, whether decisions are made by humans or machines. Identifying appropriate fairness criteria for a system requires accounting for user experience, cultural, social, historical, political, legal, and ethical considerations, several of which may have tradeoffs. Is it more fair to give loans at the same rate to two different groups, even if they have different rates of payback, or is it more fair to give loans proportional to each group’s payback rates? Is neither of these the most fair approach? At what level of granularity should groups be defined, and how should the boundaries between groups be decided? When is it fair to define a group at all versus better factoring on individual differences? Even for situations that seem simple, people may disagree about what is fair, and it may be unclear what point of view should dictate policy, especially in a global setting.
Addressing fairness and inclusion in AI is an active area of research, from fostering an inclusive workforce that embodies critical and diverse knowledge, to assessing training datasets for potential sources of bias, to training models to remove or correct problematic biases, to evaluating machine learning models for disparities in performance, to continued testing of final systems for unfair outcomes. In fact, ML models can even be used to identify some of the conscious and unconscious human biases and barriers to inclusion that have developed and perpetuated throughout history, bringing about positive change. Far from a solved problem, fairness in AI presents both an opportunity and a challenge. Google is committed to making progress in all of these areas, and to creating tools, datasets, and other resources for the larger community. Our current thinking at Google is outlined below.
Recommended practices
It is important to identify whether or not machine learning can help provide an adequate solution to the specific problem at hand. If it can, just as there is no single “correct” model for all ML tasks, there is no single technique that ensures fairness in every situation. In practice, researchers and developers should consider using a variety of approaches to iterate and improve.
General recommended practices for AI Interpretability Privacy Security
Interpretability
Automated predictions and decision making can improve lives in a number of ways, from recommending music you might like to monitoring a patient’s vital signs. Interpretability is crucial to being able to question, understand, and trust AI systems. Interpretability also reflects our domain knowledge and societal values, provides scientists and engineers with better means of designing, developing, and debugging models, and helps to ensure that AI systems are working as intended.
These issues apply to humans as well as AI systems—after all, it's not always easy for a person to provide a satisfactory explanation of their own decisions. For example, it can be difficult for an oncologist to quantify all the reasons why they think a patient’s cancer may have recurred—they may just say they have an intuition, leading them to order follow-up tests for more definitive results. In contrast, an AI system can list a variety of information that went into its prediction: biomarker levels and corresponding scans from 100 different patients over the past 10 years, but have a hard time communicating how it combined all that data to estimate an 80%% chance of cancer and recommendation to get a PET scan. Understanding complex AI models, such as deep neural networks, can be challenging even for machine learning experts.
Understanding and testing AI systems also offers new challenges compared to traditional software. Traditional software is essentially a series of if-then rules, and interpreting and debugging performance largely consists of chasing a problem down a garden of forking paths. While that can be gnarly, a human can generally track the path taken through the code, and understand a given result.
With AI systems, the “code path” may include millions of parameters and mathematical operations, and it is much harder to pinpoint one specific bug that leads to a faulty decision. However, with good AI system design, those millions of values can be traced back to the training data or to model attention on specific data or features, resulting in discovery of the bug. That contrasts with one of the key problems in traditional decision-making software, which is the existence of “magic numbers”—decision rules or thresholds set without explanation by a now-forgotten programmer, often based on their personal intuition or a tiny set of trial examples.
Overall, an AI system is best understood by the underlying training data and training process, as well as the resulting AI model. While this poses new challenges, the collective effort of the tech community to formulate guidelines, best practices, and tools is steadily improving our ability to understand, control, and debug AI systems. This is an area of intense research and development at Google, and we’d like to share some of our current work and thinking in this area.
Recommended practices
While a complete solution to interpretability and accountability is an active area of research at Google and in the ML community, here we share some of our recommended practices to date.
General recommended practices for AI Fairness Privacy Security
Privacy
ML models learn from training data and make predictions on input data. Sometimes the training data, input data, or both can be quite sensitive. Although there may be enormous benefits to building a model that operates on sensitive data (e.g., a cancer detector trained on a dataset of biopsy images and deployed on individual patient scans), it is essential to consider the potential privacy implications in using sensitive data. This includes not only respecting the legal and regulatory requirements, but also considering social norms and typical individual expectations. For example, what safeguards need to be put in place to ensure the privacy of individuals considering that ML models may remember or reveal aspects of the data they have been exposed to? What steps are needed to ensure users have adequate transparency and control of their data?
Fortunately, the possibility that ML models reveal underlying data can be minimized by appropriately applying various techniques in a precise, principled fashion. Google is constantly developing such techniques to protect privacy in AI systems. This is an ongoing area of research in the ML community with significant room for growth. Below we share the lessons we have learned so far.
Recommended practices
Just as there is no single “correct” model for all ML tasks, there is no single correct approach to ML privacy protection across all scenarios. In practice, researchers and developers must iterate to find an approach that appropriately balances privacy and utility for the task at hand; for this process to succeed, a clear definition of privacy is needed, which can be both intuitive and formally precise .
General recommended practices for AI Fairness Interpretability Security
Security
Safety and security entails ensuring AI systems behave as intended, regardless of how attackers try to interfere. It is essential to consider and address the security of an AI system before it is widely relied upon in safety-critical applications. There are many challenges unique to the security of AI systems. For example, it is hard to predict all scenarios ahead of time, especially when ML is applied to problems that are difficult for humans to solve. It is also hard to build systems that provide both the necessary restrictions for security as well as the necessary flexibility to generate creative solutions or adapt to unusual inputs. As AI technology develops, attackers will surely find new means of attack; and new solutions will need to be developed in tandem. Below are our current recommendations from what we’ve learned so far.
Recommended practices
Security research in ML spans a wide range of threats, including training data poisoning, recovery of sensitive training data, model theft and adversarial examples. Google invests in research related to all of these areas, and some of this work is related to practices in AI and privacy. One focus of security research at Google has been adversarial learning—the use of one neural network to generate adversarial examples that can fool a system, coupled with a second network to try to detect the fraud.
Currently, the best defenses against adversarial examples are not yet reliable enough for use in a production environment. It is an ongoing , extremely active research area. Because there is not yet an effective defense, developers should think about whether their system is likely to come under attack, consider the likely consequences of a successful attack and in most cases should simply not build systems where such attacks are likely to have significant negative impact.