The Psychology of Data Science

[An article published by The Group of Analysts in April 2018 here]

In marketing, research, or product development, data science is a fixed component in various aspects of our economy. While a couple of years ago, the creation of insights by using data led to a competitive advantage, it has become a requirement to stay competitive, and most companies apply it. However, only a few make use of the powerful combination of psychology and data science.

Psychology is a scientific discipline dealing with the human mind, its cognition and its behaviour by combining scientific methods from natural science as well as humanities. It makes use of qualitative and quantitative research methods. In the case of data science, psychology plays a major role considered from multiple perspectives.

The data scientist is always subject to psychological phenomena such as cognitive biases. One of the most important ones is the confirmation bias: The human tendency to confirm, rather than disconfirm pre-existing hypotheses. However, not only the data scientist is affected. There is also the potential pool of clients or customers who are affected through personalisation and psychology-based segmentation. This article aims to give an idea of how psychological concepts can easily be used in combination with data science. This is highlighted by an example from the field of data-based marketing. Data science is a modern applied scientific discipline with the goal of generating actionable insights from various data sources. By making use of different technologies, it is possible to analyse data from multiple types and sources. There are multiple definitions of data science and there is no universal understanding of the required skillset of a data scientist. Often, however, the profession is understood as a combination of relevant business skills, statistics and an ability to use state-of-the-art technology to turn data into actionable insights.

Since data science is always practiced by humans, psychology always implicitly plays a role. One of the most important aspects of psychology in data science comes into play when results are interpreted: The confirmation bias. Therefore, the bias can lead to severe mistakes in the interpretation of results. It can drive the analyst’s attempts to make a statistical model to fit their hypothesis in order to find what they want, instead of accepting to find possible unexpected results which may lead to new insights.

On that account, the topic of cognitive biases is an extensive one while being an undesirable psychological aspect in data science. Still, we can gain psychological insights that can be used for the benefit of a business.

Each type of behavioural data reflects on human habits, attitudes or personality. Behaviour is interpreted as the reflection of latent constructs, such as personality traits. [...]

[Full text as pdf in English // in German]