With the dawn of the digital age, our ability to gather and process data has increased exponentially, permeating practically every aspect of life. This evolution has amplified the growing significance and relevance of econometrics, a field dedicated to utilizing statistical methods and economic data. Among its many applications, econometrics allows us to quantify (economic) phenomena and make informed forecasts. Recently, with the rise of artificial intelligence (AI) and machine learning (ML), algorithms are emerging that can create more accurate models and forecasts than traditional econometric approaches [1]. This begs the question of whether econometrics and econometricians will retain their relevance in the future.

Before delving further into this discussion, it is crucial to clarify the definitions of econometrics, AI and ML, and to understand the distinctions between them.

• Econometrics combines economics, statistics and mathematics to analyze and test economic theories and models. It uses observed data to model (economic) relations, enabling econometrists to quantify these relationships and make forecasts [2].
• Artificial intelligence refers to the simulation of human intelligence processes by computer systems. These processes include, amongst others, learning, reasoning, problem-solving and language understanding [2]. AI can be categorized into two types: narrow AI, which is designed to perform a specific task, and general AI, which can perform any intellectual task akin to a human.
• Machine learning is a subset of AI which automates the analytical model-building process. It empowers computers to discover hidden patterns and insights without explicitly programming the approach to data [2].

Since machine learning — a subset of AI dedicated to constructing models that reflect the underlying reality for future predictions — holds significant implications for the field of econometrics, it will be our primary focus in this discourse.

Methodological differences

Econometrics and machine learning follow different methodological paradigms. Econometric models are typically anchored in economic theory, with model specifications derived from hypothesized economic relationships. Conversely, machine learning tends to be more data-centric. It empowers algorithms to learn directly from the data, forming models without any preconceived notions about underlying relationships [2].

The complex inner workings of many machine learning models — often referred to as a ‘black box’ due to their hidden nature — may seem contradictory to the transparent theoretical approach in econometrics. However, this difference can also be seen as an advantage. Machine learning excels at discovering intricate patterns and non-linear relationships in large datasets, which can be challenging for traditional econometric models that often rely on linear and predefined structures [2].

Prediction vs explanation

One of the primary distinctions between machine learning and econometrics lies in their objectives. Machine learning predominantly focuses on predictive accuracy. Its algorithms aim to minimize the difference between predicted and actual outcomes, often sacrificing interpretability in the process [3]. Machine learning models excel in optimizing predictions, often without requiring a deep understanding of the underlying data relationships. For example, they can accurately predict events like customer churn in phone subscriptions.

In contrast, econometrics is primarily concerned with understanding and interpreting relationships between variables. Econometric models aim to provide insights into the structure of the (economic) phenomena being studied, allowing econometrists to make causal inferences and test hypotheses [3]. Econometric methods are preferred in investigating causal impact, e.g. education on labour market outcomes. These methods address potential endogeneity issues and provide policymakers with valuable insights for designing effective education and labour policies, while prioritizing interpretability and rigorous inference.

Complementary Tools

Despite their differences, machine learning and econometrics are not mutually exclusive; in fact, they often complement each other. Machine learning techniques can aid in feature selection for econometric models, identifying the most impactful variables to include. Conversely, econometric techniques can provide a robust theoretical foundation for machine learning models, significantly enhancing their interpretability and overall robustness [3].

Bridging the Gap

Researchers are increasingly focusing on harnessing the predictive power of machine learning within econometrics, without compromising the interpretability and theoretical grounding characteristic of the latter. Techniques such as LASSO (Least Absolute Shrinkage and Selection Operator) and ridge regression, originally birthed in the machine learning realm, are now being utilized in econometrics due to their adeptness at managing multicollinearity and selecting features.

Additionally, the advent of ‘explainable AI’ or ‘XAI’ represents a promising stride towards bridging this divide. XAI endeavours to enhance the interpretability of the decision-making processes within machine learning models. Consequently, it may pave the way for a harmonious blend of machine learning’s predictive prowess and the explanatory power that is central to econometrics.

Conclusion

In conclusion, the era of artificial intelligence and machine learning has undoubtedly transformed our approach to (economic) data analysis. The power of these emerging technologies is undeniable, with their predictive accuracy and ability to handle large datasets surpassing traditional econometric models. However, the role of econometrics remains vital in our quest for understanding and quantifying (economic) phenomena. Its emphasis on theory, interpretability, and causal inference renders it indispensable in making informed policy decisions.

As we continue to navigate through the digital age, the dichotomy between machine learning and econometrics is becoming less pronounced. The two are increasingly seen as complementary tools, each contributing unique strengths to the field of economic analysis. The integration of machine learning techniques into econometrics and the ongoing development of explainable AI reflects a promising trend towards harnessing the best of both worlds. This fusion provides an avenue for enhancing predictive capabilities while maintaining a solid theoretical grounding and interpretability.

Far from making econometrics irrelevant, the rise of AI and machine learning offers an opportunity to augment econometric analysis and bring new insights to light. It is a testament to the evolving nature of the field and to the adaptability of econometricians. In this era of AI, the relevance of econometrics is not diminished; it is redefined and expanded, ready to continue its essential role in understanding the complex world of economics.

[1] Zavadskaya, A. (2017). Artificial intelligence in finance: Forecasting stock market returns using artificial neural networks.

[2] Charpentier, A., Flachaire, E., & Ly, A. (2018). Econometrics and machine learning. Economie et Statistique, 505(1), 147-169.

[3] Hastie, T., Tibshirani, R., Friedman, J. H., & Friedman, J. H. (2009). The elements of statistical learning: data mining, inference, and prediction (Vol. 2, pp. 1-758). New York: springer.