Design and Development of Automatization Pipelines for Machine Learning Projects based on the MLOps Frameworks

Aitor Martín Romo. (2024). Design and Development of Automatization Pipelines for Machine Learning Projects based on the MLOps Frameworks. Final Career Project (TFM). Universidad Politécnica de Madrid, ETSI Telecomunicación.

Abstract:
This Master Thesis (TFM) focuses on the evaluation, selection and deployment of MLOps environments, highlighting the analysis, design, implementation and validation of systems for the management of Machine Learning projects. This study compares multiple MLOps platforms and tools available in the market, considering criteria such as workflow automation, model management, scalability and continuous integration. After a thorough analysis, the most suitable option is chosen and implemented in a real case study. In addition to addressing the initial objectives, this TFM aims to outline the conclusions reached and the objectives achieved. Designed to solve existing challenges in deploying machine learning models in production, the project yields substantial and transformative results. Inspired by Deborah Leff’s observations on the alarming failure rate of data science projects in reaching production, the overall objectives are meticulously designed to address these problems and improve the success rate of machine learning deployments. Throughout the project lifecycle, meticulous requirements analysis is conducted, resulting in a customised architecture that not only meets organisational needs, but also demonstrates scalability, security and compliance. The automated process of continuous integration and deployment fosters agility and reliability in model updates, ensuring efficient operations. The culmination of the case study provides tangible evidence of the effectiveness of the MLOps framework, exemplifying successful model deployment, valuable lessons learned and key performance metrics. By addressing past deployment challenges, integrating robust security measures and measuring business value through key performance indicators, the project demonstrates a holistic approach.