The role of the project manager is both critical and challenging. They are responsible for the project plan and its execution. At the beginning of the project, they help define the plan and set deadlines based on stakeholder requests and the capabilities of the technical team. Throughout the project, they constantly monitor progress. If the actual status of tasks or deliveries deviates from the plan, they must alert and coordinate with the teams. As a result, they spend most of their time communicating with different teams, senior managers, and business stakeholders. Two important challenges in your work are:
- Interdependence between technical teams: This makes the role challenging because the outputs of one team (e.g., data engineers ingesting the data) serve as inputs to another team (e.g., data scientists consuming the data). Any delay or change in the first step impacts the second step. Project managers, although not typically very technical, must be aware of these changes and ensure proper communication between teams.
- Competing business priorities: Business stakeholders often change their priorities, or there may be competing priorities between different teams that need to be aligned. Project managers must navigate these changes and align the various teams to keep the project on track.
By effectively managing these challenges, project managers play a critical role in successfully executing machine learning projects.
The experience and knowledge of fraud analysts are crucial for the development and evaluation of fraud prediction models. From the beginning of the project, they provide information on active fraud trends, common fraudulent scenarios and red flags, as well as exceptions or “green flags.” Data scientists incorporate this knowledge during the feature creation/engineering phase. Once the model is running in production, constant monitoring is required to maintain or improve performance. At this stage, fraud analysts are essential to identify the true or false positives of the model. This identification may result from a thorough investigation of the client's history or by contacting the client for confirmation. Feedback from fraud analysts is an integral part of the feedback loop process.
C-level managers and C-level executives play a crucial role in the success of ML/ai fraud projects. Your support is essential to eliminate obstacles and generate consensus on the strategic direction of the project. Therefore, it is necessary to update them periodically on the progress of the project. So that they can support the promotion of investments in the necessary equipment, tools and processes based on the specific requirements of the project and ensure that appropriate resources are allocated. Additionally, they are responsible for holding internal and external parties accountable for data privacy and compliance with industry standards. By fostering a culture of accountability and providing clear leadership, they help ensure that the project meets its objectives and integrates seamlessly with the organization's overall strategy. Your participation is vital to address any regulatory concerns, manage risk, and drive the project toward successful implementation and long-term sustainability.
Data engineers provide us (data scientists) with the data we need to build models, which is an essential step in any ML project. They are responsible for designing and maintaining data pipelines, whether for real-time data streams or batch processes in data warehouses. Data engineers, involved from the beginning of the project, identify data requirements, sources, processing needs, and SLA requirements for data accessibility.
They create pipelines to collect, transform, and store data from various sources, essentially handling the ETL process. They also manage and maintain these channels, addressing scalability requirements, monitoring data quality, optimizing queries and processes to improve latency and reducing costs.
On paper, data scientists create machine learning algorithms to predict various types of information for the business. We actually wear many different hats throughout the day. We begin by identifying the business problem, understanding the data and available resources, and defining a solution, translating it into technical requirements.
Data scientists collaborate closely with data engineers and MLOps engineers to implement solutions. We also work with business stakeholders to communicate results and receive feedback. Model evaluation is another critical responsibility, which involves selecting appropriate metrics to evaluate model performance, continuously monitoring and reporting it, and observing any deterioration in performance.
The continuous improvement process is fundamental to the role of a data scientist, to ensure that models remain accurate and relevant over time.
Once data engineers and scientists build the data pipelines and model, it is time to put the model into production. MLOps engineers play a crucial role in this phase by bridging the gap between development and operations. In the context of fraud prediction, time is of the essence, as the company needs to prevent fraud before it occurs, which requires a pipeline process that runs in less than a second. Therefore, Mlops engineers ensure that models integrate seamlessly into production environments, maintaining reliability and scalability. MLOps engineers design and manage the infrastructure necessary for model deployment, implement continuous integration and continuous deployment (CI/CD) pipelines, and monitor model performance in real time. They also handle version control, automate testing, and manage model retraining processes to keep them up to date. By addressing these operational challenges, MLOps engineers enable the smooth and efficient deployment of machine learning models, ensuring they deliver consistent and valuable results for the business.
We talked about the roles that I have identified in my work experience. These roles interact differently depending on the stage of the project and each specific company. In my experience, at the beginning of the project, fraud analysts, senior managers, and data scientists work together to define the strategy and requirements. Data scientists play an important role in identifying the business problem. They collaborate with Mlops and Engineering to translate it into a technical solution. Data engineers should join us to discuss the necessary developments in the processes. A common challenge is when there is a disconnect between these teams and it simply arises at the time of execution. This can affect the deadlines and quality of the deliverable. Therefore, the greater the integrity between these teams, the smoother the implementation and delivery will be.
Comment below about the roles in your company. How are things different in your experience?
