ML OpsData Engineer

ML Ops/Python Data Engineer - Data Ingestion Team We are transforming HR data into actionable insights through AI and machine learning, enabling organizations to improve performance, engagement, and retention. The Data Ingestion Team plays a critical role in ensuring that our data platform is robust, scalable, and prepared for the future of AI-driven insights. As an ML Ops/Python Data Engineer, you will work closely with data engineers, data scientists, and software engineers to build and maintain the infrastructure that powers our machine learning models and data ingestion systems. You'll collaborate on designing, deploying, and refining data pipelines, chatbots for Slack, and various API integrations, ensuring seamless interaction between systems and supporting AI and ML initiatives. Key Responsibilities: Develop and maintain data pipelines using Python, DBT, Airflow, and Redshift, with a focus on scalability and efficiency. Explore and potentially integrate Snowflake for future data storage solutions. Work on building chatbots and data engines that leverage APIs, particularly for Slack, enabling dynamic and real-time data interaction. Collaborate with cross-functional teams, including data engineers, data scientists, and software engineers, to train, deploy, and maintain ML models. Design and implement data platforms, focusing on scalability, efficient decision-making, and seamless system integration. Contribute to both the ML Engineering/ML Ops efforts, ensuring the proper deployment and lifecycle management of models. Work closely with the DevOps team to integrate Kafka for event-driven architecture, with a focus on triggers and events that track units of action across the board. Collaborate with product teams to define and evolve platform features, ensuring a product-oriented approach that meets client needs. Provide input on system design and decision-making processes, especially as the team shifts toward a more AI-focused roadmap over the next 6 months. Requirements: Mid to Senior-level experience in Python development, with a focus on data engineering, API integrations, and ML Ops. Experience with data platforms and pipelines using tools such as DBT, Airflow, Redshift, and potentially Snowflake. Familiarity with Databricks for machine learning and data processing. Strong understanding of system design and decision-making in a data-driven environment. Experience working with event-driven systems, particularly using Kafka.

Created: 2024-11-05