MLOps engineer

Job Description:

Overall Responsibilities

• Design, develop, deploy, and optimize MLOps pipelines
• Monitor and troubleshoot existing data and model pipelines to identify performance bottlenecks
• Collaborate with Dagster engineers, AWS Cloud engineers, and other AI/ML experts to enhance infrastructure support for AI/ML workloads

Technical Responsibilities

• Implement automated model retraining pipelines with SageMaker Pipelines.
• Optimize feature engineering, data ingestion, and preprocessing workflows.
• Enable MLOps pipelines to orchestrate feature engineering, model training, and deployment pipelines.
• Experience with AWS services like SageMaker, SageMaker Pipelines, SageMaker Feature Store, Lambda
• Expertise in building robust, modular code modules using Python.
• Deploy ML models using SageMaker Endpoints, Lambda, or ECS.
• Set up model monitoring, drift detection, and explainability frameworks.
• Implement model versioning and lineage tracking using MLflow or SageMaker Model Registry.
• Automate model evaluation, testing, and A/B testing frameworks.
• Ensure ML workflows follow best practices for security, performance, and cost-efficiency.
• Integrate Dagster workflows with MLOps pipelines for efficient creating workflows.
• Ensure data pipeline reliability, fault tolerance, and scalability.
• Implement observability and monitoring for pipeline execution and failures.
• Optimize execution time and parallel processing for ML workloads
• Experience with DevOps practices, CI/CD pipelines, and GitHub/GitLab workflows.
• Strong team player with ability to collaborate effectively across different teams