Senior Machine Learning Engineer, Accelerated Apache Spark

What you'll do

• Design and implement machine learning solutions for performance prediction and optimization of GPU-accelerated Apache Spark workloads.
• Develop advanced algorithms and adaptive systems to improve Spark performance on GPUs.
• Create AI-based agents and tools to automate system issue resolution and application optimization.
• Deploy complex machine learning solutions for key partners and customers across various environments.
• Maintain deep domain expertise in the latest ML systems and algorithms.
• Provide technical mentorship and leadership in data science and machine learning to a team of engineers.

What we're looking for

• BS, MS, or PhD in Machine Learning, Data Science, Computer Science, or a closely related field, or equivalent experience.
• 12+ years of professional experience in designing, implementing, and productionizing high-quality ML/DL solutions.
• Experience as a technical lead in ML model development.
• 2+ years of hands-on experience with large-scale data processing platforms such as Apache Spark.
• Excellent programming skills in Python and libraries including numpy, pandas, scikit-learn, scipy, pytorch, and tensorflow.
• Deep experience with LLM/GenAI, reinforcement learning, and adaptive, on-line ML systems.
• Strong expertise in feature engineering, feature importance assessment, and developing boosted tree model solutions like XGBoost.
• Understanding of Apache Spark architecture, NVIDIA GPUs, CUDA, and experience coding in Scala, Java, and/or C++ (preferred).