Senior Machine Learning Engineer, Accelerated Apache Spark

What you'll do

• Design and implement machine learning solutions for performance prediction of GPU-accelerated Apache Spark workloads.
• Develop advanced algorithms and adaptive systems to optimize the performance of Spark applications on GPUs.
• Create AI-based agents and tools to automate system issue resolution and application optimization.
• Deploy complex machine learning solutions in various environments for key partners and customers.
• Maintain deep domain expertise in the latest ML systems, algorithms, and large-scale data processing platforms.
• Provide technical mentorship and leadership in data science and machine learning to a team of engineers.

What we're looking for

• BS, MS, or PhD in Machine Learning, Data Science, Computer Science, or a related field.
• 12+ years of professional experience designing, implementing, and productionizing high-quality ML/DL solutions.
• Experience as a technical lead in ML model development.
• 2+ years of hands-on experience with large-scale data processing platforms like Apache Spark.
• Proficiency in Python and libraries including numpy, pandas, scikit-learn, scipy, pytorch, and tensorflow.
• Deep experience with LLM/GenAI, reinforcement learning, and adaptive, on-line ML systems.
• Expertise in feature engineering, feature importance assessment, and boosted tree model solutions like XGBoost.
• Familiarity with NVIDIA GPUs, CUDA, and programming languages such as Scala, Java, or C++.