Senior Principal Engineer II, Kafka

What you'll do

• Define and drive the multi-year technical strategy and roadmap for the Background Plane architecture.
• Design and optimize high-scale processing platforms including distributed scheduling, task coordination, and elastic autoscaling.
• Ensure mathematical and operational correctness for all data and metadata workflows under extreme concurrency.
• Build systems that maintain sub-millisecond foreground latencies while scaling to massive storage and throughput demands.
• Lead cross-organizational negotiations to establish architectural contracts for data references, encryption models, and task schedulers.
• Establish global engineering standards for fault tolerance, chaos testing, and deep observability to reduce systemic risk.
• Act as the primary technical authority during high-severity incident responses and complex architectural trade-off discussions.
• Mentor senior engineers and represent the company’s technical vision through research, publications, and open-source contributions.

What we're looking for

• Bachelor's Degree is required; a Master's Degree is preferred.
• Proven track record of architecting and operating mission-critical distributed systems at planetary scale.
• Deep expertise in storage engines, distributed log stores, stream processing, replication, or consensus protocols.
• Deep insight into Apache Kafka internals including storage semantics, log compaction, and tiered storage.
• Demonstrated ability to deliver complex multi-year architectural roadmaps across multi-disciplinary engineering organizations.
• Exceptional fluency in reasoning about race conditions, distributed concurrency, state machine transitions, and backpressure mechanisms.
• Proven ability to establish measurable SLA/SLO metrics and automated recovery practices for global infrastructure.
• Outstanding written and oral communication skills to distill technical concepts for executive leadership.