As a Senior Software Engineer in the distributed systems team based in Austin, TX, you will design and develop critical infrastructure components such as metadata services, coordination layers, and state management systems to ensure high-throughput, fault-tolerant operations at petabyte scale for watsonx.data. Your daily tasks include implementing replication, automatic failover mechanisms, and exactly-once processing to maintain data consistency under failure scenarios. You will also contribute to the CI/CD pipeline by integrating structured logging, distributed tracing, and metrics to enhance observability. Additionally, you will collaborate with cross-functional teams to debug complex issues and design diagnostic tools, ensuring seamless integration of GPU acceleration and AI/ML capabilities. The ideal candidate has extensive experience in Java, Go, or C++, deep knowledge of consistency models and consensus protocols like Raft or Paxos, and proficiency in Kubernetes for stateful workloads.
Design and implement metadata services and distributed schedulers for petabyte-scale systems.
Build fault-tolerant infrastructure with automatic failover and exactly-once processing guarantees.
Develop diagnostic tooling to debug and resolve customer-reported production issues.
Contribute to CI/CD pipeline by instrumenting components with logging, tracing, and metrics.
Work in an Agile environment to collaborate on reliability and scalability decisions.
What we're looking for
6+ years of professional software engineering experience with at least 2 years in large-scale distributed systems.
Proficiency in Java, Go, C++, or a comparable systems language for production-level distributed-system code.
Deep understanding and hands-on experience with consistency models, replication, quorum systems, leader election, and consensus protocols (Raft/Paxos).
Experience designing fault-tolerant systems with automatic failover, idempotent operations, durable recovery, and distributed observability tools.
Ability to communicate clearly through design documents, post-mortems, and capacity analyses; Bachelor's degree in Computer Science or equivalent experience.