Software Engineer III

What you'll do

• Design and build automated frameworks for functional equivalence, performance benchmarking, and data integrity.
• Develop contract testing frameworks to ensure consistency during application and database migrations.
• Integrate Large Language Models into developer workflows to automate unit test generation and edge-case discovery.
• Curate golden datasets and adversarial test sets to evaluate LLM accuracy, safety, and relevance.
• Implement state comparison algorithms and change data capture for database modernization projects.
• Develop complex software testkits and infrastructure tools using the Java/Spring ecosystem.
• Troubleshoot and resolve complex technical issues within distributed systems.
• Contribute high-quality code and participate in peer code reviews to enforce best practices.

What we're looking for

• Must have at least 4 years of commercial software development experience with a JVM language like Java and the Spring ecosystem.
• Proven experience evaluating Large Language Models (LLMs) and using evaluation frameworks like DeepEval, Ragas, or Promptfoo.
• Experience integrating LLMs into developer workflows for automated unit test generation and edge-case discovery.
• Experience curating golden datasets, adversarial test sets, and establishing scoring rubrics to test LLM accuracy and safety.
• Experience building developer tools, infrastructure, or complex testing frameworks.
• Expertise in software architecture and designing frameworks that ensure database state equivalence across different implementations.
• Proficiency in both relational (SQL) and document data modeling using databases like Postgres and MongoDB.
• Strong understanding of algorithms, data structures, and their time and space complexity.