Senior Software Development Engineer in Test, ML

Sr Software Development Engineer in Test, ML Locations: Madison, WI and Waterloo, Ontario Canada PlayStation is seeking a Sr Software Development Engineer in Test, ML to join our Madison team and lead quality efforts for machine learning-powered products and services. You will partner closely with ML engineers, software engineers, product managers, and data partners to build scalable test automation and validation approaches for systems serving PlayStation at global scale. This role is focused on building confidence in model reliability, production readiness, and behavioral consistency. Unlike traditional software backend testing, success here requires validating probabilistic systems through score distributions, cohort-level behavior, thresholds, and statistical signals, not just deterministic pass/fail assertions. Responsibilities: 37Drive architecture-level automation solutions and scalable quality engineering practices across multiple teams and product areas.37 37Design, develop, and maintain automated test frameworks, test scripts for ML inference services and related ML platform components using python/Java37 37Build and automate realistic, representative test account creations and datasets to evaluate model behavior across different scenarios37 37Validate score distributions, ranking behavior, output quality, and other model signals for incoming ML models prior to release37 37Perform different types of testing, including functional, integration, regression and API testing working with RESTful APIs, microservices and databases, and non-functional performance and load testing to verify inference service scalability, latency requirements and reliability under production like conditions37 37Work closely with ML engineers and ops teams to ensure that all features and bug fixes come with automated test coverage, ensuring continuous integration and deployment37 37Debug and analyze test failures, model anomalies and report defects37 37Drive high standards of quality for both engineering decisions and customer-facing features with optimal test and automation strategy37 37Develop and implement best practices for test automation that is highly scalable and maintainable37 37Monitor test execution and optimize performance in test environments37 37Come up with innovative solutions to organizational problems37 Required Qualifications: 37A Bachelors degree in Computer Science or related field37 37Minimum 5 years of experience as an SDET, with strong experience testing backend systems.37 37Proficiency in Python/Java/Scala/go for test automation.37 37Hands-on experience with API testing tools (e.g., Postman or similar) for distributed systems37 37Familiarity with CI/CD pipelines and test execution in Jenkins or similar environments such as jenkins, GitHub Actions, ArgoCD, etc37 37Hands-on experience building and maintaining automated test frameworks (e.g., pytest, JUnit, TestNG, etc.)Experience in working with databases for backend validation.37 37Experience with cloud and container technologies (AWS, GCP, Kubernetes, Docker, etc.)37 37Familiarity with monitoring and observability tools (e.g., Prometheus, Grafana, Datadog, etc.) for validating service health37 37Strong understanding of software development lifecycle (SDLC) and agile methodologies.37 37Excellent problem-solving skills and excellent communication skills.37 Preferred Qualifications: 37Experience testing machine learning systems, such as recommendation systems, search ranking systems, or other probabilistic ML application systems.37 37Experience creating synthetic or seeded test data to simulate realistic customer/account behaviors.37 37Experience evaluating model outputs using statistical techniques, distribution analysis, or scenario-based validation, rather than deterministic assertions.37 37Knowledge of data pipelines, feature stores, inference systems or model-serving infrastructure (Seldon, KServe, Ray Serve, etc.) is a plus.37 37Experience with performance and load testing tools 37 37Experience with Databricks or similar ML platform tooling37 37Experience architecting and optimizing CI/CD pipelines, quality gates, and release validation systems using Jenkins or similar platforms.37 At SIE, we consider several factors when setting each roles base pay range, including the competitive benchmarking data for the market and geographic location. Please note that the base pay range may vary in line with our hybrid working policy and individual base pay will be determined based on job-related factors which may include knowledge, skills, experience, and location. In addition, this role is eligible for SIEs top-tier benefits package that includes medical, dental, vision, matching 401(k), paid time off, wellness program and coveted employee discounts for Sony products. This role also may be eligible for a bonus package. Click here to learn more. The estimated base pay range for this role is listed below. $152,000-$190,000 USD