AI Software Engineer in Test

About the role:

We are seeking a highly technical and innovative AI Software Engineer in Test (AI SEIT) to help modernize quality engineering through test automation, AI-assisted development, data validation, and intelligent software testing practices.

This role will leverage AI as an AI-enabled engineering assistant to accelerate test framework development, analyze codebases, generate and refactor automated tests, debug failures, inspect logs, improve coverage, and support rapid feedback loops across the SDLC.AI is an agentic coding tool that can read codebases, edit files, run commands, and integrate with development tools across terminal, IDE, desktop, and browser environments.

The ideal candidate will combine strong software engineering skills, quality engineering discipline, data validation expertise, and practical AI tool adoption to improve application quality, delivery velocity, and release confidence.

AI-Assisted Test Engineering:

• Use AI tools to accelerate test automation development, including generating, modifying, and refactoring test scripts across web, mobile, API, backend, and data platform applications.
• Leverage AI tools to analyze existing codebases, identify test coverage gaps, suggest edge cases, and improve automated regression coverage.
• Use AI to debug test failures, analyze stack traces, inspect logs, and recommend code or test corrections.
• Create repeatable AI workflows for test generation, defect triage, regression analysis, and automation maintenance.
• Provide verification criteria, test commands, build checks, and validation steps so AI agents can iteratively run checks and confirm whether changes pass or fail. Anthropic’s AI best practices recommend giving Claude a way to verify its work through tests, builds, linters, scripts, or similar pass/fail signals.
• Partner with engineering teams to establish safe and governed usage patterns for AI-assisted test development.

Test Automation Development:

• Design, develop, and maintain comprehensive automated test frameworks for web, mobile, backend, API, data, and AI-enabled applications.
• Build scalable automation solutions using tools and frameworks such as Selenium, Playwright, Cypress, REST Assured, JUnit, TestNG, PyTest, or equivalent technologies.
• Work closely with developers, QA engineers, product managers, and business analysts to ensure high test coverage and rapid feedback loops.
• Develop reusable automation components, shared libraries, test utilities, and framework standards.
• Maintain automation suites for functional, regression, integration, end-to-end, API, performance, security, and data validation testing.

AI, GenAI, and Model Validation:

• Develop test strategies for AI-enabled applications, Generative AI workflows, LLM-powered features, RAG solutions, conversational AI, and agentic systems.
• Validate AI outputs for accuracy, relevance, completeness, hallucination risk, toxicity, policy alignment, and business rule compliance.
• Create automated evaluation datasets, prompt test suites, golden answer sets, and scoring rubrics.
• Build regression testing approaches for prompts, model upgrades, agent workflows, and AI-assisted business processes.
• Support Responsible AI testing practices, including bias, fairness, explainability, traceability, and audit readiness.
• Partner with Architecture, Security, Data Science, and Engineering teams to define AI quality gates.

Code Quality Advocacy:

• Advocate for and contribute to code quality initiatives across development and QA teams.
• Participate in code reviews, test reviews, design discussions, and architecture reviews.
• Promote engineering best practices including clean code, maintainability, observability, testability, and secure coding.
• Use AI to assist with code review preparation, test refactoring, duplication detection, and documentation updates.
• Help teams shift quality left by embedding testability and automation requirements earlier in the development lifecycle.

CI/CD and Continuous Testing:

• Integrate automated tests into CI/CD pipelines using Jenkins, GitLab CI, Azure DevOps, or equivalent platforms.
• Ensure reliable regression execution and rapid feedback to development teams.
• Establish automated quality gates for build validation, test execution, code coverage, security checks, and release readiness.
• Use AI to troubleshoot failing pipeline jobs, analyze logs, and recommend fixes where appropriate.
• Collaborate with DevOps and Platform Engineering teams to improve deployment confidence and reduce release risk.

Requirement Analysis and Test Strategy:

• Collaborate with product managers, developers, QA analysts, architects, and stakeholders to understand user stories, business requirements, and acceptance criteria.
• Translate requirements into effective test strategies, test plans, automation scenarios, and validation criteria.
• Identify functional, technical, data, AI, integration, performance, and operational testing risks.
• Define test coverage expectations and traceability between requirements, test cases, defects, and release decisions.

Education:

• Bachelor’s or Master’s degree in Computer Science, Software Engineering, Information Systems, Data Science, Artificial Intelligence, or a related technical discipline.

Experience:

• 2+ years of hands-on experience in software development, test automation, or quality engineering.
• Experience building and maintaining automated test frameworks.
• Experience working in AI DLC, Agile delivery models, including Scrum and Kanban.
• Experience with fast-paced delivery environments and short turnaround times.
• 2 to 4+ years of database experience, preferably SQL.
• Experience using AI-assisted coding tools such as Claude Code, Kiro, Copilot, or similar tools preferred.

Our compensation structure is designed to reflect the cost of labor across various U.S. geographic markets. The base salary for this role ranges from $115,000 per year to $145,000 per year. Compensation will be determined by several factors, including relevant knowledge, skills, and experience. This role is also eligible to receive an annual bonus within a comprehensive total rewards package, alongside a full suite of medical, dental, vision, disability insurance, life insurance, 401k retirement benefits, paid time off, paid holidays, and paid personal/sick time. For further details, please visit https://www.greensky.com/benefits.

We anticipate that this position will remain open for at least 5 days, and candidates are encouraged to apply through our internal or external career sites.

If you have any questions about this job posting, please contact recruiting@greensky.com.