AI Test Engineer

Overview

This role plays a vital part in ensuring the quality, reliability, and ethical performance of AI-driven products. As an AI Tester, you will design and execute test plans, evaluate model performance, identify potential risks, and deliver actionable feedback to developers and data scientists. In this position, you will help bridge the gap between cutting-edge machine learning and real-world usability, ensuring our AI systems remain accurate, safe, fair, and user-friendly.

Contributions

• Develop and execute comprehensive test plans for AI/ML models, including functional, regression, and edge case testing.
• Design test cases to evaluate performance, accuracy, bias, fairness, and robustness of AI outputs.
• Collaborate with product managers, data scientists, and engineers to define acceptance criteria for AI features.
• Conduct adversarial and stress testing to assess model resilience under unexpected inputs.
• Document and report defects, inconsistencies, and unexpected behaviors.
• Track key quality metrics such as precision, recall, F1 score, false positives/negatives, and usability metrics.
• Validate compliance with data privacy, security, and ethical AI standards.
• Continuously update test frameworks and methodologies to align with advances in AI technology.

Qualifications

Required:

• Bachelor's degree in Computer Science, Data Science, Engineering, or related field (or equivalent practical experience).
• 2+ years of experience in QA, software testing, or validation (AI/ML testing preferred).
• Strong understanding of machine learning concepts and common evaluation metrics.
• Experience with testing frameworks (e.g., PyTest, JUnit) and automation tools.
• Familiarity with Python, SQL, or scripting for test automation.
• Excellent analytical, problem-solving, and documentation skills.

Preferred:

• Experience working with large language models, computer vision, or recommendation systems.
• Knowledge of data ethics, bias testing, and explainability techniques.
• Hands-on experience with ML frameworks (TensorFlow, PyTorch, Scikit-learn).
• Background in human-computer interaction (HCI) or usability testing.

Soft Skills

• Strong attention to detail with a curiosity for breaking things and finding edge cases.
• Clear communication skills to explain findings to both technical and non-technical stakeholders.
• Proactive mindset with the ability to anticipate risks and suggest improvements.
• Collaborative approach with a willingness to work across multidisciplinary teams.

About steampunk

S teampunk relies on several factors to determine salary, including but not limited to geographic location, contractual requirements, education, knowledge, skills, competencies, and experience. The projected compensation range for this position is $90,000 to $130,000. The estimate displayed represents a typical annual salary range for this position. Annual salary is just one aspect of Steampunk's total compensation package for employees. Learn more about additional Steampunk benefits here.

Identity Statement

As part of the application process, you are expected to be on camera during interviews and assessments. We reserve the right to take your picture to verify your identity and prevent fraud.

Steampunk is a Change Agent in the Federal contracting industry, bringing new thinking to clients in the Homeland, Federal Civilian, Health and DoD sectors. Through our Human-Centered delivery methodology, we are fundamentally changing the expectations our Federal clients have for true shared accountability in solving their toughest mission challenges. As an employee owned company, we focus on investing in our employees to enable them to do the greatest work of their careers - and rewarding them for outstanding contributions to our growth. If you want to learn more about our story, visit http://www.steampunk.com .

We are an equal opportunity employer and all qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, disability status, protected veteran status, or any other characteristic protected by law. Steampunk participates in the E-Verify program.