Job Requirements
Washington, DC
Clearance Unspecified Polygraph not specified
Career Level not specified
$120,000 - $130,000
Job Description
Clearance: Active DOD security clearance required.
The Prospective Group (TPG) is seeking an AI Test Technician to be part of the Center for Analytics program at the Department, performing duties to restore testing capacity and develop scalable LLM evaluations and automated testing for the growing portfolio of AI and analytics products.
Position Responsibilities:
• Execute daily manual and automated test cycles across data pipelines, applications, and AI/LLM-enabled tools to restore and stabilize baseline testing capacity.
• Design, build, and maintain automated test scripts and regression suites to reduce reliance on manual testing and increase test throughput.
• Develop scalable evaluation harnesses for large language models, including prompt/response accuracy checks, hallucination and bias detection, and output consistency scoring across use cases.
• Build and maintain test data sets, benchmark scenarios, and scoring rubrics to support repeatable LLM and application evaluations.
• Integrate automated tests and LLM evaluations into CI/CD pipelines to keep pace with frequent releases.
• Log, triage, and track defects and evaluation findings; coordinate with development teams to verify fixes and re-test as needed.
• Document test plans, evaluation criteria, and results in clear reports accessible to both technical and non-technical stakeholders.
• Track testing coverage and throughput metrics to identify capacity gaps and recommend process improvements.
Position Requirements:
• 2-4 years of experience in software testing or quality assurance, including building or running automated test frameworks.
• Experience with evaluating AI/LLM outputs (e.g., prompt testing, benchmark datasets, output scoring).
• Working knowledge of a scripting language (Python preferred) for building automated tests.
• Strong attention to detail and clear technical documentation skills.
• Position is located in the DC area and may require availability to be on site as required.
Education Requirements:
• A bachelor's degree, or equivalent experience.
Position Pay Range:
$120,000- $130,000 USD
Disclaimer The Prospective Group (TPG) is an IT management consulting company providing services internationally to government and commercial entities. Being prospective—leaning forward with action—defines the TPG culture. TPG and its employees have hundreds of years of experience, collectively, of successful program and project management, business process reengineering and optimization, enterprise architecture, mobile and cloud computing, cyber security, records management, and diplomatic business experience. TPG is a women-owned small business (WOSB), appraised at CMMI Level 3 with excellent performance credentials.
The Prospective Group (TPG) provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state, or local laws.
The Prospective Group (TPG) is seeking an AI Test Technician to be part of the Center for Analytics program at the Department, performing duties to restore testing capacity and develop scalable LLM evaluations and automated testing for the growing portfolio of AI and analytics products.
Position Responsibilities:
• Execute daily manual and automated test cycles across data pipelines, applications, and AI/LLM-enabled tools to restore and stabilize baseline testing capacity.
• Design, build, and maintain automated test scripts and regression suites to reduce reliance on manual testing and increase test throughput.
• Develop scalable evaluation harnesses for large language models, including prompt/response accuracy checks, hallucination and bias detection, and output consistency scoring across use cases.
• Build and maintain test data sets, benchmark scenarios, and scoring rubrics to support repeatable LLM and application evaluations.
• Integrate automated tests and LLM evaluations into CI/CD pipelines to keep pace with frequent releases.
• Log, triage, and track defects and evaluation findings; coordinate with development teams to verify fixes and re-test as needed.
• Document test plans, evaluation criteria, and results in clear reports accessible to both technical and non-technical stakeholders.
• Track testing coverage and throughput metrics to identify capacity gaps and recommend process improvements.
Position Requirements:
• 2-4 years of experience in software testing or quality assurance, including building or running automated test frameworks.
• Experience with evaluating AI/LLM outputs (e.g., prompt testing, benchmark datasets, output scoring).
• Working knowledge of a scripting language (Python preferred) for building automated tests.
• Strong attention to detail and clear technical documentation skills.
• Position is located in the DC area and may require availability to be on site as required.
Education Requirements:
• A bachelor's degree, or equivalent experience.
Position Pay Range:
$120,000- $130,000 USD
Disclaimer The Prospective Group (TPG) is an IT management consulting company providing services internationally to government and commercial entities. Being prospective—leaning forward with action—defines the TPG culture. TPG and its employees have hundreds of years of experience, collectively, of successful program and project management, business process reengineering and optimization, enterprise architecture, mobile and cloud computing, cyber security, records management, and diplomatic business experience. TPG is a women-owned small business (WOSB), appraised at CMMI Level 3 with excellent performance credentials.
The Prospective Group (TPG) provides equal employment opportunities to all employees and applicants for employment and prohibits discrimination and harassment of any type without regard to race, color, religion, age, sex, national origin, disability status, genetics, protected veteran status, sexual orientation, gender identity or expression, or any other characteristic protected by federal, state, or local laws.
group id: 90814559
