Job Requirements
Fairfax, VA
Secret Polygraph Unspecified
Career Level not specified
Salary not specified
Join Premium to unlock estimated salaries 
Job Description
Job Description
Everforth ECS is seeking a Senior Evaluation Automation Engineer to work in the National Capital Region covering the Pentagon, Falls Church, and Fairfax . Please Note: This position is contingent upon contract award.
The War Data Platform (WDP) is a key initiative within the U.S. Department of War's (DoW) AI-First strategy introduced in early 2026. The WDP focuses on operational warfighting data and aims to accelerate the deployment of artificial intelligence (AI) on the battlefield. The WDP extends to Unclassified, Secret, and Top Secret environments, and supports collaboration between Combatant Commands, Joint Staff directorates, Senior Executive Service leaders, and operational analysts.
This position leads the automation of evaluation and testing for AI/ML models in secured DoW environments.
• Develops, builds, and maintains automated test and evaluation checks executed within unclassified-first deployment pipelines supporting artificial intelligence and machine learning model serving missions for DoW programs, Joint Staff analysts, Combatant Command elements, and Senior Executive Service leadership.
• Designs and sustains automated unit, regression, and evaluation harness components that validate model artifacts across multiple enclaves and security domains.
• Executes higher-domain parity test and evaluation gate checks and resolves enclave-specific constraints to preserve gate credibility across promoted releases.
• Implements automated scanning, packaging, and deployment workflows using Kubernetes, GitLab Continuous Integration, PyTest, SonarQube, Tenable Nessus, and hardened artifact pipelines to validate reliability, operational suitability, and mission alignment of production-ready models.
• Integrates evaluation harnesses into enterprise DevSecOps pipelines to support mission assurance objectives and operational readiness requirements.
• Builds automated workflows supporting API endpoint configuration, proxy provisioning, and model zoo onboarding activities aligned with enterprise artificial intelligence model serving architectures.
• Produces mission-critical deliverables including automated test suites, evaluation harness documentation, readiness scorecards, operational risk assessments, and deployment decision artifacts.
• Collaborates with Platform One, Cloud One, multi-national engineering teams, and cross-service mission partners to advance automation maturity, strengthen deployment consistency, and reinforce program value commitments across all environments.
• Supports Tier-4 incident response actions to maintain service-level agreements and operational continuity for mission stakeholders.
• Performs other duties as assigned.
Required Skills
• Current Secret security clearance with the ability to obtain and maintain a Top Secret (TS) security clearance with Sensitive Compartmented Information (SCI).
• 10-12 years of experience developing automated evaluation and testing solutions for AI/ML models in secure environments.
• Strong proficiency with Kubernetes, GitLab CI, PyTest, SonarQube, and Tenable Nessus for automated test harnesses .
• Demonstrated ability to design and maintain multi-domain, enclave-aware deployment pipelines and gate checks .
• Experience with cross-domain data replication, model zoo integration, and API gateway configuration workflows.
• Agile development background with application lifecycle management and knowledge of programming languages such as Java, Python, and Ruby .
• CompTIA A+ certification (or equivalent foundational IT support knowledge).
Desired Skills
• Active Top Secret (TS) security clearance with Sensitive Compartmented Information (SCI) eligibility.
• Proven track record in incident-response automation and Tier-4 event handling in high-assurance environments.
• Expertise in AI/ML model lifecycle management, including versioning, drift detection, and model monitoring dashboards.
• Experience collaborating with cross-service engineering teams (Platform One, Cloud One) to standardize automation practices.
• Strong analytical skills for interpreting test scorecards, SLO metrics, and risk assessment outputs.
ECS Federal LLC is an equal opportunity employer and does not discriminate or allow discrimination on the basis any characteristic protected by law. All qualified applicants will receive consideration for employment without regard to disability, status as a protected veteran or any other status protected by applicable federal, state, or local jurisdiction law.
is the federal segment of , a $4B global organization with over 10,000 employees. Our nearly 3,500 professionals deliver advanced technology solutions in data and AI, cybersecurity, and enterprise transformation, serving defense, intelligence, and federal civilian agencies.
Our work powers mission-critical outcomes, strengthens technology partnerships, and creates meaningful opportunities for our people. We are defined by a commitment to excellence in delivery, a culture of innovation, and an environment where talent can thrive and grow.
We value:
Everforth ECS is seeking a Senior Evaluation Automation Engineer to work in the National Capital Region covering the Pentagon, Falls Church, and Fairfax . Please Note: This position is contingent upon contract award.
The War Data Platform (WDP) is a key initiative within the U.S. Department of War's (DoW) AI-First strategy introduced in early 2026. The WDP focuses on operational warfighting data and aims to accelerate the deployment of artificial intelligence (AI) on the battlefield. The WDP extends to Unclassified, Secret, and Top Secret environments, and supports collaboration between Combatant Commands, Joint Staff directorates, Senior Executive Service leaders, and operational analysts.
This position leads the automation of evaluation and testing for AI/ML models in secured DoW environments.
• Develops, builds, and maintains automated test and evaluation checks executed within unclassified-first deployment pipelines supporting artificial intelligence and machine learning model serving missions for DoW programs, Joint Staff analysts, Combatant Command elements, and Senior Executive Service leadership.
• Designs and sustains automated unit, regression, and evaluation harness components that validate model artifacts across multiple enclaves and security domains.
• Executes higher-domain parity test and evaluation gate checks and resolves enclave-specific constraints to preserve gate credibility across promoted releases.
• Implements automated scanning, packaging, and deployment workflows using Kubernetes, GitLab Continuous Integration, PyTest, SonarQube, Tenable Nessus, and hardened artifact pipelines to validate reliability, operational suitability, and mission alignment of production-ready models.
• Integrates evaluation harnesses into enterprise DevSecOps pipelines to support mission assurance objectives and operational readiness requirements.
• Builds automated workflows supporting API endpoint configuration, proxy provisioning, and model zoo onboarding activities aligned with enterprise artificial intelligence model serving architectures.
• Produces mission-critical deliverables including automated test suites, evaluation harness documentation, readiness scorecards, operational risk assessments, and deployment decision artifacts.
• Collaborates with Platform One, Cloud One, multi-national engineering teams, and cross-service mission partners to advance automation maturity, strengthen deployment consistency, and reinforce program value commitments across all environments.
• Supports Tier-4 incident response actions to maintain service-level agreements and operational continuity for mission stakeholders.
• Performs other duties as assigned.
Required Skills
• Current Secret security clearance with the ability to obtain and maintain a Top Secret (TS) security clearance with Sensitive Compartmented Information (SCI).
• 10-12 years of experience developing automated evaluation and testing solutions for AI/ML models in secure environments.
• Strong proficiency with Kubernetes, GitLab CI, PyTest, SonarQube, and Tenable Nessus for automated test harnesses .
• Demonstrated ability to design and maintain multi-domain, enclave-aware deployment pipelines and gate checks .
• Experience with cross-domain data replication, model zoo integration, and API gateway configuration workflows.
• Agile development background with application lifecycle management and knowledge of programming languages such as Java, Python, and Ruby .
• CompTIA A+ certification (or equivalent foundational IT support knowledge).
Desired Skills
• Active Top Secret (TS) security clearance with Sensitive Compartmented Information (SCI) eligibility.
• Proven track record in incident-response automation and Tier-4 event handling in high-assurance environments.
• Expertise in AI/ML model lifecycle management, including versioning, drift detection, and model monitoring dashboards.
• Experience collaborating with cross-service engineering teams (Platform One, Cloud One) to standardize automation practices.
• Strong analytical skills for interpreting test scorecards, SLO metrics, and risk assessment outputs.
ECS Federal LLC is an equal opportunity employer and does not discriminate or allow discrimination on the basis any characteristic protected by law. All qualified applicants will receive consideration for employment without regard to disability, status as a protected veteran or any other status protected by applicable federal, state, or local jurisdiction law.
is the federal segment of , a $4B global organization with over 10,000 employees. Our nearly 3,500 professionals deliver advanced technology solutions in data and AI, cybersecurity, and enterprise transformation, serving defense, intelligence, and federal civilian agencies.
Our work powers mission-critical outcomes, strengthens technology partnerships, and creates meaningful opportunities for our people. We are defined by a commitment to excellence in delivery, a culture of innovation, and an environment where talent can thrive and grow.
We value:
- Attracting and developing top talent and high-performing teams
- Fostering a culture that is engaging, accountable, and mission-driven
group id: 10112231A