Job Requirements
Annapolis Junction, MD
Top Secret/SCI Full Scope Polygraph
Mid Level Career (5+ yrs experience)
Salary not specified
Join Premium to unlock estimated salaries 
Job Description
Candidates must already possess an active Top Secret/SCI w/ Full Scope Polygraph to be considered.
Summary:
• Design, implement, and maintain AI model and agentic AI benchmarks.
• Develop evaluation datasets, scoring frameworks, visualizations, and analyses.
• Utilize Python, MongoDB, data visualization, statistical analysis, data cleaning and validation, Git, Jira, Confluence, and containerized environments.
Qualifications & Compensation:
• Degree: Technical bachelor's degree or equivalent experience
• Years of experience: 7+ years
• Total Compensation: $259k+ yearly
Job Description:
• Design, implement, and maintain benchmark suites that measure model performance against defined evaluation criteria, including dataset curation, scoring, and results reporting.
• Develop benchmarks and test scenarios that evaluate agentic workflows and multi-step task performance, including tool-use, task completion, and reliability metrics.
• Curate, validate, and version evaluation datasets and test scenarios, including any adversarial or edge-case sets needed to stress agent behavior.
• Produce scoring frameworks, visualizations, and written analyses that translate raw evaluation results into actionable findings for stakeholders.
• Agentic AI
• Model Training
• Software Testing
• Python
• MongoDB
• Git
• JIRA
• Confluence
• Software Development
• Data Science
• Data Visualization
• Docker
• Kubernetes
About SYSTOLIC:
SYSTOLIC is dedicated to giving our employees the best possible company experience so that they can focus on providing outstanding support to their customer’s mission. Our company is founded on integrity, enthusiasm, and a relentless commitment to supporting the Intelligence Community. You can learn more about us and submit an application to be considered against our current and future openings at https://systolic.com.
To learn about our compensation ranges, visit our Pay Transparency page at: https://systolic.com/pay-transparency
Summary:
• Design, implement, and maintain AI model and agentic AI benchmarks.
• Develop evaluation datasets, scoring frameworks, visualizations, and analyses.
• Utilize Python, MongoDB, data visualization, statistical analysis, data cleaning and validation, Git, Jira, Confluence, and containerized environments.
Qualifications & Compensation:
• Degree: Technical bachelor's degree or equivalent experience
• Years of experience: 7+ years
• Total Compensation: $259k+ yearly
Job Description:
• Design, implement, and maintain benchmark suites that measure model performance against defined evaluation criteria, including dataset curation, scoring, and results reporting.
• Develop benchmarks and test scenarios that evaluate agentic workflows and multi-step task performance, including tool-use, task completion, and reliability metrics.
• Curate, validate, and version evaluation datasets and test scenarios, including any adversarial or edge-case sets needed to stress agent behavior.
• Produce scoring frameworks, visualizations, and written analyses that translate raw evaluation results into actionable findings for stakeholders.
• Agentic AI
• Model Training
• Software Testing
• Python
• MongoDB
• Git
• JIRA
• Confluence
• Software Development
• Data Science
• Data Visualization
• Docker
• Kubernetes
About SYSTOLIC:
SYSTOLIC is dedicated to giving our employees the best possible company experience so that they can focus on providing outstanding support to their customer’s mission. Our company is founded on integrity, enthusiasm, and a relentless commitment to supporting the Intelligence Community. You can learn more about us and submit an application to be considered against our current and future openings at https://systolic.com.
To learn about our compensation ranges, visit our Pay Transparency page at: https://systolic.com/pay-transparency
group id: 10527119