Job Requirements
Annapolis Junction, MD
Top Secret/SCI Full Scope Polygraph
Mid Level Career (5+ yrs experience)
Salary not specified
Join Premium to unlock estimated salaries 
Job Description
Candidates must already possess an active Top Secret/SCI w/ Full Scope Polygraph to be considered.
Summary:
• Design and implement AI model and agentic AI benchmarks.
• Curate evaluation datasets and analyze performance metrics.
• Utilize Python, MongoDB, Git, Jira, Confluence, and data visualization tools to support software engineering efforts.
Qualifications & Compensation:
• Degree: Technical bachelor's degree or equivalent experience
• Years of experience: 7+ years
• Total Compensation: $259k+ yearly
Job Description:
• Design, implement, and maintain benchmark suites that measure AI model performance against defined evaluation criteria, including dataset curation, scoring, and results reporting.
• Develop benchmarks and test scenarios that evaluate agentic workflows and multi-step task performance, including tool-use, task completion, and reliability metrics.
• Curate, validate, and version evaluation datasets and test scenarios, including adversarial or edge-case sets needed to stress agent behavior.
• Produce scoring frameworks, visualizations, and written analyses that translate raw evaluation results into actionable findings for stakeholders.
• Apply key skills including Agentic AI, Software Development, Software Testing, Python, MongoDB, Git, JIRA, Confluence, Data Visualization, and Machine Learning/Artificial Intelligence.
About SYSTOLIC:
SYSTOLIC is dedicated to giving our employees the best possible company experience so that they can focus on providing outstanding support to their customer’s mission. Our company is founded on integrity, enthusiasm, and a relentless commitment to supporting the Intelligence Community. You can learn more about us and submit an application to be considered against our current and future openings at https://systolic.com.
To learn about our compensation ranges, visit our Pay Transparency page at: https://systolic.com/pay-transparency
Summary:
• Design and implement AI model and agentic AI benchmarks.
• Curate evaluation datasets and analyze performance metrics.
• Utilize Python, MongoDB, Git, Jira, Confluence, and data visualization tools to support software engineering efforts.
Qualifications & Compensation:
• Degree: Technical bachelor's degree or equivalent experience
• Years of experience: 7+ years
• Total Compensation: $259k+ yearly
Job Description:
• Design, implement, and maintain benchmark suites that measure AI model performance against defined evaluation criteria, including dataset curation, scoring, and results reporting.
• Develop benchmarks and test scenarios that evaluate agentic workflows and multi-step task performance, including tool-use, task completion, and reliability metrics.
• Curate, validate, and version evaluation datasets and test scenarios, including adversarial or edge-case sets needed to stress agent behavior.
• Produce scoring frameworks, visualizations, and written analyses that translate raw evaluation results into actionable findings for stakeholders.
• Apply key skills including Agentic AI, Software Development, Software Testing, Python, MongoDB, Git, JIRA, Confluence, Data Visualization, and Machine Learning/Artificial Intelligence.
About SYSTOLIC:
SYSTOLIC is dedicated to giving our employees the best possible company experience so that they can focus on providing outstanding support to their customer’s mission. Our company is founded on integrity, enthusiasm, and a relentless commitment to supporting the Intelligence Community. You can learn more about us and submit an application to be considered against our current and future openings at https://systolic.com.
To learn about our compensation ranges, visit our Pay Transparency page at: https://systolic.com/pay-transparency
group id: 10527119