Job Requirements
Remote
Secret Polygraph not specified
Career Level not specified
Salary not specified
Join Premium to unlock estimated salaries 
Job Description
Summary:
Yorktown Systems Group, Inc. (YSG) is seeking a highly qualified Evaluation and Research Director to provide comprehensive Warfighter Training Support Services at the United States Air Force Academy (USAFA). The purpose of this position is to support the planning, development, execution, management, and evaluation of military training for approximately 4,200 cadets annually during both the academic year and summer training programs. This position will assist the USAFA in delivering a coherent, progressive, and operationally relevant 47-month cumulative military training continuum aligned with Department of the Air Force (DAF) readiness objectives, Joint Force expectations, and USAFA cadet development outcomes.
Specific duties may include, but are not limited to:
• Serves as the lead and primary Evaluation and Research representative, providing management and oversight of the Evaluation & Research Team, as well as being the point of contact for all program Evaluation & Research activities/efforts.
• Convert execution findings into future planning inputs
• Aggregate and analyze performance data across events, cohorts, and training cycles within the Moodle/TUTOR VSP to identify trends, systemic strengths, and recurring performance gaps.
• Work with USAFA leadership, instructional designers, and training support personnel to ensure evaluation methodologies, performance measures, and assessment requirements remain aligned with program objectives.
• Convert evaluation findings, performance trends, and lessons learned into recommendations that support refinement of warfighter competencies, training effectiveness, learning outcomes, and program improvement initiatives.
• Converts execution observations, metrics, and lessons learned into resource planning inputs that inform future assessments, refinement, and recommendations.
• Capture observations, rubric results, AAR inputs, instructor feedback, participant performance data, and lessons learned during and after training events.
• Collectively execute all program evaluation activities across the WTIS framework.
• Apply Kirkpatrick’s four-level training evaluation model within a repeatable process that integrates assessment tool development, disciplined data capture, structured AAR consolidation, and performance analysis within the required environment and tools.
• Define and maintain the data structure and workflows required to capture, store, and export evaluation data in a manner that supports consistent analysis.
• Conducts analytical review of existing and available Program of Instruction (POI), courseware, lesson materials, exercise scenarios, training schedules, TLOs, ELOs, primary tasks and objectives, evaluation criteria, recurring training events, historical planning products, and after-action inputs.
• Use assessment data and other evaluation inputs to develop dashboard reporting, leadership briefings, annual program assessments, and evidence-based recommendations for program improvement.
• Design and execute qualitative and quantitative research studies that assess cadet development, training effectiveness, and progression across the military training continuum.
• Prepare executive-level reports, dashboards, briefings, and recommendations for USAFA leadership utilizing evaluation findings and performance trends.
• Ensure evaluation methodologies, data collection activities, and reporting processes meet accepted standards for program evaluation, research integrity, and statistical validity
Required Qualifications:
• Must possess 8+ years experience as an evaluation and/or research lead.
• Graduate degree in Program Evaluation, Education Statistics, Social Science, Research Methods, Organizational Development, or related field.
• Training/certification in evaluation methodologies (e.g., ROI Institute, Kirkpatrick).
• Experience supporting assessment frameworks, AAR methodology, rubric development, data analysis, trend reporting, and presentation of findings to stakeholders.
• Experience evaluating programs/projects; creation of evaluation plans and logic models; developing data collection tools and procedures; facilitating data collection; conducting statistical analysis of data; writing evaluation reports and providing statistical analysis of programs of similar scale and duration.
• Possess an in-depth and current understanding of program evaluation including the four major categories of standards (propriety, utility, feasibility, and accuracy) when conducting a program evaluation.
• Ability to evaluate the implementation and effectiveness of key initiatives and programs through the use of quantitative and qualitative research methods, and to provide results that inform policymakers and stakeholders' decisions regarding program viability and improvement.
• Ability to capture, conduct analysis, and organize collected.
• Possess an in-depth and current understanding of research and evaluation methods and techniques.
Clearance: Must be able to obtain a SECRET security clearance.
Location: Colorado Springs, CO
Travel: Some travel will be required
Yorktown Systems Group, Inc. (YSG) is seeking a highly qualified Evaluation and Research Director to provide comprehensive Warfighter Training Support Services at the United States Air Force Academy (USAFA). The purpose of this position is to support the planning, development, execution, management, and evaluation of military training for approximately 4,200 cadets annually during both the academic year and summer training programs. This position will assist the USAFA in delivering a coherent, progressive, and operationally relevant 47-month cumulative military training continuum aligned with Department of the Air Force (DAF) readiness objectives, Joint Force expectations, and USAFA cadet development outcomes.
Specific duties may include, but are not limited to:
• Serves as the lead and primary Evaluation and Research representative, providing management and oversight of the Evaluation & Research Team, as well as being the point of contact for all program Evaluation & Research activities/efforts.
• Convert execution findings into future planning inputs
• Aggregate and analyze performance data across events, cohorts, and training cycles within the Moodle/TUTOR VSP to identify trends, systemic strengths, and recurring performance gaps.
• Work with USAFA leadership, instructional designers, and training support personnel to ensure evaluation methodologies, performance measures, and assessment requirements remain aligned with program objectives.
• Convert evaluation findings, performance trends, and lessons learned into recommendations that support refinement of warfighter competencies, training effectiveness, learning outcomes, and program improvement initiatives.
• Converts execution observations, metrics, and lessons learned into resource planning inputs that inform future assessments, refinement, and recommendations.
• Capture observations, rubric results, AAR inputs, instructor feedback, participant performance data, and lessons learned during and after training events.
• Collectively execute all program evaluation activities across the WTIS framework.
• Apply Kirkpatrick’s four-level training evaluation model within a repeatable process that integrates assessment tool development, disciplined data capture, structured AAR consolidation, and performance analysis within the required environment and tools.
• Define and maintain the data structure and workflows required to capture, store, and export evaluation data in a manner that supports consistent analysis.
• Conducts analytical review of existing and available Program of Instruction (POI), courseware, lesson materials, exercise scenarios, training schedules, TLOs, ELOs, primary tasks and objectives, evaluation criteria, recurring training events, historical planning products, and after-action inputs.
• Use assessment data and other evaluation inputs to develop dashboard reporting, leadership briefings, annual program assessments, and evidence-based recommendations for program improvement.
• Design and execute qualitative and quantitative research studies that assess cadet development, training effectiveness, and progression across the military training continuum.
• Prepare executive-level reports, dashboards, briefings, and recommendations for USAFA leadership utilizing evaluation findings and performance trends.
• Ensure evaluation methodologies, data collection activities, and reporting processes meet accepted standards for program evaluation, research integrity, and statistical validity
Required Qualifications:
• Must possess 8+ years experience as an evaluation and/or research lead.
• Graduate degree in Program Evaluation, Education Statistics, Social Science, Research Methods, Organizational Development, or related field.
• Training/certification in evaluation methodologies (e.g., ROI Institute, Kirkpatrick).
• Experience supporting assessment frameworks, AAR methodology, rubric development, data analysis, trend reporting, and presentation of findings to stakeholders.
• Experience evaluating programs/projects; creation of evaluation plans and logic models; developing data collection tools and procedures; facilitating data collection; conducting statistical analysis of data; writing evaluation reports and providing statistical analysis of programs of similar scale and duration.
• Possess an in-depth and current understanding of program evaluation including the four major categories of standards (propriety, utility, feasibility, and accuracy) when conducting a program evaluation.
• Ability to evaluate the implementation and effectiveness of key initiatives and programs through the use of quantitative and qualitative research methods, and to provide results that inform policymakers and stakeholders' decisions regarding program viability and improvement.
• Ability to capture, conduct analysis, and organize collected.
• Possess an in-depth and current understanding of research and evaluation methods and techniques.
Clearance: Must be able to obtain a SECRET security clearance.
Location: Colorado Springs, CO
Travel: Some travel will be required
group id: 10480131
