4356 Data Scientist

OVERVIEW:

As the Data Scientist, you will report to the onsite Project Team leader and work closely with a small exercise support team. You will own problems end-to-end, to include designing evaluation frameworks, improving prompts and retrieval pipelines, analyzing interview data, and shipping

measurable improvements into production.

GENERAL DUTIES:

LLM Quality & Evaluation

• Improve prompting strategies and structured outputs across reporting formats
• including law enforcement, intelligence, after-action, interview summaries, and
• survey analysis
• Design evaluation sets, scoring rubrics, and automated evaluation pipelines
• (including LLM-as-judge approaches) for relevance, coherence, completeness,
• and error modes
• Reduce hallucinations and improve traceability and attribution

RAG & Knowledge Pipelines

• Build and iterate on RAG pipelines, curated knowledge packs, and question-tree
• triggers
• Create and maintain base datasets (follow-up triggers, Essential Elements of
• Information/Critical Information Requirements, glossaries, watchlist cues) with
• versioning and documentation
• Tune retrieval and reranking to perform reliably under edge constraints (limited
• compute, memory, and connectivity)

Interview Analytics

• Analyze transcripts to surface evasiveness, inconsistencies, and actionable leads
• Develop labeling strategies, analytic rubrics, and ground-truth datasets
• Conduct quantitative and qualitative analysis of interview data to identify patterns
• and support operational decisions

Measurement & Documentation

• Build lightweight dashboards and metrics for model performance and field
• reliability
• Document methods and maintain audit trails so outputs remain defensible for
• government end users
• Partner with engineering to validate and ship improvements into production

REQUIRED QUALIFICATIONS:

• 2-5 years of applied data science or NLP experience
• Strong Python skills (pandas, NumPy, scikit-learn) with comfort standing up
• experiments and pipelines
• Hands-on experience with LLMs: prompt engineering, output evaluation, safety,
• and quality controls
• Experience with unstructured text data - cleaning, labeling, building evaluation
• metrics
• Proficiency in data analysis, reporting, and visualization for technical and non-
• technical audiences
• Ability to work on-site in the El Paso / Las Cruces / WSMR area

DESIRED QUALIFICATIONS:

• RAG implementation experience (vector databases, embeddings, reranking)
• Experience with structured evaluation frameworks (RAGAS, custom LLM-as-
• judge, or equivalent)
• Familiarity with edge or offline deployment constraints
• Exposure to interview analytics, structured debriefing, structured reporting, or
• HUMINT-adjacent workflows
• Experience delivering in classified or regulated environments

CLEARANCE:

• Active Secret clearance