LLM Penetration Tester

Job Description

Job Title: LLM Penetration Tester
Location: DMV Area
Clearance: TS/SCI w Full Scope Poly
Employment Type: Full Time

Education: Bachelor's in Cybersecurity, Information Security, or a related field
Work Status: DMV Area
Salary: $140,000-$180,000

Benefits: Competitive salary and bonus structure, comprehensive health insurance, 401(k) with company match, generous PTO and flexible work options.

Application: Apply here or on our Careers Page @ Careers - Metronome, or email your resume to Careers@wearemetronome.com

Overview:
Metronome is seeking a highly skilled and creative LLM Penetration Tester to join our AI Red Team. This role focuses on evaluating the security posture and behavioral integrity of large language models (LLMs) through the design and execution of advanced linguistic adversarial attacks. You'll be responsible for developing complex prompt-based attacks that test the limits of LLM guardrails and detect vulnerabilities such as prompt injection, jailbreaking, and data leakage.

This is an ideal position for individuals with a red team mindset, deep linguistic awareness, and hands-on experience with LLM platforms. You will work at the intersection of cybersecurity and AI safety to uncover critical insights into how modern language models behave under adversarial pressure.

Key Responsibilities:

• Design and Execute Adversarial Attacks: Create and implement advanced linguistic adversarial attacks, including context-shifting prompts, semantic alterations, and nuanced language manipulations, specifically designed to bypass LLM security filters and elicit unintended or malicious responses.
• Question-Answer Pair Generation: Develop comprehensive sets of question-answer pairs that effectively demonstrate an LLM's vulnerability or resilience to various attack vectors.
• Red-Teaming LLMs: Actively engage in red-teaming exercises, attempting to "trick" LLMs into following instructions or commands that they are designed to resist, thereby testing their guardrails and security boundaries.
• Behavioral Analysis: Observe and analyze LLM behavior under attack conditions, understanding how different linguistic manipulations impact their responses and security posture.

Required Skills

Required Skills:

• 5+ years of experience in threat intelligence, penetration testing, or incident response
• LLM Red-Teaming Expertise: Demonstrated professional, project, or personal experience in red-teaming large language models, including a strong understanding of common LLM vulnerabilities (e.g., prompt injection, jailbreaking, data leakage, hallucination, bias).
• Advanced Linguistic Proficiency: Exceptional understanding of language nuances, semantics, syntax, context, and pragmatics, with the ability to creatively manipulate these elements to craft sophisticated prompts.
  
  • Adversarial Mindset: A strong "attacker" mentality, capable of thinking creatively to identify and exploit potential weaknesses in AI systems.
  • Familiarity with LLM Platforms: Hands-on experience working with and understanding the APIs and behaviors of major LLM providers and models.

Certifications:

• CISSP - Certified Information Systems Security Professional
• GCIH - GIAC Certified Incident Handler

Desired Skills

Preferred Experience

Specific Applications, Technologies, and Services:

• LLM Platforms: Direct experience with models from OpenAI (e.g., GPT-3.5, GPT-4), Anthropic (e.g., Claude), Google (e.g., PaLM, Gemini), and open-source models.

• Security Concepts: Understanding of adversarial machine learning (Adversarial ML), prompt injection techniques, jailbreaking methodologies, and general cybersecurity principles as they apply to AI.

• Experience in cybersecurity, penetration testing, or ethical hacking, particularly with a focus on AI/ML systems.

• Research experience in adversarial AI, LLM security, or AI safety.

• Contributions to open-source projects related to LLM security or red-teaming.

About Metronome, LLC



Metronome is a spark that ignites passion for our customers and each other. We offer professional services to Government customers that keep projects pulsing. We believe in taking care of people so they can put their best foot forward to support the mission.



Background Screening/Check/Investigation: Successful Completion of a Background Check will be required as a condition of hire.



Benefits: Metronome offers a comprehensive benefits package that reflects our commitment to creating a diverse and supportive workplace. Benefit eligibility is determined on the type of position (full-time, part-time, temporary). Metronome's range of benefits include, but are not limited to, Medical, Vision & Dental Insurance, Life Insurance, Paid Time-Off & Company Paid Holidays, Personal Development & Learning Opportunities.



Application Process: Please follow all instructions carefully. Errors or omissions may affect your consideration for employment.



Select Create Profile and Apply to Requirement
Complete your Profile and Answer the Questionnaire
Upload a current resume
Complete the Online Application and Submit



Evaluation Process: Metronome will evaluate applicants based on how well they meet the qualifications of the position above. Your completed application (including questionnaire, resume, and online application) will be used to determine your eligibility and how well you meet the qualifications for this position.



Your responses to the questionnaire may be compared to your resume and application; if either your resume or application contradicts or does not support your responses, you will disqualify yourself and not receive further consideration for this job.



Equal Employment Opportunity Policy



Metronome does not discriminate in employment on the basis of race, color, religion, sex (including pregnancy and gender identity), national origin, political affiliation, sexual orientation, marital status, disability, genetic information, age, membership in an employee organization, retaliation, parental status, military service, or other non-merit factor.



Reasonable Accommodation Policy



Metronome is committed to providing reasonable accommodations to applicants with disabilities where appropriate. A reasonable accommodation is any change to a job, the work environment, or the way things are usually done that enables an individual with a disability to apply for a job, perform job duties or receive equal access to job benefits.



Applicants requiring reasonable accommodation for any part of the application process or hiring process should contact Metronome Human Resources at hr@wearemetronome.com or 703-957-4082. Determinations on requests for reasonable accommodation will be made on a case-by-case basis.