Job Requirements
Tysons, VA
Intel Agency (NSA, CIA, FBI, etc) Full Scope Polygraph
Early Career (2+ yrs experience)
$180,000 - $260,000
Job Description
About This Role
We're seeking a Spark Data Engineer to join a funded program supporting the Sponsor in Tysons, VA. You'll build and manage data pipelines that ingest, process, and validate data at scale, using Python and Apache Spark on AWS. The work spans writing Spark jobs, querying relational databases, handling data across formats such as CSV, XML, JSON, Avro, and Parquet, and scripting in Linux, all backed by solid unit testing. Experience orchestrating pipelines with NiFi or Airflow, tuning Spark performance, or administering EMR clusters will help you stand out. This is a hands-on engineering seat for someone who enjoys making data flow reliably from source to analyst.
Compensation
$180,000 - $260,000 (depending on experience)
Security Clearance Required
TS/SCI w/ Full Scope Poly
Key Responsibilities
Design, build, and manage data pipelines that ingest, transform, and deliver data for mission use
Develop and maintain Spark jobs in Python to process large datasets on AWS
Write and optimize SQL queries against relational databases such as PostgreSQL, Oracle, and MySQL
Parse, analyze, and convert data across formats including CSV, XML, JSON, Avro, and Parquet
Write Linux shell scripts to automate pipeline and system tasks
Write and validate unit tests to keep pipeline code reliable
Orchestrate pipeline workflows with tools such as NiFi or Apache Airflow
Tune Spark job performance and support EMR/Spark cluster administration
Contribute to cloud-based security solutions for data processing environments
Follow the program's configuration management process to review and deploy code as part of releases
Requirements
Active TS/SCI clearance with Full Scope Polygraph
Demonstrated experience building and managing data pipelines
Demonstrated experience with Python
Demonstrated experience with cloud computing using AWS services
Demonstrated experience processing data using Apache Spark
Demonstrated experience with an RDBMS (PostgreSQL, Oracle, or MySQL) and writing SQL queries
Demonstrated experience with Linux and shell scripting
Demonstrated experience analyzing data in different file formats such as CSV, XML, JSON, Avro, and Parquet
Demonstrated experience writing and validating unit tests
Preferred Qualifications
Experience with NiFi, Apache Airflow, or an equivalent tool for orchestrating data pipelines
Experience with Java or Scala
Experience administering an EMR/Spark cluster
Experience conducting performance tuning of Spark jobs
Experience developing cloud-based security solutions
Experience following a configuration management process to review and deploy code as part of releases
Prior experience supporting the Sponsor or other IC customers
Bachelor's degree in Computer Science, Data Engineering, or a related field (or equivalent experience)
What We Offer
Competitive salary: $180,000 - $260,000 (based on experience)
6% 401(k) match (immediate eligibility and vesting) + 4% annual profit sharing
Comprehensive health insurance
25 days PTO + 11 federal holidays
Mission-critical work supporting national security
Onsite at client location in Tysons, VA
Small company environment with significant growth potential
We're seeking a Spark Data Engineer to join a funded program supporting the Sponsor in Tysons, VA. You'll build and manage data pipelines that ingest, process, and validate data at scale, using Python and Apache Spark on AWS. The work spans writing Spark jobs, querying relational databases, handling data across formats such as CSV, XML, JSON, Avro, and Parquet, and scripting in Linux, all backed by solid unit testing. Experience orchestrating pipelines with NiFi or Airflow, tuning Spark performance, or administering EMR clusters will help you stand out. This is a hands-on engineering seat for someone who enjoys making data flow reliably from source to analyst.
Compensation
$180,000 - $260,000 (depending on experience)
Security Clearance Required
TS/SCI w/ Full Scope Poly
Key Responsibilities
Design, build, and manage data pipelines that ingest, transform, and deliver data for mission use
Develop and maintain Spark jobs in Python to process large datasets on AWS
Write and optimize SQL queries against relational databases such as PostgreSQL, Oracle, and MySQL
Parse, analyze, and convert data across formats including CSV, XML, JSON, Avro, and Parquet
Write Linux shell scripts to automate pipeline and system tasks
Write and validate unit tests to keep pipeline code reliable
Orchestrate pipeline workflows with tools such as NiFi or Apache Airflow
Tune Spark job performance and support EMR/Spark cluster administration
Contribute to cloud-based security solutions for data processing environments
Follow the program's configuration management process to review and deploy code as part of releases
Requirements
Active TS/SCI clearance with Full Scope Polygraph
Demonstrated experience building and managing data pipelines
Demonstrated experience with Python
Demonstrated experience with cloud computing using AWS services
Demonstrated experience processing data using Apache Spark
Demonstrated experience with an RDBMS (PostgreSQL, Oracle, or MySQL) and writing SQL queries
Demonstrated experience with Linux and shell scripting
Demonstrated experience analyzing data in different file formats such as CSV, XML, JSON, Avro, and Parquet
Demonstrated experience writing and validating unit tests
Preferred Qualifications
Experience with NiFi, Apache Airflow, or an equivalent tool for orchestrating data pipelines
Experience with Java or Scala
Experience administering an EMR/Spark cluster
Experience conducting performance tuning of Spark jobs
Experience developing cloud-based security solutions
Experience following a configuration management process to review and deploy code as part of releases
Prior experience supporting the Sponsor or other IC customers
Bachelor's degree in Computer Science, Data Engineering, or a related field (or equivalent experience)
What We Offer
Competitive salary: $180,000 - $260,000 (based on experience)
6% 401(k) match (immediate eligibility and vesting) + 4% annual profit sharing
Comprehensive health insurance
25 days PTO + 11 federal holidays
Mission-critical work supporting national security
Onsite at client location in Tysons, VA
Small company environment with significant growth potential
group id: 91172547