Job Requirements
Remote
Secret Polygraph not specified
Mid Level Career (5+ yrs experience)
Salary not specified
Join Premium to unlock estimated salaries 
Job Description
Data Engineer
Position Summary
We are seeking a Data Engineer to support a secure federal data and analytics environment. The ideal candidate will have hands-on experience building, managing, and optimizing large-scale data pipelines using Databricks, cloud data services, and modern data engineering practices.
This role will support the ingestion, transformation, validation, and delivery of complex financial and operational datasets for analytics, reporting, reconciliation, and decision-support use cases. The candidate should be comfortable working with large data volumes, structured and semi-structured data, data quality frameworks, and secure cloud-based platforms.
Experience supporting Department of Defense or federal financial management data environments is strongly preferred.
Clearance Requirement
Interim Secret clearance or higher required
U.S. Citizenship required
Key Responsibilities
Design, build, manage, and optimize data pipelines in Databricks and similar enterprise data platforms.
Develop scalable ETL/ELT processes to ingest, cleanse, transform, aggregate, and publish data for downstream analytics and reporting.
Work with large-scale federal financial management, accounting, and general ledger datasets.
Support data reconciliation, transaction-level validation, and Universe of Transactions analysis.
Implement data pipelines using Databricks capabilities such as Delta Lake, Autoloader, workflows, notebooks, and job scheduling.
Support structured data architecture patterns, including Bronze, Silver, and Gold data layers.
Develop automated data quality checks, validation routines, exception handling, and reconciliation logic.
Use Python, PySpark, Pandas, and SQL to perform data transformation, data wrangling, and performance optimization.
Work with AWS GovCloud services, including S3 or similar cloud storage services.
Support data governance, lineage, access control, and cataloging activities; Unity Catalog experience is a plus.
Troubleshoot pipeline failures, performance bottlenecks, data quality issues, and source-to-target discrepancies.
Collaborate with data analysts, product owners, architects, and stakeholders to understand data requirements and deliver reliable data products.
Participate in Agile delivery activities, code reviews, version control, testing, and deployment support.
Maintain technical documentation for data pipelines, data mappings, transformation logic, validation rules, and operational procedures.
Required Qualifications
3+ years of experience in data engineering, ETL/ELT development, data integration, or large-scale data platform support.
Demonstrated experience building and optimizing data pipelines in Databricks or comparable big data platforms.
Hands-on experience with Delta Lake, data ingestion, data transformation, and scheduled data jobs.
Advanced Python experience, including PySpark and Pandas.
Expert-level SQL skills for complex queries, joins, transformations, aggregations, and data validation.
Experience designing and implementing ETL/ELT processes for large structured datasets.
Experience implementing data quality frameworks, automated validation checks, and reconciliation processes.
Familiarity with cloud data environments, preferably AWS GovCloud.
Experience working with cloud storage services such as Amazon S3.
Familiarity with Medallion architecture, including Bronze, Silver, and Gold data layers.
Strong understanding of CI/CD principles and version control tools such as GitLab.
Strong analytical, troubleshooting, and problem-solving skills.
Strong communication skills and ability to collaborate with technical teams, analysts, product owners, and stakeholders.
Desired Qualifications
Experience with Advana, the War Data Platform, or similar DoD enterprise analytics platforms.
Experience with federal financial management data, general ledger data, accounting transactions, obligations, disbursements, budget execution, or audit-readiness datasets.
Knowledge of Universe of Transactions reconciliation concepts.
Experience with Unity Catalog or similar data governance/cataloging capabilities.
Knowledge of SAP data structures is a significant plus.
Experience supporting DoD, federal financial management, audit, logistics, or ERP data environments.
Experience with data lineage, metadata management, data access controls, and governed data sharing.
Experience supporting analytics, dashboards, operational reporting, or decision-support products.
Position Summary
We are seeking a Data Engineer to support a secure federal data and analytics environment. The ideal candidate will have hands-on experience building, managing, and optimizing large-scale data pipelines using Databricks, cloud data services, and modern data engineering practices.
This role will support the ingestion, transformation, validation, and delivery of complex financial and operational datasets for analytics, reporting, reconciliation, and decision-support use cases. The candidate should be comfortable working with large data volumes, structured and semi-structured data, data quality frameworks, and secure cloud-based platforms.
Experience supporting Department of Defense or federal financial management data environments is strongly preferred.
Clearance Requirement
Interim Secret clearance or higher required
U.S. Citizenship required
Key Responsibilities
Design, build, manage, and optimize data pipelines in Databricks and similar enterprise data platforms.
Develop scalable ETL/ELT processes to ingest, cleanse, transform, aggregate, and publish data for downstream analytics and reporting.
Work with large-scale federal financial management, accounting, and general ledger datasets.
Support data reconciliation, transaction-level validation, and Universe of Transactions analysis.
Implement data pipelines using Databricks capabilities such as Delta Lake, Autoloader, workflows, notebooks, and job scheduling.
Support structured data architecture patterns, including Bronze, Silver, and Gold data layers.
Develop automated data quality checks, validation routines, exception handling, and reconciliation logic.
Use Python, PySpark, Pandas, and SQL to perform data transformation, data wrangling, and performance optimization.
Work with AWS GovCloud services, including S3 or similar cloud storage services.
Support data governance, lineage, access control, and cataloging activities; Unity Catalog experience is a plus.
Troubleshoot pipeline failures, performance bottlenecks, data quality issues, and source-to-target discrepancies.
Collaborate with data analysts, product owners, architects, and stakeholders to understand data requirements and deliver reliable data products.
Participate in Agile delivery activities, code reviews, version control, testing, and deployment support.
Maintain technical documentation for data pipelines, data mappings, transformation logic, validation rules, and operational procedures.
Required Qualifications
3+ years of experience in data engineering, ETL/ELT development, data integration, or large-scale data platform support.
Demonstrated experience building and optimizing data pipelines in Databricks or comparable big data platforms.
Hands-on experience with Delta Lake, data ingestion, data transformation, and scheduled data jobs.
Advanced Python experience, including PySpark and Pandas.
Expert-level SQL skills for complex queries, joins, transformations, aggregations, and data validation.
Experience designing and implementing ETL/ELT processes for large structured datasets.
Experience implementing data quality frameworks, automated validation checks, and reconciliation processes.
Familiarity with cloud data environments, preferably AWS GovCloud.
Experience working with cloud storage services such as Amazon S3.
Familiarity with Medallion architecture, including Bronze, Silver, and Gold data layers.
Strong understanding of CI/CD principles and version control tools such as GitLab.
Strong analytical, troubleshooting, and problem-solving skills.
Strong communication skills and ability to collaborate with technical teams, analysts, product owners, and stakeholders.
Desired Qualifications
Experience with Advana, the War Data Platform, or similar DoD enterprise analytics platforms.
Experience with federal financial management data, general ledger data, accounting transactions, obligations, disbursements, budget execution, or audit-readiness datasets.
Knowledge of Universe of Transactions reconciliation concepts.
Experience with Unity Catalog or similar data governance/cataloging capabilities.
Knowledge of SAP data structures is a significant plus.
Experience supporting DoD, federal financial management, audit, logistics, or ERP data environments.
Experience with data lineage, metadata management, data access controls, and governed data sharing.
Experience supporting analytics, dashboards, operational reporting, or decision-support products.
group id: 10207042