Data Engineer - Hybrid

Zachary Piper Solutions is seeking a Data Engineer HYBRID in Elkridge, MD

Responsibilities of the Data Engineer include:

• Serve as a Databricks subject matter expert, leveraging its full suite of tools to build, deploy, and manage scalable data and AI solutions.
• Design, develop, and maintain ETL pipelines using Databricks, Spark, Delta Lake, Delta Live Tables, and Databricks Workflows.
• Transform and ingest data from external systems using protocols such as HTML and SFTP.
• Collaborate with developers and external teams to validate data ingestion and integration processes.
• Enhance the ETL platform with reusable components and automation to reduce integration timelines.
• Ensure high data quality, performance, and reliability across large-scale datasets.
• Provide detailed documentation of system architecture, development processes, and enhancements.

Qualifications for the Data Engineer include:

• Proven experience with Databricks for data engineering and integration.
• Strong proficiency in Spark, Python, and SQL.
• Hands-on experience with Delta Lake, Delta Live Tables, and Databricks Workflows.
• Familiarity with cloud platforms such as AWS, Azure, etc.
• TS/SCI with CI Polygraph

Compensation for the Data Engineer include:

· Salary: $150,000-$210,000 **Depending on Experience and Education Level**

· Comprehensive Benefits: Medical, Dental, Vision, 401K, PTO, Sick Leave as required by law, and Holidays

This job opens for applications on 6/30/25. Applications for this job will be accepted for at least 30 days from the posting date

Keywords: Databricks, Apache Spark, Delta Lake, Delta Live Tables, Databricks Workflows, ETL Pipelines, Data Integration, Data Engineering, Data Transformation, Big Data, Python, SQL, SFTP, HTML, Cloud Platforms, AWS, Azure, IC ITE, TS/SCI, CI Polygraph, Clearance Required, Secure Data Ingestion, Government Data Systems, Advana, Databricks Runtime, Databricks SQL, Databricks Notebooks, Databricks Repos, Git, Version Control, Collaboration, Documentation, System Architecture, Cross-functional Teams, Data Validation, Agile Development, Feature Enhancement, Automation