Posted today
Secret
Senior Level Career (10+ yrs experience)
Unspecified
Occasional travel
IT - Data Science
We are seeking a Data Engineer to join a high-impact team building mission-critical products deployed to U.S. submarines and maritime systems. This role is for someone who obsesses over performance, technical polish, and the correctness of data pipelines in complex, real-world environments.
Primary Responsibilities
• Architect real-time and offline pipelines using MQTT and Redpanda for stream processing, complemented by Dagster for robust batch workflows.
• Engineer high-performance data warehouses leveraging Postgres, Apache Iceberg, Parquet, and S3 to support high-velocity queries.
• Parse and process complex binary message formats from diverse sensor sources, ensuring data is validated, normalized, and resilient.
• Optimize the development lifecycle by improving local environments and CI/CD pipelines using modern tooling and GitHub Actions.
• Design sophisticated data models tailored for time-series analysis, including windowing, resampling, and interpolation.
• Collaborate and mentor within a remote-first team, breaking down complex technical concepts to drive collective engineering excellence.
Required Experience
• Active Security Clearance.
• 6+ years of Software/Data Engineering experience with a fanatical focus on code quality, linting, and 100% test coverage.
• Proficiency in Python and deep experience with both OLTP and OLAP databases.
• Hands-on experience with Redpanda/Kafka and orchestrators like Dagster or Airflow.
• Proven ability to build scalable architectures and handle binary message formats like Protobuf.
• Demonstrated experience optimizing query speeds and system performance for mission-critical downstream consumers.
Preferred Experience
• Experience with IoT devices, sensors, or digital signal processing (DSP).
• Background in geospatial analysis (GIS) and working within monorepo environments.
• Familiarity with maritime or defense-related data environments.
Primary Responsibilities
• Architect real-time and offline pipelines using MQTT and Redpanda for stream processing, complemented by Dagster for robust batch workflows.
• Engineer high-performance data warehouses leveraging Postgres, Apache Iceberg, Parquet, and S3 to support high-velocity queries.
• Parse and process complex binary message formats from diverse sensor sources, ensuring data is validated, normalized, and resilient.
• Optimize the development lifecycle by improving local environments and CI/CD pipelines using modern tooling and GitHub Actions.
• Design sophisticated data models tailored for time-series analysis, including windowing, resampling, and interpolation.
• Collaborate and mentor within a remote-first team, breaking down complex technical concepts to drive collective engineering excellence.
Required Experience
• Active Security Clearance.
• 6+ years of Software/Data Engineering experience with a fanatical focus on code quality, linting, and 100% test coverage.
• Proficiency in Python and deep experience with both OLTP and OLAP databases.
• Hands-on experience with Redpanda/Kafka and orchestrators like Dagster or Airflow.
• Proven ability to build scalable architectures and handle binary message formats like Protobuf.
• Demonstrated experience optimizing query speeds and system performance for mission-critical downstream consumers.
Preferred Experience
• Experience with IoT devices, sensors, or digital signal processing (DSP).
• Background in geospatial analysis (GIS) and working within monorepo environments.
• Familiarity with maritime or defense-related data environments.
group id: 91165268
