Posted today
Secret
Mid Level Career (5+ yrs experience)
Unspecified
Engineering - Systems
Lead Data Engineer
Cloud & Big Data Platforms
Location: 100% Remote (open to travel as needed)
Position Overview
We are seeking an experienced Lead Data Engineer to design, build, and scale modern cloud-based data platforms that power advanced analytics and business decision-making. This role is ideal for a hands-on technical leader who excels in AWS, big data technologies, and distributed data processing, while mentoring teams and driving best practices across the data engineering lifecycle.
Key Responsibilities
Design, provision, and manage cloud resources in AWS.
Build and maintain data ingestion pipelines from diverse sources, including:
Relational databases (RDBMS)
REST/HTTP APIs
Flat files
Streaming platforms
Time-series data from proprietary systems
Implement scalable data ingestion and processing using big data technologies.
Perform data processing and transformation using Spark and cloud-native services.
Translate business logic into efficient, production-ready data pipelines using platform-supported languages.
Develop and maintain automated data quality and validation checks.
Architect and support infrastructure to collect, transform, combine, and distribute customer data.
Identify and drive process improvement opportunities to optimize data collection, analytics, and reporting.
Ensure data solutions are scalable, performant, accurate, complete, and accessible.
Analyze complex datasets to identify trends, patterns, and actionable insights.
Build frameworks leveraging data visualization tools to present consolidated insights to business stakeholders.
Actively participate in Agile Scrum ceremonies and collaborate with cross-functional teams.
Develop complex queries, reports, and presentations for both technical and non-technical audiences.
Mentor junior engineers and promote industry best practices in data engineering.
Required Technical Experience
7+ years of experience as a Data Engineer, preferably in consumer finance or a comparable industry (e.g., loans, collections, servicing, insurance, financial products).
Strong academic or professional background in mathematics, statistics, computer science, data science, or a related field.
Advanced proficiency in at least one of the following programming languages:
Java
Scala
Python
C#
Proven production experience with:
HDFS, YARN, Hive, Spark
Kafka
Oozie or Airflow
Amazon Web Services (AWS)
Docker / Kubernetes
Snowflake
Proficiency with:
Data mining and programming tools (SAS, SQL, R, Python)
Database technologies (PostgreSQL, Redshift, Snowflake, Greenplum)
Data visualization platforms (Tableau, Looker, MicroStrategy)
Strong organizational skills with the ability to manage multiple priorities and deadlines.
Excellent written and verbal communication skills, including the ability to present insights to non-technical audiences.
Solid understanding of business intelligence and analytics tools, technologies, and techniques.
Demonstrated ability to learn and deploy new technologies and tools quickly.
Preferred / Additional Experience
AWS Certification
Spark Streaming
Kafka Streaming / Kafka Connect
ELK Stack
NoSQL databases (Cassandra, MongoDB)
CI/CD pipelines and tools:
Jenkins
GitLab
Jira
Confluence
Education
Bachelor’s degree required in:
Computer Science
Data Science
Software Engineering
Mathematics
Or a related technical discipline
Why This Role
Redhawk is advancing a critical Navy enterprise platform designed to modernize data enablement through API-led infrastructure. This role places you at the forefront of government digital transformation, enabling scalable, interoperable solutions that directly support Department of the Navy operations.
Cloud & Big Data Platforms
Location: 100% Remote (open to travel as needed)
Position Overview
We are seeking an experienced Lead Data Engineer to design, build, and scale modern cloud-based data platforms that power advanced analytics and business decision-making. This role is ideal for a hands-on technical leader who excels in AWS, big data technologies, and distributed data processing, while mentoring teams and driving best practices across the data engineering lifecycle.
Key Responsibilities
Design, provision, and manage cloud resources in AWS.
Build and maintain data ingestion pipelines from diverse sources, including:
Relational databases (RDBMS)
REST/HTTP APIs
Flat files
Streaming platforms
Time-series data from proprietary systems
Implement scalable data ingestion and processing using big data technologies.
Perform data processing and transformation using Spark and cloud-native services.
Translate business logic into efficient, production-ready data pipelines using platform-supported languages.
Develop and maintain automated data quality and validation checks.
Architect and support infrastructure to collect, transform, combine, and distribute customer data.
Identify and drive process improvement opportunities to optimize data collection, analytics, and reporting.
Ensure data solutions are scalable, performant, accurate, complete, and accessible.
Analyze complex datasets to identify trends, patterns, and actionable insights.
Build frameworks leveraging data visualization tools to present consolidated insights to business stakeholders.
Actively participate in Agile Scrum ceremonies and collaborate with cross-functional teams.
Develop complex queries, reports, and presentations for both technical and non-technical audiences.
Mentor junior engineers and promote industry best practices in data engineering.
Required Technical Experience
7+ years of experience as a Data Engineer, preferably in consumer finance or a comparable industry (e.g., loans, collections, servicing, insurance, financial products).
Strong academic or professional background in mathematics, statistics, computer science, data science, or a related field.
Advanced proficiency in at least one of the following programming languages:
Java
Scala
Python
C#
Proven production experience with:
HDFS, YARN, Hive, Spark
Kafka
Oozie or Airflow
Amazon Web Services (AWS)
Docker / Kubernetes
Snowflake
Proficiency with:
Data mining and programming tools (SAS, SQL, R, Python)
Database technologies (PostgreSQL, Redshift, Snowflake, Greenplum)
Data visualization platforms (Tableau, Looker, MicroStrategy)
Strong organizational skills with the ability to manage multiple priorities and deadlines.
Excellent written and verbal communication skills, including the ability to present insights to non-technical audiences.
Solid understanding of business intelligence and analytics tools, technologies, and techniques.
Demonstrated ability to learn and deploy new technologies and tools quickly.
Preferred / Additional Experience
AWS Certification
Spark Streaming
Kafka Streaming / Kafka Connect
ELK Stack
NoSQL databases (Cassandra, MongoDB)
CI/CD pipelines and tools:
Jenkins
GitLab
Jira
Confluence
Education
Bachelor’s degree required in:
Computer Science
Data Science
Software Engineering
Mathematics
Or a related technical discipline
Why This Role
Redhawk is advancing a critical Navy enterprise platform designed to modernize data enablement through API-led infrastructure. This role places you at the forefront of government digital transformation, enabling scalable, interoperable solutions that directly support Department of the Navy operations.
group id: 10176392
