Job Requirements
Washington, DC
Public Trust Polygraph None
Career Level not specified
Salary not specified
Join Premium to unlock estimated salaries 
Job Description
Our client is currently seeking a Data Automation Engineer
In this role, you will:
Required Qualifications:
Preferred Qualifications:
In this role, you will:
- Design and implement scalable data automation workflows using AWS services, with integration to selected Azure data platforms where required.
- Develop ETL/ELT processes to ingest, transform, and move data across Amazon DynamoDB, SQL Server hosted on AWS, Azure SQL, and other enterprise data sources.
- Design, develop, and support batch and near-real-time ingestion pipelines using Apache Spark and technologies such as Kafka or Flume, and collaborate with the search engineering team to integrate those pipelines with the existing Apache Solr platform.
- Evaluate and apply Generative AI services and frameworks, such as Amazon Bedrock, Azure OpenAI, Hugging Face, and LangChain, to:
- Prototype and evaluate selected GenAI-assisted capabilities, such as metadata enrichment, data-quality analysis, structured data extraction, anomaly identification, and natural-language access to enterprise data. Recommend suitable use cases for future implementation.
- Develop scalable data-processing solutions using Amazon EMR and containerized deployment environments such as AWS Fargate or Kubernetes.
- Integrate Amazon Connect customer-interaction data into analytical data stores for operational reporting and analytics.
- Apply source-control, build, containerization, and CI/CD practices using tools such as GitHub, Azure DevOps, Jenkins, and Docker.
- Implement data solutions in accordance with established security and compliance controls, including identity and access management, KMS encryption, VPC isolation, role-based access control, and firewall policies.
- Support Agile DevOps processes with sprint-based delivery of pipeline and AI-enabled features.
Required Qualifications:
- Bachelor's degree in Computer Science or a related field and 5+ years of experience in data engineering, data automation, or a related discipline.
- Candidates must be able to independently design, develop, test, and troubleshoot Python- and SQL-based data pipelines in AWS environments, including integrations with Azure services where required, and clearly explain their personal contribution to production implementations.
- Strong hands-on experience with Apache Spark and working knowledge of at least one streaming or ingestion technology, such as Apache Kafka or Apache Flume.
- Hands-on experience with multiple AWS data and integration services, including several of the following: Amazon S3, AWS Glue, AWS Lambda, Amazon EMR, AWS Step Functions, and at least one AWS database service.
- Practical experience integrating at least one LLM platform or model service, such as Amazon Bedrock, Azure OpenAI Service, or an open-source model, into a Python-based workflow.
- Experience integrating REST APIs and external services into Python-based data pipelines and automated workflows.
- Experience using Jira and one or more source-control, build, or CI/CD platforms, such as GitHub, Azure DevOps, or Jenkins.
- Strong troubleshooting and performance-optimization skills across SQL, Spark, batch pipelines, and near-real-time ingestion workflows.
- Experience supporting production data platforms, including SLA monitoring, incident resolution, root-cause analysis, data reconciliation, performance troubleshooting, vulnerability remediation, and recurring maintenance.
- Good communication and presentation skills.
- US Citizenship and ability to obtain Federal government Public Trust clearance.
Preferred Qualifications:
- Relevant certifications, such as AWS Certified Data Engineer - Associate, AWS Certified Machine Learning - Specialty, Microsoft Certified: Azure AI Engineer Associate, or Databricks Certified Data Engineer.
- Familiarity with retrieval-augmented generation pipelines, embeddings, and vector-search technologies such as Apache Solr, Amazon OpenSearch Service, pgvector, or similar platforms.
- Experience with multi-cloud data integration (AWS and Azure).
- Experience with Docker and Kubernetes for containerized deployment, scalable data processing, and orchestration.
- Experience operationalizing Generative AI workflows, including prompt and model configuration, evaluation, observability, monitoring, and lifecycle management.
- Knowledge of data lineage/governance tools (Purview, Unity Catalog, AWS Glue Catalog).
- Familiarity with infrastructure-as-code tools, such as Terraform, AWS CloudFormation, or Bicep, for automated deployments.
- Experience with compliance frameworks (FedRAMP, PCI-DSS, HIPAA).
Medical, dental, and vision insurance are available to qualified candidates who meet eligibility requirements.
group id: cxjudgpa
