Posted today
Unspecified
Mid Level Career (5+ yrs experience)
$100,000 - $125,000
No Traveling
IT - Database
Blu Omega is looking for a highly skilled and experienced Cloud Data Engineer to join our growing data team. The ideal candidate will play a key role in designing and implementing scalable, secure, and efficient data solutions to support our analytics and business intelligence initiatives. You will collaborate with cross-functional teams, including data scientists, analysts, and software engineers, to develop robust data pipelines, optimize data storage, and ensure the integrity and accessibility of data across the organization.
Essential Functions
• Data Pipeline Development: Design and build scalable pipelines to transform large datasets into BI-ready formats using Python, SQL, and Microsoft Azure services (e.g. Azure Blob Storage, Azure Data Factory, Azure Functions, Azure Database for PostgreSQL, Azure DevOps, Azure Databricks).
• Data Architecture: Develop and implement best practices for data architecture, including designing schema models, managing data storage, and optimizing data flow within cloud environments.
• Automation: Leverage Azure Data Factory and cloud-based integration platform (e.g. SnapLogic) to orchestrate and automate data workows, ensuring efficient and reliable data processing.
• Containerization: Utilize Docker for containerizing applications and data services, ensuring consistency and ease of deployment across environments.
• Data Quality & Governance: Ensure data quality, standardization, integrity, and governance standards are met by establishing validation frameworks, file formats, monitoring solutions, and enforcing security protocols.
• Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data needs and develop solutions that drive business insights.
• Visualization: Partner with analytics and BI teams to create and support data visualization solutions using tools like PowerBI, making data accessible and actionable for key stakeholders.
• Optimization: Continuously improve system performance, optimize queries, and refine data structures to reduce processing time and enhance scalability.
Experience:
• 3-5 years of experience in data engineering, with a focus on building scalable cloud data pipelines and architectures.
• Strong programming skills in Python and advanced proficiency in SQL for data manipulation, query optimization, and pipeline development.
• Proven experience working with Azure Cloud, including services like Azure Blob Storage, Azure Data Factory, Azure Functions, Azure Database for PostgreSQL, Azure DevOps, Azure Databricks
• Experience with GCP is highly preferred, as many environments and tools are GCP-native. AWS experience is also a plus.
• Strong hands-on experience with relational databases (PostgreSQL and/or SQL Server required). NoSQL experience is a plus… as well as data warehousing concepts.
• Experience with data visualization tools such as PowerBI or similar platforms.
• Solid understanding of ETL/ELT processes, data modeling, and building data lakes/warehouses.
• Experience in CI/CD tools (Azure DevOps) and version control (Git, GitLab, Jenkins).
• Strong problem-solving and communication skills, with the ability to translate business requirements into technical solutions.
• Familiarity with data science practices including feature engineering, feature selection, model training using linear, nonlinear, deep learning frameworks, model scoring, model automation, automated retraining preferred.
• Familiarity with machine learning platforms and integration of model outputs into data pipelines.
Preferred:
• Familiarity in healthcare or health-tech industry working with healthcare data standards and regulations (e.g., HIPAA) is a plus.
• Hands-on experience with Docker for containerization and deployment of data applications preferred.
• Hands on exposure to IaC (Infrastructure as Code) frameworks like Terraform preferred.
• Familiarity with SnapLogic preferred.
Education:
• Bachelor’s Degree or higher, technical or STEM discipline preferred.
Salary- $100,000-$125,000
Essential Functions
• Data Pipeline Development: Design and build scalable pipelines to transform large datasets into BI-ready formats using Python, SQL, and Microsoft Azure services (e.g. Azure Blob Storage, Azure Data Factory, Azure Functions, Azure Database for PostgreSQL, Azure DevOps, Azure Databricks).
• Data Architecture: Develop and implement best practices for data architecture, including designing schema models, managing data storage, and optimizing data flow within cloud environments.
• Automation: Leverage Azure Data Factory and cloud-based integration platform (e.g. SnapLogic) to orchestrate and automate data workows, ensuring efficient and reliable data processing.
• Containerization: Utilize Docker for containerizing applications and data services, ensuring consistency and ease of deployment across environments.
• Data Quality & Governance: Ensure data quality, standardization, integrity, and governance standards are met by establishing validation frameworks, file formats, monitoring solutions, and enforcing security protocols.
• Collaboration: Work closely with data scientists, analysts, and other stakeholders to understand data needs and develop solutions that drive business insights.
• Visualization: Partner with analytics and BI teams to create and support data visualization solutions using tools like PowerBI, making data accessible and actionable for key stakeholders.
• Optimization: Continuously improve system performance, optimize queries, and refine data structures to reduce processing time and enhance scalability.
Experience:
• 3-5 years of experience in data engineering, with a focus on building scalable cloud data pipelines and architectures.
• Strong programming skills in Python and advanced proficiency in SQL for data manipulation, query optimization, and pipeline development.
• Proven experience working with Azure Cloud, including services like Azure Blob Storage, Azure Data Factory, Azure Functions, Azure Database for PostgreSQL, Azure DevOps, Azure Databricks
• Experience with GCP is highly preferred, as many environments and tools are GCP-native. AWS experience is also a plus.
• Strong hands-on experience with relational databases (PostgreSQL and/or SQL Server required). NoSQL experience is a plus… as well as data warehousing concepts.
• Experience with data visualization tools such as PowerBI or similar platforms.
• Solid understanding of ETL/ELT processes, data modeling, and building data lakes/warehouses.
• Experience in CI/CD tools (Azure DevOps) and version control (Git, GitLab, Jenkins).
• Strong problem-solving and communication skills, with the ability to translate business requirements into technical solutions.
• Familiarity with data science practices including feature engineering, feature selection, model training using linear, nonlinear, deep learning frameworks, model scoring, model automation, automated retraining preferred.
• Familiarity with machine learning platforms and integration of model outputs into data pipelines.
Preferred:
• Familiarity in healthcare or health-tech industry working with healthcare data standards and regulations (e.g., HIPAA) is a plus.
• Hands-on experience with Docker for containerization and deployment of data applications preferred.
• Hands on exposure to IaC (Infrastructure as Code) frameworks like Terraform preferred.
• Familiarity with SnapLogic preferred.
Education:
• Bachelor’s Degree or higher, technical or STEM discipline preferred.
Salary- $100,000-$125,000
group id: 91121246
