TTG-376 - Big Data & ETL Engineer (PySpark, Python, SQL) - $348,

TTG Solutions is seeking an experienced Data Engineer to support the development and enhancement of data systems and applications in diverse cloud computing environments. We require a professional who is adept in managing and expanding data processing capabilities and is proficient in leveraging cloud-based analytics. The selected candidate will support the development of new system capabilities, maintain and enhance data pipelines, and improve applications within cloud environments. Key responsibilities include data mapping, extraction, transformation, and loading (ETL), analytic report building using tools like CloudWatch and Kibana, and processing OS and Data Logs into dashboards. The role involves regular interaction with customers and integration partners to clarify objectives and contribute to Agile development processes by defining tasks, scoping, and reviewing.

Subject Matter Expert Normally requires at least sixteen (16) years of experience specific to the skill or occupation. For specific computer languages, systems or technologies, expertise with the subject matter or technology outweighs the number of years of experience.

Requirements: 16 yrs+

• Experience with big data processing technologies such as Spark, Pyspark, and Python
• Proficiency in data mapping, extraction, transformation, and loading (ETL)
• Experience building analytic reports with tools like CloudWatch and Kibana
• Ability to process and convert OS and Data Logs into reports and metrics dashboards
• Proficiency with Regular Expressions (RegEx)
• Experience with SQL, MySQL, PostgreSQL
• Proficiency in processing data file types such as XML and JSON
• Experience with IDEs and Data Modeling using tools like Notebooks and Visual Studio
• Experience performing data quality analysis and ETL documentation
• Ability to interface with customers and integration partners
• Support Agile development through task definition, scoping, and review

• Experience deploying on the Databricks unified analytics platform
• Experience with Elastic Map Reduce for big data workloads
• Proficiency in joining complex data sets using Spark
• Experience in tuning Spark streaming and batch jobs
• Proficiency in deploying complex, notebook-based pipelines
• Experience with Python data analysis libraries like Pandas
• Familiarity with cloud services such as Lambda, SNS/SQS, or EC2
• Experience with DevOps tools including Cloudwatch, Lambda, SQS, Dynamo, and RDS
• Experience with the ElasticSearch/Logstash/Kibana (ELK stack)

This position requires an active TS/SCI clearance with Polygraph. Total compensation is the total combination of salary, benefits, retirement, bonuses, etc... and it is calculated using the following formula:

Total Compensation = Base Salary + Benefits Budget + Retirement + Bonus + Training Budget + Phone Stipend

Although each company has their own blend of salary and benefits, comparing total compensation is typcially the best way to compare the financial impact of each position. At TTG Solutions, we work hard to ensure our particular blend always meets or exceeds the market average.

Total Compensation = $348,150