Job Summary
The Data Engineer will support data transformation and modernization initiatives focused on transitioning legacy platforms to modern cloud-enabled platforms. The role requires strong hands-on experience solving complex data engineering problems using Databricks, AWS, Python, PySpark, and SQL. The ideal candidate will also have experience with Snowflake and cloud-based, microservices-oriented platforms.

Key Responsibilities
• Create, manage, and operate tools, processes, technologies, and best practices that enable the development and ongoing operation of modern cloud-based platforms.
• Develop horizontal tools, technologies, and best practices that support engineering teams in building, debugging, testing, releasing, managing, and securing modern microservices-based cloud infrastructure.
• Collaborate with engineering teams to define methodologies and ways of working that support efficient software and data platform development.
• Promote and implement best practices and supporting technologies for source control management, CI/CD, performance, SLAs, security, audit, and monitoring.
• Define common frameworks and software libraries that support efficient, scalable, and highly available architectures.
• Support the management of running platform architectures by troubleshooting issues and identifying opportunities for continuous improvement in quality, performance, and security.
• Work closely with architects, technical product managers, and engineering teams to translate system architecture and product requirements into well-designed, implemented, and tested solutions.
• Develop and optimize data transformation solutions using Databricks, Python, and PySpark.
• Leverage AWS services including S3, Lambda, EC2, DynamoDB, RDS, API Gateway, and Fargate to support cloud-based solutions.
• Apply data engineering best practices across distributed systems, cloud platforms, and modern data architectures.
• Work in an Agile environment to deliver high-quality solutions incrementally.
• Promote adoption of best-in-class engineering practices, frameworks, and tools while providing technical guidance to other developers.

Required Qualifications
• 8+ years of hands-on software or data engineering experience focused on cloud-based systems.
• Strong hands-on experience with Databricks.
• Strong hands-on experience with AWS cloud services, including S3 and Lambda.
• Strong Python development skills.
• Strong PySpark experience.
• Strong SQL skills and experience working with data transformation and processing.
• Experience solving complex data engineering problems and supporting data modernization initiatives.
• Experience developing cloud-based services and platforms.
• Experience developing microservices and complex microservice-based architectures.
• Strong understanding of distributed systems and designing for scalability, performance, and availability.
• Experience with relational databases and NoSQL data persistence technologies.
• Experience with cloud security, reliability, monitoring, and audit practices.
• Knowledge of distributed network architectures and network security.
• Strong technical communication and collaboration skills.
• 3+ years of experience working in teams using modern Agile software development practices.
• Bachelor's degree in Computer Science, Engineering, or a related field.

Preferred Qualifications
• Strong hands-on experience with Snowflake.
• Experience with Java, preferably Java 8 or later.
• Experience with Golang or JavaScript/Node.js.
• Experience with AWS services such as EC2, DynamoDB, RDS, API Gateway, and Fargate.
• Experience with cloud-based SaaS and PaaS platforms.
• Experience with CI/CD architectures, source control management, performance engineering, and platform monitoring.
• Master's degree in Computer Science, Engineering, or a related field.
• Experience with data transformation and migration from legacy platforms to modern cloud architectures.