Qualifications

• 8+ years of hands-on experience with Databricks and Apache Spark (PySpark, Scala, or SQL)
• 5+ years of proficiency in at least one programming language (Python, Scala, or Java)
• 5+ years of strong understanding of cloud data architectures (Azure Data Lake, AWS S3, GCP Storage)
• 5+ years of experience with CI/CD tools and DevOps practices for data engineering
• 5+ years of familiarity with data warehousing concepts and tools (Delta Lake, SQL, etc.)
• 5+ years of solid grasp of distributed computing, performance tuning, and big data best practices
• 5+ years of excellent communication and problem-solving skills

Responsibilities

• Design and implement data ingestion pipelines from multiple sources
• Optimize data processing workflows for performance
• Collaborate with cross-functional teams to ensure data accessibility and quality
• Utilize CI/CD tools to streamline data engineering processes
• Leverage cloud data architectures to support scalable data solutions
• Troubleshoot and resolve issues related to data ingestion and processing
• Advise on best practices for big data management and performance tuning

Benefits

• Flexible work arrangements
• Continuous learning and training opportunities
• Access to cutting-edge technologies
• Collaborative team environment
• Supportive company culture enhancing work-life balance