We9re seeking a Senior Data Engineer to join our Delivery Team. In this role, you9ll design and implement modern data architectures that enable our clients to make data-driven decisions. You9ll lead the strategy, design, and technical direction of scalable data ecosystems across cloud platforms - ensuring integration, performance, and compliance.
As a Senior Data Engineer, you9ll work closely with business stakeholders, data engineers, and analytics teams to design data solutions that align with client goals.
Responsibilities:
- Design and implement end-to-end data architectures, including data lakes, data warehouses, and analytics platforms.
- Define data integration and transformation strategies, ensuring scalability, security, and performance.
- Collaborate with stakeholders to translate business requirements into technical solutions that support analytics, reporting, and AI initiatives.
- Develop data models, ETL/ELT pipelines, and frameworks for structured and unstructured data.
- Provide technical leadership and mentorship to data engineers and developers, promoting best practices in data management and governance.
- Ensure compliance with data governance, security, and privacy standards across platforms.
- Optimize existing data architectures and processes for improved performance and reliability.
- Stay current with industry trends, cloud data services, and emerging technologies such as Databricks, Snowflake, Azure Synapse, and AWS Redshift.
- Act as a trusted advisor to clients, guiding them on architecture decisions and best practices for data modernization.
Required Skills & Experience
- 5+ years of experience in data architecture, data engineering, or analytics solution design.
- Hands-on experience with data lake and warehouse technologies (e.g., Databricks, Snowflake, Redshift, Synapse).
- Deep understanding of data modeling, data integration, and ETL/ELT design.
- Proficiency in SQL and one or more programming languages (Pyspark/Python, Scala), particularly for complex data transformations and optimization within Spark
- Solid understanding of data governance, security, and privacy best practices.
- Proven experience in designing, implementing, and optimizing large-scale ingestion pipelines using Databricks Autoloader
- Deep practical knowledge of building and managing reliable, self-managing ETL/ELT pipelines using Delta Live Tables
- Experience in building high-throughput, low-latency streaming data ingestion solutions using Apache Kafka, Spark Structured Streaming, and Databricks Streaming
- Extensive experience in successfully applying and enforcing the Medallion architecture (Bronze, Silver, Gold layers) within a Databricks environment
- Experience in designing and implementing CI/CD pipelines (using tools like Azure DevOps, GitHub Actions, GitLab CI) specifically tailored for Databricks workflows, notebooks, and cluster configurations, enabling automated deployment and testing
- Experience in planning and executing data migration projects from traditional data warehouses into the Databricks Lakehouse
- Strong working knowledge of at least one major cloud provider (AWS, Azure) regarding data storage, networking, and security concepts relevant to Databricks deployment.
- Proven ability to engage with clients, present technical solutions, and communicate complex ideas clearly.
- Bachelor9s or Master9s degree in Computer Science, Information Systems, Engineering, or a related field.
- Excellent problem-solving, communication, and collaboration skills.
Nice to Have:
- Experience with AI/ML integration and data science workflows.
- Knowledge of data cataloging and metadata management tools.
- Prior consulting or client-facing experience in a technology services firm.
