OverviewLead Data Engineers at Excella drive the architecture, development, and optimization of data solutions, ensuring they align with business goals and technical best practices. They take ownership of large-scale data initiatives, mentor team members, and provide strategic input on data platform decisions. Lead Data Engineers oversee technical delivery, ensuring data solutions are robust, scalable, and maintainable.
Responsibilities - Lead the design, development, and optimization of scalable data solutions.
- Define data architecture strategies and ensure best practices are followed.
- Oversee and guide the creation and automation of data pipelines and platforms.
- Establish and enforce data quality and governance frameworks.
- Collaborate with Architects, Product Owner, Data Scientists, and DevOps to align data solutions with business needs.
- Research and evaluate emerging data technologies and methodologies.
- Ensure seamless integration of data management solutions into client environments.
- Develop risk mitigation strategies and implement data recovery plans.
- Lead the development of data repositories, including data warehouses, data lakes, and operational data stores.
- Mentor and support the development of junior and senior team members.
Qualifications - 6+ years of experience in data engineering and related technical domains.
- Designed and built scalable, extensible data pipelines for production environments with strong expertise in SQL, Python, and PySpark.
- 2+ years of hands-on experience with Databricks, including development, data processing, and pipeline optimization in a cloud-based environment
- Hands-on experience with big data and distributed systems, Parquet, Delta Lake, HDInsight
- Extensive work with cloud-native data platforms and services, such as Azure Data Factory, Synapse, Azure Storage Accounts, Streaming Analytics, Azure Fabric
- Deep understanding of data lake architecture, secure and performant data store design, and analytical platform patterns.
- Proficient in orchestration tools and CI/CD workflows, infrastructure-as-code (e.g., Terraform, Azure DevOps Pipelines), and observability best practices.
- Advanced Git and Azure DevOps Repos expertise and strong familiarity with version control, branching strategies, and collaborative development.
- Leadership experience mentoring engineers and guiding cross-functional technical teams.
- Excellent communicator, capable of translating complex technical ideas for both technical and non-technical audiences.
- Agile practitioner with experience leading and delivering within Scrum/Kanban frameworks, and a continuous learning mindset.
- Technically savvy, entrepreneurial spirit who thrives in environments that reward self-initiative and resourcefulness.
Preferred Qualifications- Advocating for adopting industry tools and practices at the right time
- Appreciate the importance of schema design and can evolve an analytics schema on top of unstructured data.
- Leadership and mentoring of team members on project and within Excella.
- Excited to try out new technologies and produce proofs-of-concept that balance technical advancement and user experience.
- Empathetic working with stakeholders, listening to them, asking the right questions, and collaboratively developing the best solutions for their needs.
- Champion for data privacy and integrity, and always act in the best interest of clients.
