WHO WE ARE LOOKING FOR:
We are looking for an experienced Data Engineer to join our Solutions Architecture team.
- You have a sustained track record of making significant, self-directed, and end-to-end contributions to large and impactful software projects, particularly in the data engineering space. You think beyond just the task at hand to deeply understand the 'why' behind what you are doing.
- You understand engineering fundamentals. At our scale, many off-the-shelf techniques, and existing technologies (open source and enterprise) simply don't work. You can work from first principles to evaluate solutions and adapt them to a unique environment.
- You are a broadly skilled engineer, accustomed to developing web services. You have experience building always-on systems, working across a variety of technologies and service layers.
- You have a product-focused mindset. You have the passion and ability to contribute to the process of discovering what will delight our clients and push forward one of the world’s largest and most influential industries toward a vision of openness, transparency, and evidence-based decision-making.
- You work with confidence and without ego. Our engineers have deep knowledge and exercise a high degree of leadership in their daily work. You have strongly-held, defensible ideas, and advocate for what you believe is right. You are also adept at identifying and evaluating trade-offs, willing to be proven wrong, and quick to walk through fire to support your fellow teammates.
- You value, seek out, and foster diversity. We are a global team from many diverse backgrounds, with different experiences and perspectives. To complement this team, you will welcome ideas that are different from your own and be skilled at finding and building from common ground.
- You are a creative thinker, not bound by "the way things have always been done". What you know is less important than how well you learn and innovate. We don't need engineers who know all the answers; we need engineers who can invent the answers no one has thought of yet, to the questions yet to be asked.
ABOUT THE ROLE:
- You will be an end-to-end owner. You will interact with some of our largest clients to understand their requirements and business goals. Based on their business goals, you will design and build large scale data processing systems, data pipelines, computationally intensive statistical modelling and advanced analytics solutions that incorporate AI or ML models proposed by our data scientists.
- You will create data tools for data science team members that assist them in building and optimizing those models in the first place. You will build or extend the pipelines required for assembly, optimal extraction, transformation, and loading of large data sets from a wide variety of data sources using SQL and AWS ‘big data’ technologies with an understanding of how they will be used in the models.
- You will identify, design, and implement improvements for existing big-data applications: automating manual processes, optimizing data delivery, re-designing for greater scalability and reliability, etc.
- You will monitor and manage custom solutions, debug application related issues causing solution outage or degradation, and implement necessary fixes.
- You will manage client projects where you will plan tasks, set timelines, manage and update stakeholders, resolve blockers and drive the process forward.
- You will help in building up our team through a combination of mentoring, technical leadership, making others better by raising the bar.
WHAT YOU BRING TO THE TABLE:
* 7+ years of work experience as a Data engineer in a SaaS or technology services company
* 5+ years of engineering experience using big data technologies such as Spark, Hadoop, etc., and building large scale data processing and analysis infrastructure using AWS services like EMR, Kinesis, and Athena
* Deep knowledge of querying and extracting information from data warehouses like Vertica, Snowflake and relational databases like SQL Server, Postgres
* Advanced coding expertise across one or more of Python, C#, Spark, or Scala.
* Understanding of data privacy principles
* Experience in deploying infrastructure as code using tools like CloudFormation, Terraform, etc. is a plus
* Knowledge of multiple integration approaches such as file-based, web services, and API
* Demonstrable creative thinking and problem-solving abilities, a passion for problem solving and learning new technologies quickly
* AWS Certification including specialisations (such as Data Analytics, Machine Learning, etc.) is a plus
* Experience with a formal project management method is a plus
* Experience working with clients, product, and engineering teams in a pre-sales role or consultative experience is a plus
* Knowledge of multiple cloud platforms (AWS, GCP, Azure) is a plus
* Prior experience in implementing machine learning or AI systems is a plus