About the team and role:We are looking for a talented and passionate Data Engineer to join our team and help build scalable, reliable data solutions that power critical experiences across eBay.
In this role, you will architect, design, and develop high-performance real-time and batch data pipelines that process massive volumes of data. You will help build the infrastructure that enables real-time insights, analytics, and personalized experiences for millions of users across the eBay marketplace.
You will work with modern data technologies including Kafka, Flink, Spark, Databricks, Airflow, dbt, and AWS, solving challenging engineering problems where scalability, performance, reliability, and data accuracy are essential.
You will own projects throughout the engineering lifecycle and collaborate closely with Data Science, Product, and Engineering teams to translate complex requirements into robust data solutions.
What you will accomplish:- Build data pipelines at scale: Design, develop, and operate robust real-time and batch data pipelines using Kafka, Spark, and Flink to process large volumes of data efficiently and reliably.
- Develop reliable data workflows: Build and manage complex workflow orchestration using Airflow, ensuring data is processed accurately and available when downstream systems and teams need it.
- Improve data quality and trust: Implement and maintain automated data quality and validation frameworks using technologies such as Monte Carlo, Great Expectations, or similar tools.
- Build cloud-native data solutions: Leverage Databricks and AWS to develop scalable data platforms and services that improve developer productivity, system performance, and operational efficiency.
- Develop scalable transformations: Build maintainable data transformation workflows using dbt and establish reusable patterns for producing high-quality datasets.
- Own projects end-to-end: Drive major data engineering initiatives from architecture and design through implementation, testing, deployment, monitoring, and long-term production support.
- Solve complex scalability challenges: Identify technical risks, performance bottlenecks, and future scalability challenges and proactively recommend architectural improvements.
- Collaborate across teams: Partner with Data Science, Product, Analytics, and Engineering teams to understand data requirements, influence technical roadmaps, and deliver solutions that create meaningful customer and business impact.
- Raise the engineering bar: Advocate for continuous improvement of our data architecture, development practices, tooling, and technology stack.
- Build for long-term maintainability: Develop reusable libraries and engineering patterns while maintaining clear documentation for critical systems, pipelines, and architectural decisions.
What you will bring:- 3+ years of professional experience in software engineering or data engineering, with a focus on building data pipelines, distributed systems, or backend services.
- B.S. or M.S. in Computer Science or a related technical discipline, or equivalent practical experience.
- Strong programming experience with Python, Java, and/or Scala, supported by solid computer science fundamentals.
- Strong understanding of data structures, algorithms, concurrency, and multithreaded programming.
- Hands-on experience building distributed data processing systems using technologies such as Kafka, Spark, and Flink.
- Experience developing and operating production data workloads using Databricks.
- Hands-on experience designing and deploying solutions in AWS or a comparable cloud environment.
- Experience building and managing data workflows using Apache Airflow.
- Experience developing data transformations using dbt or similar frameworks.
- Familiarity with data observability, quality, and reliability technologies such as Monte Carlo, Great Expectations, or equivalent tools.
- Solid understanding of distributed systems architecture, relational databases, data modeling, and NoSQL technologies.
- Strong software engineering fundamentals, including object-oriented design, design patterns, testing, and production support.
- Strong problem-solving skills and the ability to identify, diagnose, and resolve complex data and distributed-system issues.
- Ability to communicate effectively and collaborate across Product, Data Science, Analytics, and Engineering organizations.
Ideally, you will also have:- Experience designing and building high-volume, low-latency streaming data platforms.
- A proven track record of architecting reusable libraries, frameworks, and common engineering patterns for large-scale applications.
- Experience optimizing distributed data workloads for performance, scalability, reliability, and cost efficiency.
- Experience working in the ecommerce, marketplace, fintech, or payments domain.
- Experience working in Agile/Scrum development environments and with tools such as JIRA.
- Advanced knowledge of object-oriented design principles, MVC architecture, and software design patterns.
- Experience influencing technical architecture and driving engineering improvements across teams.
- A strong ownership mindset and the ability to take complex technical initiatives from concept through successful production deployment.
Additional DetailsThe base pay range for this position is expected in the range below:
C$110,400 - C$147,400
Base pay offered may vary depending on multiple individualized factors, including location, skills, and experience. The total compensation package for this position may also include other elements, including a target bonus and restricted stock units (as applicable) in addition to a full range of medical, financial, and/or other benefits (including RRSP eligibility, various paid time off benefits, such as PTO and parental leave). Details of participation in these benefit plans will be provided if an employee receives an offer of employment.
This job posting relates to an existing vacancy within eBay.
