Job DescriptionThe Lead Data Engineer is a senior technical leader responsible for guiding the design, development, and optimization of Visa's large-scale data platforms and cloud-based analytics environments. This role provides architectural direction, leads complex engineering initiatives, and mentors teams while remaining deeply hands-on with modern data technologies. The Lead Data Engineer drives technical best practices, ensures platform scalability, and influences data engineering strategy for key products and business domains.
Responsibilities:
- Lead the architecture and delivery of large-scale, high-performance data pipelines and processing frameworks across Hadoop and multi-cloud environments.
- Design scalable data models, lakehouse structures, and distributed data processing solutions that support analytics, machine learning, and real-time data needs.
- Provide technical leadership to Senior and Staff Data Engineers, conducting design reviews, guiding implementation decisions, and ensuring engineering excellence.
- Partner with cross-functional teams to translate business and product requirements into robust technical designs and data solutions.
- Develop and improve engineering best practices for data governance, quality, observability, testing, and cloud resource optimization.
- Drive adoption of cloud-native data technologies, automation frameworks, and reusable components that improve development velocity and system reliability.
- Lead complex data modernization efforts, including cloud migration, data lake/lakehouse consolidation, and performance optimization of critical pipelines.
- Evaluate new tools and technologies, influencing platform evolution within the scope of assigned domains or product areas.
- Collaborate with product, analytics, and platform teams to ensure alignment on data strategy and architectural roadmaps.
- Mentor engineers at all levels, providing technical coaching and fostering a culture of continuous improvement.
This is a hybrid position. Expectation of days in the office will be confirmed by your Hiring Manager.
QualificationsBasic Qualifications
Preferred Qualifications
• Advanced expertise in building and optimizing large-scale distributed data systems using Hadoop, Spark, and modern lakehouse architectures.
• Strong programming proficiency in PySpark, Scala, and Python with experience implementing scalable, production-grade data applications.
• Deep experience designing and tuning RDBMS, NoSQL, and distributed SQL systems.
• Mastery of SQL and distributed query engines such as Presto, Trino, Hive, and SparkSQL.
• Strong knowledge of data modeling, ETL/ELT design, and data warehousing methodologies.
• Proven experience architecting and operating data solutions on AWS, GCP, and Azure, including cloud data lakes, orchestration tools, and cost-effective storage/compute designs.
Advanced proficiency in Databricks, including:
• Building and optimizing notebooks and production jobs
• Delta Lake design and optimization
• Cluster configuration and workspace administration
• CI/CD integration for data workloads
• Performance tuning for large distributed jobs
• Demonstrated ability to lead technical initiatives, communicate architectural decisions, and influence engineering direction across multiple teams.
• Strong problem-solving skills with the ability to troubleshoot complex data and performance issues.
U.S. Applicants OnlyThe estimated salary range for this position is $173,100.00 to $ 276,800.00 USD per year, which may include potential sales incentive payments (if applicable). Salary may vary depending on job-related factors which may include knowledge, skills, experience, and location. In addition, this position may be eligible for bonus and equity.Visa has a comprehensive benefits package for which this position may be eligible that includes Medical, Dental, Vision, 401(k), FSA/HSA, Life Insurance, Paid Time Off, and Wellness Program.
Work HoursVaries upon the needs of the department.
Travel RequirementsThis position requires travel 5-10% of the time.
Mental/Physical RequirementsThis position will be performed in an office setting. The position will require the incumbent to sit and stand at a desk, communicate in person and by telephone, frequently operate standard office equipment, such as telephones and computers.
