Software Engineer - Spark Runtime in San Francisco, CA

View All Enterprise Technology jobs


Enterprise Technology   •  

5 - 7 years

Posted 7 weeks ago


As an engineer at Databricks, you are responsible for developing the future of Internet-scale distributed production pipelines. You will be empowered to take a customer-first approach to solving problems. Our users range from small companies doing periodic analysis to Fortune 500 companies creating just-in-time data warehouses to do real-time machine learning and artificial intelligence on web-scale data. Your team designs, plans, implements and maintains the platform they build upon. This demands a high degree of critical thinking, customer focused design, and problem solving. We are looking for engineers who will bring revolutionary ideas to an area that is largely silo'ed to data engineering, data science, and machine learning teams in large organizations. You will help us change that, freeing everyone to analyze and extract value from their data.


  • Build the core features and technologies that make Databricks Runtime the world's best Spark execution environment in terms of performance, stability, security and scalability.
  • Write code for Databricks Runtime and provide feedback to the pull requests of other team members.
  • Work both as an individual and in a team to deliver features and architectural improvements.
  • Work directly with Product Management and Design to plan, develop and iterate on prototypes.
  • Full ownership including: Designing, Implementing, Testing and Metric Analysis.


  • Excellent problem analysis and solving skills.
  • Excellent communication and teamwork.
  • Strong foundation in algorithms and data structures and their real-world use cases.
  • Solid understanding of computer systems and networks.
  • Production quality coding standards and patterns.
  • Passion for developing amazing customer experiences.
  • 4+ years of general software programming experience.
  • 4+ years of modern, production level experience in one of: Java, Scala, JavaScript, or C++.
  • BS in Computer Science, Math, related technical field or equivalent practical experience.
  • Experience with: Large scale distributed computing, Database internals, Big Data engines e.g. Spark, Hadoop


  • Medical, dental, vision
  • 401k Retirement Plan
  • Unlimited Paid Time Off
  • Catered lunch (everyday), snacks, and drinks
  • Gym reimbursement
  • Employee referral bonus program
  • Awesome coworkers
  • Maternity and paternity plans.