Atlassian

Senior Machine Learning Engineer, AI-Native Evaluation & Search

Atlassian$206K — $269K *
US-Anywhere
+ 3 other locationsRemote
Information Technology
Less than 5 years of experience
Job Overview by Ladders

Qualifications

  • Bachelor’s or Master’s in Computer Science, Machine Learning, Data Science, or equivalent experience
  • 4+ years of experience developing and deploying production machine learning or AI systems
  • Strong programming skills in Python and familiarity with Java, Kotlin, or TypeScript
  • Experience with SQL and large-scale data processing using technologies like Spark
  • Proven track record in building, evaluating, deploying, and scaling models with large datasets
  • Experience in defining and analyzing evaluation strategies for model improvement
  • Familiarity with cloud-based ML platforms such as AWS or Databricks

Responsibilities

  • Develop and deploy machine learning models and AI systems for search and intelligence
  • Own projects through the entire ML lifecycle from definition to continuous improvement
  • Create AI-native experiences involving retrieval, ranking, and conversational systems
  • Design evaluation-driven development processes with metrics and benchmarks
  • Build datasets and pipelines for AI systems including search and chat
  • Conduct various types of evaluation to analyze model behavior
  • Mentor other engineers and contribute to best engineering practices

Benefits

  • Comprehensive health and wellbeing resources
  • Paid volunteer days
  • Opportunities to engage with local community
  • Flexible work arrangements
  • Professional development programs
Full Job Description
Overview

Atlassian is looking for a Senior Machine Learning Engineer to join our Search & Intelligence organization. We build AI-native experiences, agentic systems, models, evaluation frameworks, and data platforms that power Atlassian’s AI products.

Responsibilities

Your role and impact

You will develop and productionize machine learning systems for search, retrieval, ranking, conversational experiences, and AI agents. You will own projects across the ML lifecycle, from problem definition and data development through experimentation, evaluation, deployment, and continuous improvement.

A key part of this role is making evaluation a first-class part of AI development. You will help define quality, build evaluation datasets and benchmarks, analyze model behavior, and use results to guide product and engineering decisions.

You will collaborate with product managers, software engineers, data scientists, research scientists, and platform teams. You will independently solve complex problems, contribute to technical direction, and mentor other engineers.

What you’ll do

  • Build and productionize machine learning models and AI systems for Search & Intelligence products.

  • Own projects from concept through deployment, monitoring, evaluation, and iteration.

  • Develop AI-native experiences across search, retrieval, ranking, recommendations, conversational systems, and agentic workflows.

  • Design evaluation-driven development processes, including quality metrics, test cases, benchmarks, and acceptance criteria.

  • Build evaluation datasets, regression suites, and pipelines for search, RAG, chat, and agentic systems.

  • Conduct offline and online evaluations, human assessments, model-based evaluations, and error analysis.

  • Design scalable architectures that meet requirements for quality, reliability, latency, privacy, and cost.

  • Apply modern techniques in information retrieval, ranking, embeddings, NLP, deep learning, and large language models.

  • Communicate technical decisions and results clearly across technical and non-technical audiences.

  • Mentor engineers and contribute to engineering and ML best practices.

Compensation

At Atlassian, we strive to design equitable, explainable, and competitive compensation programs. We follow consistent hiring practices and account for each candidate's skills, knowledge, and experience when setting base pay within the range.

Please visit go.atlassian.com/payzones for more information on which locations are included in each of our geographic pay zones. However, please confirm the zone for your specific location with your recruiter.

This role may also be eligible for benefits, bonuses, commissions, and equity.

Pay Ranges

In The United States, we have three geographic pay zones. For this role, our current base pay ranges for new hires in each zone are:

Zone A: $206,100 - $269,075

Zone B: $185,490 - $242,168

Zone C: $171,063 - $223,332

Qualifications

On your first day, we’ll expect you to have

  • A Bachelor’s or Master’s degree in Computer Science, Machine Learning, Data Science, or a related field, or equivalent experience.

  • At least 4 years of experience developing and deploying machine learning or AI systems in production.

  • Strong Python skills and experience with Java, Kotlin, TypeScript, or another production language.

  • Experience with SQL and large-scale data processing technologies such as Spark.

  • Experience building, evaluating, deploying, and scaling models with large datasets.

  • Experience designing evaluation strategies, analyzing model quality, and using results to guide improvements.

  • Familiarity with cloud-based ML and data platforms such as AWS or Databricks.

  • Ability to independently solve ambiguous problems and deliver practical, production-quality solutions.

  • Strong communication and collaboration skills.

  • An agile mindset and commitment to continuous improvement.

It’s great, but not required, if you have

  • Experience building AI-native products, RAG systems, conversational assistants, or tool-using agents.

  • Experience with evaluation-driven development, benchmarks, regression testing, human evaluation, or LLM-as-a-judge.

  • Experience evaluating agent planning, tool use, task completion, or multi-step reasoning.

  • Experience with search relevance, ranking, recommendations, personalization, embeddings, or NLP.

  • Experience fine-tuning, post-training, evaluating, or optimizing large language models.

  • Experience with ML platforms, model serving, data pipelines, observability, or responsible AI.

  • Experience mentoring engineers or influencing technical direction within a team.

Benefits & Perks

Atlassian offers a wide range of perks and benefits designed to support you, your family and to help you engage with your local community. Our offerings include health and wellbeing resources, paid volunteer days, and so much more. To learn more, visit go.atlassian.com/perksandbenefits.

About Atlassian

Atlassian is a leading provider of collaboration, development, and issue tracking software for teams. With over 194,000 customers worldwide, including 85 of the Fortune 100, Atlassian is changing the way teams work. Our products help teams organize, discuss, and complete shared work. Atlassian has a unique business model that allows us to deliver software to teams of all sizes, from small startups to large enterprises. Our products are available on a subscription basis, with no upfront fees or long-term commitments. Atlassian was founded in 2002 and is headquartered in San Francisco, California.
Learn more about Atlassian
Size
6,433 employees
Market Cap
$31.9 billion
Industry
Net Income
-$1.1 billion
Founded
2002
5 Year Trend
+34.9%
Revenue
$1.8 billion
NASDAQ

Similar Jobs

More Jobs at Atlassian

More Information Technology Jobs

Find similar Senior Machine Learning Engineer, AI-Native Evaluation & Search jobs: