OverviewThe Crawler Team, part of Microsoft Search's Web Data organization, builds and operates Microsoft's internet-scale web crawling platform that powers Bing Search, Microsoft Copilot, indexing, content discovery, web freshness, and AI grounding experiences. The team develops highly scalable distributed systems that discover, prioritize, retrieve, and process web content at massive scale while continuously improving content quality, freshness, efficiency, and reliability. As a Principal Software Engineer, you will drive the technical vision, architecture, and execution of critical infrastructure that enables Microsoft's Search and AI platforms to understand and leverage the web.
Starting January 26, 2026, Microsoft AI (MAI) employees who live within a 50- mile commute of a designated Microsoft office in the U.S. or 25-mile commute of a non-U.S., country-specific location are expected to work from the office at least four days per week. This expectation is subject to local law and may vary by jurisdiction.
Responsibilities- Lead the architecture, design, and evolution of large-scale distributed systems responsible for web crawling, content acquisition, content processing, and retrieval at internet scale.
- Drive technical strategy and innovation in crawl scheduling, content freshness, real-time content discovery, and intelligent crawl prioritization systems.
- Build and operate highly reliable, performant, and efficient services that process billions of web crawling and content acquisition requests daily.
- Apply AI, machine learning, and data-driven approaches to improve crawl efficiency, content quality, resource utilization, and platform intelligence.
- Provide technical leadership across multiple teams and organizations, influencing architecture, engineering practices, and long-term platform direction.
- Partner closely with engineers, applied scientists, product managers, and platform teams across Search and AI to deliver shared infrastructure and end-to-end solutions.
- Lead efforts to improve platform scalability, reliability, observability, operational excellence, and developer productivity.
- Mentor senior engineers and drive engineering excellence through technical guidance, design reviews, and cross-team collaboration.
QualificationsRequired Qualifications:- Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
- OR equivalent experience.
- Experience designing, building, and operating scalable distributed services at scale.
Preferred Qualifications:- Master's Degree in Computer Science or related technical field AND 8+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
- OR Bachelor's Degree in Computer Science or related technical field AND 12+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python
- OR equivalent experience.
#MicrosoftAI #Search #Copilot
Software Engineering IC5 - The typical base pay range for this role across the U.S. is USD $142,800 - $274,800 per year. There is a different range applicable to specific work locations, within the San Francisco Bay area and New York City metropolitan area, and the base pay range for this role in those locations is USD $188,000 - $304,200 per year.
Certain roles may be eligible for benefits and other compensation. Find additional benefits and pay information here:
https://careers.microsoft.com/us/en/us-corporate-pay
This position will be open for a minimum of 5 days, with applications accepted on an ongoing basis until the position is filled.
