Software Engineer II

Overview

Join a dynamic team dedicated to delivering generative AI at a global scale by developing and managing cloud services that handle extensive AI model inferences. You will engage in solving complex problems arising from distributed systems, performance, and high-capacity challenges at hyperscale.

Job Purpose

As a Software Engineer, your focus will be creating robust backend services that manage high-volume, asynchronous AI workloads. You'll design intelligent traffic-routing mechanisms to optimize compute utilization, gaining hands-on experience with large-scale scheduling, distributed systems, and performance tuning. Collaborate with a talented team in a flexible hybrid setup, with up to half of your work done remotely.

Responsibilities

• Design, develop, and maintain scalable backend systems to efficiently handle asynchronous AI inference workloads at scale.
• Create smart scheduling and routing features that optimize the utilization of compute resources based on load awareness.
• Enhance system throughput, reduce latency, and improve cost-effectiveness by profiling, tuning, and thoughtful system architecture.
• Implement comprehensive telemetry, monitoring, and alerting frameworks; actively participate in on-call duties to uphold service availability.
• Collaborate closely with product and platform teams to integrate new AI models and support expanding enterprise deployments.
• Produce clean, maintainable, and testable code; engage constructively in design discussions and code reviews.
• Drive continuous improvements in reliability, scalability, and overall engineering excellence.

Qualifications

• Required: Bachelor's degree in Computer Science or related field plus minimum three years of engineering experience or commensurate equivalent.
• At least two years of experience developing backend or distributed services using modern programming languages such as C#, Java, Go, or C++.
• Minimum one year experience with public cloud platforms like Azure, AWS, or GCP in building and managing services.
• Preferred: Over a year working with large-scale distributed systems, batch or asynchronous data processing workloads.
• Experience with systems handling scheduling, queuing, load balancing, or request routing.
• Proficiency with observability tools and managing production services, including participating in on-call rotations.
• Familiar with containerization and orchestration tools such as Docker and Kubernetes.

Other Requirements

The role requires meeting Microsoft's security and government screening protocols, including passing the Microsoft Cloud Background Check upon hiring and every two years thereafter.

Equal Opportunity and Application Notes

Microsoft is committed to equal employment opportunities regardless of diverse personal and demographic factors. Accommodations are available for applicants needing assistance during the application process.