Bachelor's degree in Computer Engineering, Electrical Engineering, Computer Science, or related field required.
Three years of experience in XPU or AI interconnect system architecture and micro architecture.
Strong understanding of computer architecture, including GPU, NPU, and CPU design principles.
Familiarity with large language model architectures and distributed training/inference concepts.
Demonstrated analytical ability through published research or quantitative project work.
Strong cross-functional leadership and communication skills, with experience collaborating with compiler, software, SOC, and backend teams.
Responsibilities
Model the memory and interconnection architecture linking XPUs to memory across node, rack, and cluster boundaries.
Solve communication and topology bottlenecks during distributed LLM training and inference.
Define XPUs' utilization of memory and interconnect fabric to ensure efficient workload mapping.
Collaborate with silicon, memory, networking, and software teams to specify interconnect fabric requirements.
Apply hands-on NPU hardware architecture judgment to interconnect and system design decisions.
Monitor leading open LLM architectures to anticipate impacts on interconnect and system topology.
Benefits
Discretionary bonus eligibility.
Equity options available.
Full range of medical benefits.
Full Job Description
Job description
Team Credo is looking for a Senior AI System Architect to join our team in San Jose, CA, reporting to AVP, XPU system AI interface. This role needs to solve the system-level interconnection challenges that connect XPUs (NPU/GPU/custom silicon) into working AI infrastructure. This role needs to follow and define how compute elements are wired, networked, and orchestrated together at node, rack, and cluster scale so that large-scale LLM training and inference run efficiently across the platform. We strongly prefer candidates with a hands-on NPU hardware architect background and real experience taking NPU silicon into production deployment at scale.
Base salary range is $160,000 - $220,000 a year. The base salary offer will depend on factors such as education, experience, training, skills, qualifications, and location. This position is also eligible for a discretionary bonus, equity and a full range of medical and other benefits.
Qualifications
Basic Qualifications
Bachelor's degree in Computer Engineering, EE, CS, or related field required.
Three years in XPU or AI interconnect system architecture and micro architecture working experience.
Strong foundational understanding of computer architecture, including GPU, NPU, and/or CPU design principles.
Familiarity with large language model architectures and distributed training/inference concepts (e.g., parallelism strategies, model serving) through coursework, research, or personal projects.
Demonstrated analytical ability - for example, through published research, thesis work, or quantitative project work - with a track record of independently studying and synthesizing technical material.
Strong cross-functional leadership and communication skills, daily experience working with compiler, software, SOC & backend team preferred.
Preferred Qualifications
Master's or PhD in Computer Science, Artificial Intelligence, Engineering, or a related field.
Real, hands-on NPU architecture background, with direct experience participating in an NPU design from architecture definition through silicon bring-up, validation, and volume production deployment.
Working knowledge of high-speed interconnect or networking concepts (e.g., PCIe, Ethernet, RDMA) is a plus.
Job responsibilities
Model the memory and interconnection architecture linking XPUs to memory and across node, rack, and cluster boundaries - not the internal design of the NPUs/GPUs/CPUs themselves.
Solve communication and topology bottlenecks that arise from all kinds of parallelism strategies during distributed LLM training and inference.
Define how XPUs utilize the memory and interconnect fabric, ensuring workloads map cleanly onto the network topology.
Work with silicon, memory, networking, and software teams to specify interconnect fabric requirements and resolve integration issues between compute nodes.
Bring hands-on NPU hardware architecture judgment to interconnect and system design decisions, informed by direct experience taking NPU silicon from definition through bring-up, validation, and production deployment.
Track leading open LLM architectures to anticipate how model structure will stress interconnect and system topology.
