AI Hardware / Memory Systems ArchitectReports To: VP, Engineering
Position Summary:This is a senior technical architecture role for an individual who understands how AI workloads translate into memory, bandwidth, latency, compute, interconnect, power, and system-level requirements.
The successful candidate will work closely with the Chief AI Systems Architect, AI Product Strategist, AI Inference / Systems Software Architect, and R&D engineering teams to translate customer problems and AI workload requirements into innovative hardware and memory architectures.
We are particularly interested in investigating opportunities involving:
- Evolution of memory hierarchy as AI models and context sizes grow
- Type of data that will reside in HBM, DDR, CXL memory, storage
- Moving computation closer to memory
- Reduce energy and cost of moving data
- Increase effective memory capacity without proportionally increasing system cost and power
- New memory or hardware architecture that can provide a meaningful advantage over today's solutions
The role is for someone who can design the memory and hardware architecture to solve the problem with the best performance, power, capacity, and cost.
Key Responsibilities:AI Hardware and Memory Architecture- Develop system and hardware architectures for next-generation AI and inference platforms
- Analyze AI workloads and translate workload characteristics into memory hierarchy, bandwidth, latency, capacity, and compute requirements
- Architect solutions spanning HBM, DDR, CXL memory, memory pooling and disaggregation, memory controllers, cache hierarchy, AI accelerators, near-memory computing, processing-in-memory and storage and persistent memory
- Develop architectural alternatives and conduct quantitative trade-off analysis
- Define performance, power, capacity, bandwidth, latency, area, and cost targets
AI Workload Characterization- Develop a deep understanding of the hardware implications of LLM inference, generative AI, agentic AI, long-context inference, KV-cache growth and management, retrieval-augmented generation (RAG), AI data movement and edge inference
- Analyze workload bottlenecks and determine whether they are primarily caused by compute, memory capacity, memory bandwidth, latency, interconnect and software
- Use workload characterization to identify architectural opportunities
- Investigate and architect new approaches to memory hierarchy and utilization, including HBM/DRAM tiering, CXL memory expansion, CXL memory pooling, memory disaggregation, intelligent caching, data placement, prefetching, memory compression, KV-cache management, model-weight storage, persistent and tiered memory and shared memory architectures
Memory Hierarchy InnovationNear-Memory and In-Memory Computing- Investigate opportunities to reduce data movement by moving computation closer to memory. Potential areas include near-memory acceleration, processing-in-memory (PIM), memory-side computation, compression/decompression near memory, encryption/security processing, attention/KV-cache operations, embedding/vector operations and sparse computation
- Determine which workloads can benefit sufficiently to justify dedicated hardware
Hardware / Software Co-Design- Work closely with the AI Inference / Systems Software Architect to ensure that architectural features and hardware capabilities can be effectively exposed and utilized by the software stack. Collaborate on AI runtime architecture, memory management, KV-cache management, scheduling, data placement, prefetching, compression, workload orchestration, APIs and software interfaces and hardware telemetry and monitoring
- Ensure that hardware innovations translate into measurable application-level benefits rather than isolated component-level benchmarks
Architecture Modeling and Prototyping- Build or direct architectural models and simulations to evaluate proposed architectures
- Establish workload-driven benchmarks for AI training and inference
- Quantify performance, power, capacity, bandwidth, latency, and TCO improvements
- Develop proof-of-concept implementations and prototypes
- Work with R&D teams to move promising architectures from concept to silicon or system prototype
- Define technical specifications and validation methodologies
Product Definition and Technical Roadmap- Work with the AI Product Strategist and Chief AI System Architect to translate customer problems into product architectures. Contribute to Product requirements, System specifications, Architecture specifications, Performance targets, Technical differentiation, Product roadmap, Build-versus-buy decisions and Technology readiness assessment
- Provide technical leadership from initial concept through product definition and development
- Work with product strategy and executive management to do technical due diligence and evaluate emerging technologies, startup companies, intellectual property, licensing opportunities, strategic investments and acquisition candidates
Strategic Technology Evaluation and M&A- Work with product strategy and executive management to do technical due diligence and evaluate emerging technologies, startup companies, intellectual property, licensing opportunities, strategic investments and acquisition candidates
Required Qualifications:- Bachelor's or Master's degree in Electrical Engineering, Computer Engineering, Computer Science, or a related field; PhD is a plus
- 10+ years of experience in computer architecture, memory systems, semiconductor design, AI hardware, or related areas
- Deep understanding of modern computer architecture and memory hierarchy
- Strong experience with one or more of HBM, DDR, CXL, PCIe, UCIe, High-speed interconnect and AI accelerator architectures
- Strong understanding of AI/ML workloads and their architectural implications
- Experience performing quantitative architecture trade-off analysis
- Experience translating system requirements into hardware architecture and specifications
- Demonstrated experience contributing to or leading the development of complex semiconductor or computing systems
- Strong understanding of performance, power, area, bandwidth, latency, and cost trade-offs
- Excellent communication and technical leadership skills
Preferred Qualifications:Experience with several of the following:
- LLM inference architecture
- KV-cache systems
- Transformer architectures
- CUDA / GPU architecture
- vLLM or other inference runtimes
- Memory compression
- CXL 2.x / 3.x or later
- CXL switching and pooling
- Memory disaggregation
- Processing-in-memory
- Near-memory computing
- AI networking
- Storage-class memory
- ASIC / SoC architecture
- RTL / microarchitecture
- Architectural simulation and performance modeling
- Hardware/software co-design
- Edge AI systems
Salary Range: $285,000 
