AWS/Databricks Cloud Data Engineer

We are seeking a Senior Cloud Data Engineer to join our consulting team. In this role, you will design, build, and maintain scalable data pipelines and acquisition workflows across cloud and on-premises environments, contributing to the full spectrum of data engineering - from ingestion through transformation and delivery. You will collaborate with cross-functional teams to translate business requirements into robust data solutions, while upholding best practices in architecture, cloud infrastructure, documentation, and continuous improvement. This position sits within our genomics organization, providing senior-level, strategic technical leadership alongside two existing engineers to build out AWS-based data pipelines that support genomic data science models - hands-on, production AWS pipeline-building experience is the top priority for this role. Role Responsibilities • Participate in team stand-ups and other agile ceremonies to support project delivery • Assist in building and maintaining data acquisition pipelines from diverse sources including databases, flat-files, and REST APIs • Leverage development design patterns and best practices across workloads. • Support data modeling efforts including dimensional models and semantic layers for analytics under senior guidance • Proactively troubleshoot and resolve pipeline failures, query performance issues, data quality concerns, notebook timeouts, data model inconsistencies, semantic model refresh errors, and capacity throttling using monitoring tools and query optimization techniques • Participate in requirement gathering meetings and translate business needs into technical specifications • Engage in ongoing learning through online trainings, certifications, and mentorship opportunities • Create and maintain technical documentation, operational documentation, data flow diagrams, and data mapping documents • Collaborate effectively with cross-functional teams and communicate progress to technical and non-technical stakeholders • Build and maintain AWS-native pipelines (Lambda, ECS, EKS, S3) that ingest and store genomic data retrieved via API calls • Generate and manage sequencing and post-sequencing data processing workflows in support of genomics data science models Requirements • Top priority: proven, hands-on experience designing and building AWS-based data pipelines end-to-end (API ingestion through storage and delivery) • Must-have: hands-on experience working with High Performance Computing (HPC) clusters • Experience with dimensional modeling, SCDs, and medallion architecture • Previous cloud experience required; AWS preferred with hands-on exposure to Lambda, S3, and Docker • Experience implementing and managing role-based access control (RBAC), including group/permissions management for secure system access • Strong Python experience across pipeline development, ETL, and data engineering tasks, with exposure to complex datasets from disparate batch and streaming sources and an emphasis on low latency, high throughput, and cost-efficiency; pandas experience a plus • Leveraging SQL and python data tooling develop scalable, optimized ETL/ELT pipelines to ingest, transform, and load large, complex datasets from disparate sources (batch and streaming). Streamlining for low latency, high throughput and cost-efficiency. • Experience with API development in both consumer and producer capacities (REST APIs) • CI/CD experience with modern DevOps tooling and deployment pipelines • Experience with API-based access patterns for retrieving and integrating data from external systems • Lambda functions must be developed and deployed via code/infrastructure-as-code (e.g., CloudFormation, Terraform, CDK, SAM) - GUI/console-based Lambda creation is not acceptable • Hands-on experience with Amazon ECS and/or EKS for container orchestration • Demonstrated ability to learn new technologies quickly and apply them to solve problems • Strong attention to detail and commitment to producing high-quality work • Excellent oral and written communication skills • Professionalism and ethics including trust, positive attitude, commitment, honesty, collaboration, and approachability • Proven ability to adapt in a fast-paced consulting environment, including managing time effectively and handling multiple tasks simultaneously Preferred Skills • Experience with version control systems (Git) and agile development frameworks • Design and document end-to-end testing plans for Python-based applications • PySpark experience for large-scale distributed data processing • Experience with Datadog or similar observability and monitoring platforms • Familiarity with Dagster or other modern data orchestration frameworks • Terraform or infrastructure-as-code (IaC) experience for cloud resource provisioning • Database experience with databases such as AWS RDS, MySQL, or PostgreSQL • Familiarity with D3.js for building custom data visualizations Where You'll Work Moser Consulting has 2 local, Indianapolis, IN offices. You will be able to work a hybrid/onsite schedule, flexing your time between either office. Salary At Moser Consulting, we believe in pay transparency and fairness. The $85k-$125k salary range for this role is not just a number-it's a reflection of the value we place on the skills and experience our employees bring to our team. We are committed to offering a competitive salary that aligns with the industry standards and the unique competencies you bring to our community. Benefits For over a quarter of a century, Moser Consulting has been a beacon for top-tier IT talent who excel in self-management. Our people are our greatest asset. We don't just hire the best-we welcome them into our family, connect them with opportunities, and empower them to create innovative solutions to technology challenges. Our unique culture is our competitive edge. It fosters happiness, health, and low stress, even in an industry known for its demands. This is why we're consistently recognized as one of the Best Places to Work in Indiana. We provide our employees with an inspiring workspace, a fun and collaborative atmosphere, and a generous compensation package. But that's not all. We also offer a suite of benefits designed to support and enrich our employees' lives. These include: • Training Opportunities: We believe in lifelong learning and provide numerous avenues for skill enhancement. • Fully Invested 401K Plan: We help secure your future with a fully invested 401K plan. • PPO and HDHP Medical Plans: Choose the health insurance program that best fits your needs. • Employer-Paid Dental and Vision Plans: We cover dental and vision plans, ensuring our employees have access to comprehensive health care. • Onsite Fitness Center: Stay fit and healthy with our state-of-the-art fitness center. • Wellness Program: We promote a healthy lifestyle with our wellness program. • Catered Lunches: Enjoy delicious catered lunches regularly. At Moser Consulting, we don't just offer jobs-we offer careers, growth, and a chance to join a thriving community. Come, be a part of our family.