8+ years in Data Engineering/Architecture/Cloud Architecture.
4+ years with Databricks and modern cloud data platforms.
Hands-on experience with AWS tools and technologies.
Proficient in Databricks architecture and development.
Strong hands-on experience with the AWS cloud platform.
Solid understanding of Enterprise Data Architecture.
Proficient in PySpark and SQL development.
Familiarity with AWS Bedrock and GenAI solutions.
Knowledgeable in data modelling, governance, and ETL processes.
Responsibilities
Design and implement scalable architecture using Databricks and AWS.
Define data architecture patterns for ingestion and processing.
Optimize data models and platforms.
Establish data governance and quality frameworks.
Develop data pipelines using PySpark and SQL.
Architect cloud-native and serverless solutions on AWS.
Integrate APIs for data solutions.
Enable GenAI applications with AWS Bedrock.
Collaborate with teams to develop scalable technical solutions.
Ensure compliance with security and governance standards.
Benefits
Full remote flexibility for better work-life balance.
Diverse projects in cutting-edge data and AI technologies.
Opportunity to work with emerging GenAI and AI-agent capabilities.
Collaboration with cross-functional teams to shape innovative solutions.
Full Job Description
Role - Senior Data Engineer Location - Remote USA Employment type - Fulltime
Job Summary:
We are looking for an experienced Data & AI Architect with strong expertise in Databricks, AWS, Enterprise Data Architecture, PySpark, and AWS Bedrock.
The candidate will be responsible for designing scalable, secure, and governed data and AI solutions across the enterprise.
The role requires a strong understanding of modern data platforms, data modelling, data governance, data products, ETL, APIs, and cloud-native/serverless architectures, along with hands-on experience in emerging GenAI, AI Agents, AWS Bedrock, and Databricks Genie capabilities.
Key Responsibilities:
Design and implement scalable enterprise data architectures using Databricks and AWS platforms.
Define data architecture patterns covering data ingestion, processing, storage, transformation, serving, and consumption.
Design and optimize data models, data marts, data warehouses, and enterprise data platforms.
Establish and implement data governance, security, quality, lineage, and access-control frameworks. Develop and optimize data processing pipelines using PySpark, SQL, and ETL frameworks. Work with Databricks capabilities for data engineering, analytics, AI/ML, and data product enablement.
Design cloud-native and serverless solutions on AWS using appropriate AWS services.
Architect and integrate solutions using APIs and API-based data integrations.
Work with AWS Bedrock to design and enable GenAI and AI-agent-based solutions.
Provide architecture guidance for AI Agents, Databricks Genie, and enterprise GenAI use cases.
Design solutions that enable reusable, scalable, and governed data products for business and analytical consumption.
Perform SQL query optimization, performance tuning, and data-processing optimization.
Collaborate with Data Engineers, Data Scientists, AI/ML Engineers, Product Owners, and business stakeholders to translate business requirements into scalable technical solutions.
Define architecture standards, reusable patterns, technical documentation, and best practices.
Conduct technical reviews and provide guidance to development teams on architecture, performance, scalability, and maintainability.
Ensure solutions align with enterprise security, governance, compliance, and cloud architecture standards.
Qualifications
Must have skills :
8+ years of overall experience in Data Engineering / Data Architecture / Cloud Architecture.
4+ years of experience with Databricks and modern cloud data platforms.
Hands-on experience with AWS and modern data/AI technologies.
Strong experience in Databricks architecture and development.
Strong hands-on experience with AWS cloud platform and services.
Strong understanding of Enterprise Data Architecture.
Strong experience with PySpark and SQL.
Experience with AWS Bedrock and GenAI/AI-agent solutions.
Strong knowledge of data modelling, data governance, DBMS, data warehouses, and data marts.
Experience in ETL/data pipeline architecture and development.
Experience with SQL query optimization and performance tuning.
Exposure to designing and consuming REST/API-based integrations.
Experience in designing and enabling enterprise data products.
Good-to-Have Skills:
Understanding of serverless architecture and cloud-native design patterns.
Experience with Databricks Genie / Genie Spaces. Experience with AI Agents / Agentic AI frameworks and architectures.
Experience with AWS serverless services such as Lambda, API Gateway, Glue, Step Functions, and related services.
Experience with Delta Lake / Lakehouse architecture.
Knowledge of Data Catalog, metadata management, lineage, and data quality frameworks.
Experience with CI/CD and DevOps practices for data and AI platforms.
Exposure to LLM application architecture, RAG, vector databases, and prompt engineering.
Experience working in large-scale enterprise data transformation or modernization programs.
Experience Strong experience in designing and delivering enterprise-scale data solutions.
Experience leading technical discussions and working with cross-functional stakeholders.
