Tata Consultancy Services

Databricks Architect

Tata Consultancy Services • $110K — $140K *
Marlborough, MA 01752In-Person
Information Technology
8 - 10 years of experience
Today
Be an Early Applicant
Job Overview by Ladders

Qualifications

  • 10+ years of data engineering experience, with 3+ years in Databricks/Spark ecosystem
  • Expertise in Databricks Lakehouse, Delta Lake, and related technologies
  • Hands-on experience with Unity Catalog for business semantics and KPIs management
  • Knowledge of AI/BI tools such as Genie and Mosaic AI
  • Advanced proficiency in Python, PySpark, and SQL
  • Strong understanding of data modeling and warehousing concepts
  • Experience with major cloud providers like AWS, Azure, or GCP and data storage solutions.

Responsibilities

  • Design, code, and deploy scalable batch and streaming data pipelines
  • Implement complex data transformations using the Medallion architecture
  • Establish data quality frameworks to ensure the accuracy of Customer 360 data
  • Monitor and optimize Databricks performance for efficiency
  • Manage Databricks environments and deploy jobs using Infrastructure as Code and CI/CD practices
  • Implement data governance features using Unity Catalog for compliance
  • Collaborate with cross-functional teams to deliver structured datasets.

Benefits

  • Discretionary Annual Incentive
  • Comprehensive Medical Coverage including Vision and Dental
  • Maternal & Parental Leave support
  • Insurance options including Identity Theft Protection
  • Convenience benefits for commute and professional growth training
  • Generous time off policies inclusive of vacations and sick leave
  • Legal and financial assistance options including 401K and college funds.
Full Job Description
Must Have Technical/Functional Skills

  • Experience: 10+ years of hands-on data engineering experience, with at least 3 years focused on the Databricks/Spark
  • Ecosystem
  • Databricks Expertise: Deep, hands-on expertise with the Databricks Lakehouse Platform, including Delta Lake,
  • Structured Streaming, Lakeflow Declarative Pipelines (formerly Delta Live Tables), Databricks SQL, and cluster/serverless configuration and optimization.
  • Business Semantics & Governance: Hands-on experience with Unity Catalog Business Semantics, including Metric Views (measures, dimensions, materialization), Domains, and Pages/Glossary to define governed, reusable KPIs once and serve them consistently across SQL, BI, and AI agents.
  • Agentic & Conversational Analytics: Working knowledge/exposure of AI/BI Genie (Genie Spaces, Genie Ontology, trusted assets), Genie Code for agentic pipeline/SQL development, Databricks One for business-user consumption, and Mosaic AI for building and serving AI/ML models and agents.
  • Programming Mastery: Expert-level proficiency in Python and PySpark. Advanced SQL skills are essential.
  • Data Warehousing Concepts: Strong understanding of data modeling principles, including dimensional modeling
  • (Kimball), data warehousing concepts, and ETL/ELT design patterns.
  • Cloud Proficiency: Proven experience working with a major cloud provider (Azure, AWS, or GCP), particularly with
  • data storage S3 and related services.
  • Software Engineering Mindset: Experience with software engineering best practices, including version control (Git),
  • code reviews, testing, and CI/CD.
  • Certification: Databricks


Roles & Responsibilities

  • Data Pipeline Development: Design, code, and deploy robust and scalable batch and streaming data pipelines
  • using PySpark, Spark SQL, and Lakeflow (Lakeflow Connect for ingestion and Lakeflow Declarative Pipelines) to ingest data from sources such as Point-of-Sale (POS), e-commerce platforms, loyalty systems, and marketing clouds.
  • Data Modeling & Transformation: Implement complex data transformations and business logic within the Medallion
  • architecture (Bronze, Silver, Gold layers). Build and optimize the final "Gold" dimension tables that will
  • serve as the single source of truth. Define governed business KPIs on top of the Gold layer using Unity Catalog Metric Views (business semantics) so metrics are computed once and reused consistently across BI, SQL, and Genie.
  • Data Quality: Implement data quality frameworks and cleansing routines to ensure the accuracy and trustworthiness
  • of the Customer 360 data.
  • Performance Optimization: Proactively monitor, debug, and tune Databricks jobs and Spark clusters for performance
  • and cost-efficiency. Implement best practices for partitioning, caching, and data layout in Delta Lake.
  • Infrastructure as Code (IaC) & CI/CD: Work with DevOps teams to manage Databricks environments, clusters, and
  • job deployments using tools like Terraform and AWS DevOps/GitHub Actions. Champion and implement CI/CD best
  • practices for data pipelines.
  • Data Governance & Security: Implement data governance features within Databricks Unity Catalog, including
  • data lineage tracking, fine-grained access controls (row/column-level security and ABAC policies), tags, and data masking to ensure compliance and security across BI and AI/agent workloads.
  • C ollaboration: Partner closely with Functional Consultants, Data Scientists, and Analytics Engineers to understand
  • their data requirements and deliver well-structured, consumption-ready datasets.
  • Semantic Modeling & Business Semantics: Build and govern Unity Catalog Business Semantics, authoring Metric Views (measures, dimensions, joins, synonyms, materialization), Domains, Pages, and Glossary terms, and certifying trusted assets so every dashboard, SQL query, notebook, and AI agent works from the same governed definitions.
  • Genie & Agentic Analytics Enablement: Configure and curate AI/BI Genie Spaces and the Genie Ontology (instructions, trusted assets, example queries, synonyms) to deliver accurate natural-language, conversational analytics for business users, and enable consumption through AI/BI Dashboards and Databricks One.


TCS Employee Benefits Summary:

  • Discretionary Annual Incentive.
  • Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.
  • Family Support: Maternal & Parental Leaves.
  • Insurance Options: Aut& Home Insurance, Identity Theft Protection.
  • Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.
  • Time Off: Vacation, Time Off, Sick Leave & Holidays.
  • Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.


#LI-KR3

Salary Range-$110,000-$140,000 a year

About Tata Consultancy Services

Tata Consultancy Services (TCS) is an Indian multinational information technology (IT) services and consulting company, headquartered in Mumbai, Maharashtra, India. It is a subsidiary of Tata Group and operates in 149 locations across 46 countries. TCS is the largest Indian company by market capitalization and is ranked 11th on the Forbes Global 2000 list of the world's biggest public companies. TCS is also the second-largest IT services company in the world by revenue and the largest employer of women in India. The company provides services in areas including IT, consulting, and business solutions.
Learn more about Tata Consultancy Services
Size
469,261 employees
Industry
Information Technology
* Ladders Estimates

Similar Jobs

Ad banner background

Get Ready For Your
Next Interview

More Jobs at Tata Consultancy Services

More Information Technology Jobs

Find similar Databricks Architect jobs:

NationwideMarlborough, MA