Technical certification in multiple technologies is desirable.
Strong experience with AWS/Azure Databricks for ETL pipeline development.
Hands-on expertise in PySpark, Python, SQL, and data warehouse concepts.
Responsibilities
Design, develop, and maintain scalable ETL pipelines and data workflows.
Build and optimize data ingestion frameworks for large data volumes.
Perform data preprocessing and analytics using Python and PySpark.
Develop and manage ETL pipelines using AWS/Azure Databricks.
Create complex SQL queries to support business reporting needs.
Implement data warehouse solutions for enterprise analytics.
Collaborate with cross-functional teams to understand data requirements.
Benefits
Remote work flexibility.
Opportunity to work with cutting-edge cloud technologies.
Engagement in a fast-paced, innovative environment.
Support for professional development and technical certifications.
Full Job Description
Role description
Job Description
Lead Data Engineer
What is in it for you?
Data Engineer is responsible for building scalable and high-performance data pipelines, ETL workflows, and data warehouse solutions using Databricks, PySpark, Python, SQL, and cloud technologies.
Responsibilities: -
Design, develop, and maintain scalable and high-performance ETL pipelines and data processing workflows.
Build and optimize data ingestion frameworks for processing large volumes of structured and unstructured data.
Perform data preprocessing, transformation, and analytics using Python and PySpark.
Develop and manage ETL processing pipelines using AWS/Azure Databricks.
Build complex SQL queries and perform data analysis to support business and reporting requirements.
Design and implement data warehouse solutions to support enterprise analytics and reporting.
Work closely with cross-functional teams to understand data requirements and develop scalable solutions.
Develop reliable, efficient, and reusable data processing frameworks that support evolving business needs.
Support data validation, quality checks, performance tuning, and optimization of ETL processes.
Work in a fast-paced environment to deliver innovative data engineering solutions aligned with business objectives.
Create reports and visualizations using Tableau to support data-driven decision-making.
Experience: -
10+ Years
Location: -
Remote
Educational Qualifications: -
Engineering Degree - BE/ME/BTech/MTech/BSc/MSc.
Technical certification in multiple technologies is desirable.
Skills: -
Mandatory skills
Strong experience with AWS/Azure Databricks for developing and managing large-scale data processing and ETL pipelines.
Hands-on expertise in Spark (PySpark), Spark SQL, Python, SQL, ETL development, and Data Warehouse concepts.
Proven experience building scalable, high-performance data processing frameworks and ETL workflows.
Strong ability to develop complex SQL queries and perform detailed data analysis.
Experience in data ingestion, data transformation, and data pipeline development.
Hands-on experience working with cloud platforms such as AWS or Azure.
Strong understanding of data engineering best practices, performance optimization, and scalable architecture design.
Excellent visual, verbal, and stakeholder communication skills.
Good to Have Skills
Knowledge of ETL concepts, Data Ingestion, and Shell Scripting (DOS/Bash).
Experience building ETL processing pipelines using AWS Databricks or Azure Databricks.
Experience with Tableau for reporting and data visualization.
Exposure to AWS cloud services and data engineering ecosystems.
Experience working in Agile and fast-paced delivery environments.
