Full Job Description
Role Overview:As a Data Engineer at TheAppLabb, you will be responsible for designing, developing, and optimizing data pipelines, ensuring seamless data integration, and supporting AI-driven applications. You will collaborate with data scientists, engineers, and business stakeholders to enable data-driven decision-making.
Key Responsibilities:
• Build & Maintain Data Pipelines: Design and develop scalable ETL/ELT workflows for structured and unstructured data.
• Data Architecture & Storage: Optimize data warehouses, databases, and cloud storage solutions.
• Integration & Processing: Work with APIs, streaming data, and batch processing to consolidate data from multiple sources.
• Performance & Scalability: Monitor, troubleshoot, and improve data systems for speed and reliability.
• Collaboration & Strategy: Work closely with AI/ML teams to support data modeling and analytics initiatives.
• Data Governance & Security: Ensure data quality, security, and compliance with industry best practices.
Required Skills & Qualifications:
• Technical Proficiency:
• Strong experience with Python, SQL, and Scala for data processing.
• Hands-on experience with ETL frameworks, data pipelines, and workflow orchestration tools (e.g., Apache Airflow, DBT).
• Expertise in Big Data Technologies (e.g., Spark, Hadoop).
• Knowledge of Cloud Platforms (AWS, GCP, or Azure) and services like S3, Redshift, BigQuery,or Snowflake.
• Experience with relational and NoSQL databases (PostgreSQL, MongoDB, etc.).
• Familiarity with streaming technologies (Kafka, Kinesis, or Pub/Sub) is a plus.
• Soft Skills:
• Strong problem-solving and analytical thinking.
• Ability to work in a fast-paced, agile environment.
• Excellent communication and collaboration skills.
Nice-to-Have:
• Exposure to AI/ML workflows and data science frameworks.
• Experience with CI/CD for data engineering.
• Knowledge of data privacy regulations (GDPR, CCPA).
Why Join Us?
• Work on large-scale cloud-based data platforms with cutting-edge technologies.
• Collaborate with a dynamic, innovative, and data-driven team.
• Competitive salary
• Opportunities for career growth and continuous learning.