Your Journey to Success:The Data Engineer will design, implement, and manage scalable data pipelines, databases, and processing systems to support engineering, product, and analytics teams. This includes developing and administering platforms like Databricks and Snowflake, optimizing data architecture, and ensuring performance, scalability, and reliability.
Your Mission:- Data Infrastructure Development:
- Design, build, and maintain scalable data pipelines to process large volumes of structured and unstructured data.
- Optimize data architecture for both real-time and batch processing workflows.
- Integrate and maintain PostgreSQL, MySQL, Databricks, and Snowflake within our data ecosystem.
- Databricks Development and Administration:
- Develop and manage workflows on Databricks to support ETL/ELT processes.
- Optimize Databricks performance for distributed computing and data processing.
- Monitor, troubleshoot, and ensure the reliability of Databricks clusters and jobs.
- Snowflake Development and Administration:
- Design and optimize Snowflake schemas and data warehouses for efficient querying.
- Develop and manage data ingestion pipelines for Snowflake using modern tools and frameworks.
- Administer Snowflake environments, including account management, performance tuning, and resource optimization.
- Database Management:
- Administer PostgreSQL and MySQL databases, ensuring high availability and performance.
- Implement and enforce data governance practices to ensure data security and compliance.
- Collaboration:
- Partner with analytics and engineering teams to define data needs and deliver solutions for business insights.
- Work with product teams to align data solutions with product goals and user requirements.
- Monitoring and Troubleshooting:
- Set up monitoring and alerting systems for data platforms and pipelines.
- Proactively debug and resolve data-related issues to ensure system reliability.
- Innovation and Best Practices:
- Stay current with emerging trends and advocate for tools and practices that enhance data engineering efficiency.
- Document workflows, pipelines, and platform configurations to foster team collaboration and knowledge sharing.
What Sets You Apart - Bachelor's degree in Computer Science, Engineering, or a related field (or equivalent experience).
- 3+ years of professional experience in data engineering or related fields.
- Proficiency in Databricks development and cluster administration.
- Hands-on experience with Snowflake development and administration, including performance tuning and security configuration.
- Strong understanding of relational databases, particularly PostgreSQL and MySQL.
- Expertise in programming languages like Python, Scala, or SQL for data processing and automation.
- Familiarity with ETL/ELT tools and frameworks such as Apache Airflow or dbt.
- Experience with cloud platforms (AWS, Azure, or GCP).
Preferred Skills:- Knowledge of distributed systems and big data frameworks like Spark.
- Experience with CI/CD pipelines for deploying data workflows.
- Familiarity with real-time streaming technologies (Kafka, Kinesis, etc.).
Why Join Us?- Work with a passionate, innovative team driving the next wave of industry solutions.
- Opportunities for professional growth and continuous learning.
- Competitive compensation, comprehensive benefits, and a flexible work environment.
In return for your dedication, we offer:- Salary
- Health benefits
- Top-tier health benefits include; medical, dental, vision, LTD, and life insurance
- Mental health benefits
- Additional benefits
- Unlimited PTO
- In the WeWork office In the heart of the financial district in downtown Kansas City
- Open space concept & team-focused atmosphere
- Company-wide celebrations
- Team bonding events and happy hours
CHECK US OUT: https://www.youtube.com/watch?v=cabweOxGias&t=1s
OUR APP : https://surveysonthego.com/
