Data Engineers / Architect

Intelliswift$120K — $160K *
Information Technology
8 - 10 years of experience
Job Overview by Ladders

Qualifications

  • 8+ years of experience in data engineering and analysis
  • 3+ years in a lead or architect role
  • Expertise in Spark, Hadoop, Hive, and SQL/Hive
  • Proficiency in Python and PySpark; knowledge of Scala is a plus
  • Strong understanding of OLAP systems and data warehousing concepts
  • Experience with distributed monitoring and observability tools
  • Advanced Tableau skills for data visualization

Responsibilities

  • Work independently with minimal support to execute data engineering tasks
  • Develop and optimize complex data pipelines using Spark and Hive
  • Automate data workflows through shell scripts and job schedulers like Airflow
  • Train and educate users and peers in complex technical topics
  • Document procedures and write detailed status reports
  • Create business value by solving problems effectively and independently
  • Implement performance tuning techniques in Spark, Hive, and Teradata

Benefits

  • Flexibility in managing workload and work hours
  • Opportunity for impactful work with internal stakeholders
  • Exposure to advanced big data technologies and tools
  • Collaborative culture with the chance to positively interact with diverse teams
  • Potential for professional growth in a lead capacity
Full Job Description
2 Data Engineer / Architect
Client doesn't have much bandwidth for any handholding. They will grant required access and the candidate should work on their own with minimal Apple support. Hence, they need senior level consultant / Architect who can execute the tracks on their own.

Preferred Qualifications:
• Extensive hands-on experience with Spark, Hadoop, Hive, Unix, Teradata, Vertica database
• Must be capable of independent problem-solving and self-direction with an eye towards creating business value for our internal customers
• Experience with distributed infrastructure/application monitoring and observability, including metrics, logs, and traces
• Good interpersonal and communication skills; capable of writing justifications, training and educating users and peer technical staff in complex topics, writing status reports, documenting procedures and interacting positively with other staff and management
• Experience with Python, Spark (PySpark, Scala) programming languages.
• Good understanding of OLAP systems and Data warehousing & Data modeling concepts
• Experience working on Big Data, BI or Analytics related projects
• Advanced knowledge of SQL/Hive
• Advanced Tableau skills
• Experienced with job schedulers developing shell scripts, CRON, Airflow jobs to automate data workflows
• Performance tuning techniques in Spark/Hive/Teradata.
• Advanced knowledge of architecting and scaling complex pipelines
• 8+ years working in data engineering, analysis and infrastructure
• 3+ years functioning in a lead and architect capacity

Location : , CA.

Similar Jobs

More Jobs at Intelliswift

More Information Technology Jobs

Find similar Data Engineers / Architect jobs: