Job DescriptionWe are searching for an accountable, multitalented Data Engineer to facilitate the operations of our Data Scientists. The ideal candidate will be responsible for employing machine learning techniques to create and sustain structures that allow for the analysis of data, while remaining familiar with dominant programming and deployment strategies in the field. During various aspects of this process, you should collaborate with coworkers to ensure that your approach meets the needs of each project.
To ensure success as a Data Engineer, you should demonstrate flexibility, creativity, and the capacity to receive and utilize constructive criticism. A formidable Data Engineer will demonstrate unsatiated curiosity and outstanding interpersonal skills.
Essential Responsibilities- Design, develop, document, and test advanced data systems that bring together data from disparate sources, making it available to data scientists, analysts, and other users using scripting and/or programming languages (Python, Java, C, etc)
- Evaluate structured and unstructured datasets utilizing statistics, data mining, and predictive analytics to gain additional business insights
- Design, develop, and implement data processing pipelines at scale
- Present programming documentation and design to team members and convey complex information in a clear and concise manner.
- Extract data from multiple sources, integrate disparate data into a common data model, and integrate data into a target database, application, or file using efficient programming processes.
- Write and refine code to ensure performance and reliability of data extraction and processing.
- Communicate with all levels of stakeholders as appropriate, including executives, data modelers, application developers, business users, and customers
- Participate in requirements gathering sessions with business and technical staff to distill technical requirements from business requests.
- Partner with clients to fully understand business philosophy and IT Strategy; recommend process improvements to increase efficiency and reliability in ETL development.
- Collaborate with Quality Assurance resources to debug code and ensure the timely delivery of products.
- Some of our technologies might include: HDFS, Cassandra, Spark, Java, Scala, Informatica, SQL Server, Oracle, Ab Initio, Kafka.
Qualifications- Bachelor's degree in Data Engineering, Big Data Analytics, Computer Engineering, or related field.
- At least 3 years of proven experience as a Data Engineer.
- Expert proficiency in Python, C++, Java, R, and SQL.
- Familiarity with Hadoop or suitable equivalent.
- Excellent analytical and problem-solving skills.
- A knack for independent and group work.
- Scrupulous approach to duties.
- Capacity to successfully manage a pipeline of duties with minimal supervision.
Videos To Watchhttps://www.youtube.com/watch?v=oPAB0-RJR5g
https://www.youtube.com/watch?v=WbgJ0XsffY8
