GENERAL PURPOSE:
The Engineer II role focuses on designing, building, and optimizing data ingestion pipelines using StreamSets or similar data integration tools. It involves developing and maintaining Python-based data processing solutions to support scalable and efficient data movement into enterprise data platforms and data lakes. The role requires strong expertise in cloud-based architecture, middleware, and secure data transfer mechanisms. It also includes providing L3 production support, troubleshooting complex data pipeline issues, and enhancing system reliability and performance. Additionally, the engineer collaborates with cross-functional teams to implement best practices, automation, and continuous improvements.
The base salary range for this role is $113,200 - $195,000. The base salary range is dependent on factors including, but not limited to, experience, skills, qualifications, relevant education, certifications, seniority, and location. The range listed is just one component of the total compensation package for employees. Other rewards vary by position and location.
ESSENTIAL FUNCTIONS:
• Design, develop, and maintain scalable data ingestion pipelines using StreamSets or similar technologies.
• Develop, enhance, and optimize Python scripts for efficient data processing, transformation, and integration across multiple systems.
• Implement data ingestion solutions aligned with cloud architectures and enterprise data lake strategies.
• Monitor, troubleshoot, and resolve complex production issues, providing L3 support and ensuring system stability.
• Collaborate with cross-functional teams to design solutions, implement automation, and follow best practices in delivery.
• Ensure secure data handling by applying knowledge of networking, authentication, SSL, certificates, and related protocols.
• Maintain technical documentation, operational procedures, and ensure adherence to change management and testing standards.
COMPETENCIES:
People
• Building Effective Teams
• Developing Talent
• Collaboration
Self
• Leading by Example
• Communicates Effectively
• Ensures Accountability and Execution
• Manages Conflict
Business
• Business Acumen
• Plans, Aligns and Prioritizes
• Organizational Agility
With particular emphasis on the following specific position-related competencies:
• Dealing with Ambiguity
• Interpersonal Effectiveness
• Drive for Results
• Technical Competence/Expertise
QUALIFICATIONS AND SPECIAL SKILLS REQUIRED:
• Minimum 7 years of experience in software engineering, with extensive hands-on experience designing and managing StreamSets pipelines, including Control Hub operations, pipeline orchestration, and implementing standard data ingestion patterns for batch and real-time processing.
• Bachelor's degree in computer science, Information Technology, or related field (or equivalent experience).
• Strong hands-on experience with StreamSets or similar data ingestion/integration platforms.
• Proficiency in Python development for data processing, transformation, and automation.
• Solid understanding of cloud platforms (AWS, Azure, or GCP) and data lake architectures.
• Experience working with structured and semi-structured data formats (JSON, XML, CSV, etc.).
• Strong knowledge of operating systems (Linux/Windows), networking, and system-level concepts.
• Experience with security protocols including SSL, certificates, authentication, and secure data transfer mechanisms.
• Familiarity with JDBC, Java-based applications, and integration with databases and enterprise systems.
• Proven ability to troubleshoot complex production issues, provide L3 support, and optimize system performance.
• Strong analytical, problem solving, and debugging skills
• Excellent oral and written communication skills.
PHYSICAL REQUIREMENTS/ADA:
Job requires ability to work in an office environment, primarily on a computer.
Requires sitting, standing, walking, hearing, talking on the telephone, attending in-person meetings, typing, and working with paper/files, etc.
Consistent timeliness and regular attendance.
Vision requirements: Ability to see information in print and/or electronically.
This position may be performed remotely anywhere within the United States. #LI-Remote
SUPERVISORY RESPONSIBILITIES:
None
DISCLAIMER:
This job description is a summary of the primary duties and responsibilities of the job and position. It is not intended to be a comprehensive or all-inclusive listing of duties and responsibilities. Contents are subject to change at management's discretion.
