5-7 years of relevant experience in data engineering
BA/BS in a relevant discipline
Experience with multiple development methodologies
Proficient in relational databases like SQL Server and Oracle
Familiarity with cloud platforms (AWS, Azure, GCP) and their ETL tools
Programming experience in Python, Spark, and similar languages
Advanced knowledge of large-scale processing engines and database security
Responsibilities
Collaborate with data architects and stakeholders to define data pipeline requirements
Design, build, document, test, and maintain data pipelines following best practices
Perform ETL and ELT operations adhering to data governance standards
Develop processes for data quality and governance through validation and anomaly detection
Oversee junior data engineers and provide mentorship
Benefits
Onsite work required at least one day per week
Opportunity for professional development and mentorship
Engagement with cross-functional teams
Exposure to diverse data sources and platforms
Alignment with the organization's core values and culture
Full Job Description
The Sr Data Engineer will design and manage data systems, pipelines, and management tools. They will oversee junior data engineers, recommend and deploy data models, and lead projects involving data collection and storage systems. This role will design, build, document, test, and maintain critical data pipelines from large volumes of data from internal and external data sources cross different platforms.
*This role requires colleagues to be onsite in the office at least one day per week. Candidates must also be available to attend a final onsite interview.* PRIMARY RESPONSIBILITIES:
Collaborate with data architect, analysts, and stakeholders to identify and document needs/requirements for data pipelines and process automation
Design, build, document, test, and maintain data pipelines using industry best practices
Performs ETL, ELT operations in accordance with enterprise data governance and security standards
Develop data quality and governance process to ensure the accuracy and quality of the data through inspection, validation, processing, anomaly detection
KNOWLEDGE, SKILLS & ABILITIES:
Detail oriented and proactive in taking measures to avoid or mitigate issues.
Excellent understanding of computer systems, integrations, databases, and data storage systems.
Understanding of agile data engineering concepts and processes
Ability to capture and document business processes and requirements and work with colleagues to create functional requirements.
Ability to work with clients to translate business needs to technical solutions
Ability to provide guidance, mentoring, and training for peers
Exceptional interpersonal skills and ability to effectively communicate with teams across the entire organization.
Demonstrates the organization's core values, exuding behavior that is aligned with the firm's culture
Proficiency in Microsoft Office Suite (Word, Excel, PowerPoint, Outlook)
Ability to learn any other appropriate program or software system used by the firm as necessary
EDUCATION & EXPERIENCE:
5-7 years of relevant experience required
BA/BS in relevant discipline required
Must be able to work onsite at least one day per week
Vast experience with multiple development methodologies
Experience of building large, complicated data pipelines cross different platforms, data sources, data structures
Experience of working with relational databases such as SQL Server, Oracle, etc., and SQL scripting
Experience of cloud platforms (like AWS, Azure, GCP) and its ETL tools and techniques of sourcing, maintaining, and updating data, data warehousing, data cleansing & transformation, etc.
Programming experience in Python, Spark, and other similar languages
Advanced programming knowledge and experience with large-scale processing engines are required, as well as a deeper understanding of database security and compliance tools.
Location Requirement: This position requires onsite attendance at least one day per week, with additional onsite presence as needed for meetings, collaboration, and training.
IMPORTANT NOTICE:
This position description is intended to describe the level of work required of the person performing in the role and is not a contract. The essential responsibilities are outlined; other duties may be assigned as needs arise or as required to support the organization. All requirements may be subject to reasonable accommodation to applicants and colleagues who need them for medical or religious reasons.