Bachelor's degree in computer science, information technology, or related field (or equivalent experience)
Minimum 7 years in data engineering or operations roles focusing on Snowflake, SQL, AWS
2+ years developing and supporting production Apache Airflow workflows; Astronomer experience preferred
Strong Python development skills for building production data pipelines
Understanding of database security, backup, and recovery procedures
Excellent communication and collaboration skills
Strong problem-solving and time-management abilities
Familiarity with project management tools like Jira
Responsibilities
Design, develop, and optimize data pipelines in Astronomer using modern coding practices
Develop Airflow DAGs, operators, tasks, and reusable Python components for data ingestion
Write clean, modular, maintainable Python code
Monitor and triage data pipeline processes
Develop and maintain process and data documentation
Apply a software development approach including testing, documentation, and CI/CD workflows
Benefits
Hybrid work model (2x per week in NYC Metro Area) or fully remote in 23 select states
Opportunity to work with cutting-edge technologies like Apache Airflow and Astronomer
Collaborative environment with architects, data engineers, and analysts
Focus on professional development and modern coding practices
Engagement in a fast-paced, dynamic work environment
Full Job Description
Position Summary
As a Senior Associate, Data Platform Engineer, you will partner with architects, data engineers, and data analysts to improve different products and services. You will design, develop, test, deploy, and maintain reliable data pipelines and data integration solutions, with a strong focus on Python and Apache Airflow/Astronomer.
This role is hybrid (2x per week) for those located in the NYC Metro Area, or fully-remote in 23 select states*
Key Responsibilities
Design, develop and optimize data pipelines in Astronomer, utilizing latest coding practices and industry standards, modern design patterns and architectural principles
Develop Airflow DAGs, operators, tasks, and reusable Python components for data ingestion and transformation
Write clean, modular, maintainable Python code
Monitor and triage the data pipeline processes
Develop and maintain process and data documentation
Apply a software development approach including testing, documentation, modularity and dryness, CI/CD workflow, issue management and remediation
Required Qualifications
Bachelor's degree in computer science, information technology, or a related field (or equivalent experience)
Minimum 7 years of experience required in data engineering or operations roles with a focus on Snowflake, SQL, AWS
2+ years of hands-on experience developing and supporting production Apache Airflow workflows; Astronomer experience, strongly preferred
Strong Python development skills, including experience building production data pipelines
Strong understanding of database security, backup, and recovery procedures
Strong communication and collaboration skills
Excellent problem-solving, troubleshooting skills, and time-management skills
Ability to manage multiple tasks and priorities in a fast-paced environment
Familiarity with project management tools (e.g., Jira, or similar)
About CB Insights
CB Insights is a software and technology company that provides market intelligence and analysis to venture capitalists, corporate strategy and innovation teams, and investment bankers. The company's software platform provides data-driven insights on emerging technologies, startups, and venture capital trends. CB Insights' customers include Cisco, Salesforce, and American Express.
