Data EngineerWe have a vacancy for a Data Engineer to join our Austin based team.
Your responsibilities will include:
- Supporting Data Engineering Infrastructure:
- Contribute to the design, development, implementation and continuous improvement of our data engineering tools, workflows, processes, and platforms. This includes enhancing the architectural foundations and integrating new data management technologies.
- Writing Well-Structured Code:
- Develop clean, maintainable, well-documented code that adheres to best practices. Support best coding practices within Habitat's software, machine-learning, and data science teams.
- Enhance data engineering knowledge:
- Improve expertise within the software team and ensure their ability to support and collaborate on the data infrastructure infrastructure.
- Data Quality Management:
- Continuously enhance data quality across multiple dimensions such as accuracy, availability, performance, and accessibility to ensure a clear understanding of data within the company.
- Providing backup/escalation to the tech-on-call team.
- Communicating effectively across Software and Data Science teams.
RequirementsPreferred skills and experience:
- 3+ years of Python experience.
- 3+ years of working in technical teams, building data pipelines, delivering productionised code, building/maintaining live applications, developing tooling and improving backtesting frameworks.
- Experience in applying relational database design.
- Proficiency with Orchestration and IaC in AWS (e.g. Terraform, Kubernetes, RabbitMQ, Airflow, Prefect), Git, containerisation (Docker), database management (e.g. Postgres, Alembic).
- Fluent in Python and its wider numerical ecosystem (e.g. Pandas, NumPy, Polars, Pydantic).
'Nice to have' skills and experience:
- 2+ years of orchestrating machine learning workflows.
- Experience with OSS data warehousing tooling and management.
- Cloud infrastructure experience.
- Experience with monitoring frameworks (e.g. Prometheus).
- Experience archiving data to Parquet on S3 and creating tools for API/Grafana queries.
- Experience centralising diverse datasets for analytics, visualisation and machine learning.
- Familiarity with time-series forecasting and/or optimisation.
- Experience with data visualisation and dashboards (e.g. Grafana, Superset).
- Familiarity with machine learning and associated techniques (feature engineering, boosting methods, LightGBM).
Ultimately we are looking for someone who is a great fit for our company so we encourage you to apply even if you may not meet every requirement in this posting. We value diversity and our environment is supportive, challenging and focused on the consistent delivery of high quality, meaningful work.
In return, we'll give you a competitive salary, flexible working arrangements and a lot of personal development opportunities. We operate a hybrid working model with at least 2 days in our office in Austin.
