JOB SUMMARY Responsibilities Data Engineer Job Requirements Data Pipeline Expertise - Capable of implementing a diverse range of data solutions with significant autonomy, demonstrating the ability to independently build and maintain complex ETL (Extract, Transform, Load) pipelines Possesses advanced skills in pipeline complexity management and orchestration, with a proven track record of developing high-performance data products accessible via APIs for critical tier 1 trading applications Demonstrates expertise in archiving data products to data lakes for comprehensive analytics purposes Data Quality and Advanced Analytics - Strong proficiency in Data Quality, Data Validation, and Anomaly Detection techniques - Experience with AI/ML and Large Language Model (LLM) approaches to data analysis and validation - Ability to implement sophisticated data integrity and verification processes Key Responsibilities Data Pipeline Expertise - Capable of implementing a diverse range of data solutions with significant autonomy, demonstrating the ability to independently build and maintain complex ETL (Extract, Transform, Load) pipelines Possesses advanced skills in pipeline complexity management and orchestration, with a proven track record of developing high-performance data products accessible via APIs for critical tier 1 trading applications Demonstrates expertise in archiving data products to data lakes for comprehensive analytics purposes Data Quality and Advanced Analytics - Strong proficiency in Data Quality, Data Validation, and Anomaly Detection techniques - Experience with AI/ML and Large Language Model (LLM) approaches to data analysis and validation - Ability to implement sophisticated data integrity and verification processes Required Qualifications Technical Skill Requirements: Programming and Cloud Technologies - Expert-level skills in: - Python - SQL - AWS Cloud Services: - Cloud Storage - Lambda - ECS - Glue Data Quality - HDMI - Redshift Machine Learning - OpenSearch - RDS API and Resilience Technologies - Advanced skills in building resilient API architectures, including: - Microservices design patterns - API gateway configuration - Circuit breaker implementations - Fault tolerance and error handling mechanisms - Expertise in API development and management tools: - Swagger/OpenAPI - Postman - GraphQL - RESTful API design principles Proficiency in resilience engineering techniques: - Distributed system design - Load balancing - Failover strategies - Horizontal and vertical scaling approaches - Containerization and orchestration: - Docker - Kubernetes - Helm charts Monitoring and observability: - Prometheus - Grafana - ELK Stack (Elasticsearch, Logstash, Kibana) - Distributed tracing with Jaeger or Zipkin Additional Technical Competencies Proficient in Apache Flink - Advanced Java programming skills - Proven ability to create and optimize complex data products - Demonstrated Subject Matter Expertise (SME) in data engineering principles Preferred Qualifications Professional Attributes Capability to work independently through complex data challenges - Strong preference for candidates with prior experience in investment data domain and related technologies Certifications Key Differentiators - Deep understanding of data engineering best practices - Ability to design scalable, efficient, and resilient data solutions - Proven track record of solving intricate data infrastructure problems with robust API architectures