We are seeking a
Senior Data Engineer with deep Python and DynamoDB expertise to join a high-impact data engineering program supporting the modernization and scaling of financial services platforms. In this hands-on role, you will build and optimize data pipelines and systems that ingest, process, and analyze vast volumes of transactional and operational data, enabling real-time insights and driving data-driven decision-making at scale. Experience with Spark is a strong plus but not required.
WHAT YOU'LL DOAs a core member of the data engineering team, your responsibilities may include:
- Data Pipeline Development & Optimization
Design, build, and maintain scalable, reliable, and high-performance data pipelines using Python and AWS DynamoDB.
Implement robust ETL/ELT processes for both batch and streaming data workloads.
Collaborate with data scientists, analysts, and backend engineers to ensure data accessibility and integrity. - Data Architecture & Storage
Develop and optimize DynamoDB data models and indexing strategies to meet diverse access patterns and high throughput requirements.
Leverage cloud-native data storage solutions and serverless architecture to drive cost-efficient and scalable systems. - Big Data & Analytics
Contribute to the implementation and tuning of distributed data processing frameworks such as Apache Spark to support large-scale data transformations and analytics, where applicable.
Monitor and optimize pipeline performance, scalability, and latency. - Data Quality & Governance
Establish data validation, monitoring, and alerting mechanisms to ensure pipeline reliability and accuracy.
Collaborate with stakeholders on data governance, compliance, and security best practices. - Technology Growth & Collaboration
Participate in architectural discussions and code reviews, advocating for best practices in Python development and cloud data engineering.
Mentor junior team members and promote knowledge sharing across the engineering organization.
REQUIRED SKILLS & EXPERIENCE- Core Engineering
5+ years of professional experience in data engineering or software engineering roles focused on data-centric solutions.
Expert Python programming skills with a strong focus on data processing and pipeline development.
Deep experience designing and implementing solutions with AWS DynamoDB, including data modeling, performance tuning, and capacity planning.
Thorough understanding of cloud computing fundamentals (AWS preferred). - Data Engineering & Processing
Solid experience building high-scale ETL/ELT pipelines and working with streaming and batch data.
Familiarity with distributed data processing frameworks; experience with Apache Spark is a significant plus but not required. - Data Storage Systems
Experience working with NoSQL databases such as DynamoDB, Cassandra, or similar.
Comfortable with schema design trade-offs and query pattern optimizations in key-value and document stores. - Collaboration & Communication
Track record of collaborating effectively in cross-functional teams involving product, data science, and operations.
Strong verbal and written communication skills; able to explain technical concepts clearly to non-technical stakeholders.
Ability to thrive in fast-paced, evolving environments with changing requirements.
NICE TO HAVE- Hands-on experience with Apache Spark or other big data processing frameworks.
- Familiarity with other AWS services such as Lambda, Kinesis, Glue, or Redshift.
- Prior experience working in financial services or payments industry data platforms.
- Exposure to containerized/cloud-native deployments (e.g., Docker, Kubernetes).
- Experience with CI/CD pipelines and infrastructure-as-code tools such as Terraform or CloudFormation.
At this time, we are unable to provide sponsorship for this role.Minimum Degree Required:- Bachelor's Degree, in lieu of a degree, demonstrating in addition to the minimum years of experience required for the role, three years of specialized training and/or progressively responsible work experience in technology for each missing year of college is required
