Job Summary:
We are seeking a Database Developer to support a data-intensive team responsible for reconciling wallet transaction data. The role involves building and optimizing data pipelines, loading high volumes of data from Amazon S3 into PostgreSQL, and improving database performance. The environment processes tens of millions of transactions daily, requiring hands-on production experience in developing scalable data workflows and optimizing databases to efficiently handle large datasets.
Key Responsibilities:
• Develop and maintain data processing workflows using Python and PySpark.
• Build, maintain, and support AWS Glue ETL jobs to extract, transform, and load high-volume data.
• Load and process files from Amazon S3 into PostgreSQL databases.
• Design efficient database schemas and data structures to support transaction reconciliation workloads.
• Identify database performance bottlenecks and optimize PostgreSQL performance.
• Test and optimize data pipelines and database workloads to ensure performance and scalability.
• Collaborate with application developers to ensure database design and data processing solutions meet application requirements.
• Analyze processing volumes, identify performance challenges, and implement improvements for large-scale data workloads.
Required Qualifications:
• 6-12 years of relevant experience in database development, data engineering, or related technical roles.
• Hands-on production development experience with Python, PySpark, AWS Glue, and PostgreSQL.
• Direct experience building and maintaining AWS Glue ETL jobs; familiarity alone is insufficient.
• Demonstrated experience processing millions of records or transactions in production environments.
• Strong PostgreSQL database development, schema design, and performance tuning experience.
• Experience ingesting and processing data from Amazon S3 into relational databases.
• Ability to explain previous data processing volumes, technical ownership, performance challenges, and improvements delivered.
• Strong understanding of scalable data processing workflows and database optimization techniques.
Preferred Qualifications:
• Java development experience.
• Experience in financial services, payments, wallet data processing, or transaction reconciliation.
• Experience supporting high-volume workloads comparable to approximately 10-50 million transactions per day.
