Job Title: PySpark Developer
City: Minneapolis
State/Province: Minnesota
Job Description:
Role: PySpark Developer
Location: Irving, TX (3 Days onsite/week)
Job Description:
We are looking for an experienced PySpark Developer with over 7 years of hands-on experience in designing, developing, and optimizing large-scale data processing applications. The ideal candidate should have strong expertise in PySpark, Azure cloud services, Docker, and Azure Kubernetes Service (AKS), with the ability to build scalable, secure, and high-performing data solutions.
Key Responsibilities
• Design, develop, and maintain scalable data pipelines using PySpark for large-volume data processing.
• Work with Azure cloud services to build reliable and efficient data engineering solutions.
• Deploy, manage, and monitor containerized applications using Docker and Azure Kubernetes Service (AKS).
• Collaborate with data architects, analysts, and business stakeholders to understand requirements and deliver high-quality data solutions.
• Optimize PySpark jobs for performance, scalability, reliability, and cost efficiency.
• Implement best practices for data ingestion, transformation, validation, and processing across cloud environments.
• Identify and fix application, container, and platform vulnerabilities to ensure secure deployments.
• Support troubleshooting, production issue resolution, and performance tuning of data pipelines and cloud-based applications.
• Participate in code reviews, technical discussions, deployment planning, and documentation activities.
Required Skills
• Strong hands-on experience in PySpark development for distributed data processing.
• Experience with Azure Kubernetes Service (AKS) for deploying and managing containerized workloads.
• Proficiency in Docker for containerization, image management, and deployment workflows.
• Solid working knowledge of Azure cloud services and cloud-native application development.
• Strong understanding of data engineering concepts, ETL/ELT processes, and performance optimization.
• Ability to troubleshoot complex technical issues and deliver timely resolutions.
Good-to-Have Skills
• Working knowledge of Python for scripting, automation, and data processing support.
• Experience with SSIS for ETL workflow development and migration activities.
• Exposure to Informatica for data integration and enterprise data management.
• Experience in identifying, analyzing, and fixing security vulnerabilities across applications, containers, and cloud deployments.
Experience and Qualifications
• 7 years of overall experience in data engineering, big data development, or related technology roles.
• Strong practical experience in building and supporting production-grade PySpark-based data pipelines.
• Experience working in cloud-based environments, preferably Microsoft Azure.
• Good understanding of CI/CD practices, DevOps workflows, and container-based deployments.
• Strong analytical, problem-solving, and communication skills.
• Ability to work independently as well as collaboratively in a fast-paced delivery environment.
Preferred Candidate Profile
The ideal candidate will be a technically strong PySpark Developer who can take ownership of data engineering solutions, work effectively with Azure-based platforms, manage containerized deployments using Docker and AKS, and ensure secure, optimized, and reliable delivery of data processing applications.
Mandatory Skills: Spark Open Source.
Experience: 8-10 Years.
The expected compensation for this role ranges from $80,000 to $158,000 .
Final compensation will depend on various factors, including your geographical location, minimum wage obligations, skills, and relevant experience. Based on the position, the role is also eligible for Wipro's standard benefits including a full range of medical and dental benefits options, disability insurance, paid time off (inclusive of sick leave), other paid and unpaid leave options.
