Big Data Engineer

SP Software Solutions

$120K — $150K *
Information Technology
Less than 5 years of experience
Job Overview by Ladders

Qualifications

  • Master/Bachelor degree in Computer Science or related field; advanced degree preferred.
  • 2+ years of software development experience, focusing on data-centric projects.
  • 1+ years of experience with Python, Pandas, and SQL.
  • Expert knowledge of SQL for data manipulation and validation across various databases.
  • Experience with CI/CD processes and version control tools.

Responsibilities

  • Translate business requirements into technical solutions using various technologies.
  • Collaborate with business partners to clarify project requirements.
  • Design and implement new software programs and enhance existing ones.
  • Create documentation for installation and maintenance procedures.
  • Write and integrate unit tests into development processes.
  • Refactor existing solutions for reusability and scalability.
  • Lead cross-functional teams to build and launch applications at scale.

Benefits

  • Work in an innovation-driven environment.
  • Collaborate with a small team while leading efforts when necessary.
  • Opportunity to impact the deployment of production code.
  • Engage in continuous learning and adaptation to new technologies.
Full Job Description
Job Description
  • Translate business requirements and source system understanding into technical solutions using Opensource Tech Stack, Big Data & Java etc.
  • Work with business partners directly to seek clarity on requirements
  • Define solutions in terms of components, modules and algorithms
  • Design, develop, document and implement new programs and subprograms, as well as enhancements, modifications, and corrections to existing software.
  • Create technical documentation and procedures for installation and maintenance.
  • Write Unit Tests covering known use cases using appropriate tools
  • Integrate test frameworks in development process
  • Refactor existing solutions to make it reusable and scalable
  • Work with operations to get the solutions deployed
  • Take ownership of production deployment of code
  • Collaborating with and/or lead cross functional teams, build and launch applications and data platforms at scale, either for revenue generating or operational purposes
  • Come up with Coding and Design best practices
  • Thrive in self-motivated internal-innovation driven environment
  • Adapting fast to new application knowledge and changes


Qualifications
• Master/Bachelor degree in Computer Science, Electrical Engineering, Information Systems or other technical discipline; advanced degree preferred.
• Minimum of 2+ years of software development experience (with a concentration in data centric initiatives), with demonstrated expertise in leveraging standard development best practice methodologies.
• Minimum 1+ years of experience in Python, Pandas and SQL
• Strong knowledge of Database concepts and Python.
• Expert on SQL skills for data manipulation (DML) and validation (SQL Server, DB2, Oracle).
• Expertise in Object Oriented Programming Language - Java / Python.
• Experience using CI/CD Process, version control and bug tracking tools.
• Experience in handling very large data volume in low latency and batch mode.
• Experience with automation of job execution, validation and compares of the data files on Hadoop Env at the field level.
• Experience in leading a small team and be a team player.
• Strong communication skills with proven ability to present complex ideas and document in a clear and concise way.
• Quick learner; self-starter, detailed and in-depth.

Additional Information

Similar Jobs

More Information Technology Jobs

Find similar Big Data Engineer jobs: