Description: As a Data Engineer - Big Data and ETL, you will participate in all aspects of the software development lifecycle which includes estimating, technical design, implementation, documentation, testing, deployment and support of applications developed for our business partners. As a member working in a team environment you will take direction from solution/Platform architects and Leads on development activities. Work inside a team with Strong Data Warehousing skills, including: Data cleanup, ETL, ELT and handling scalability issues for enterprise level data warehouse. Implements, troubleshoots, and optimizes distributed solutions based on modern big data technologies like Hive, Hadoop, Spark, Elastic Search, Storm, Kafka, etc. in both an on premise and cloud deployment model to solve large scale processing problems. The software application landscape within KP-IT spans many technologies that include:
o building data platform on cloud (AWS or Azure)
o using Python, Java or any other language to solving data problems
o implementing SDLC best practices and Agile methods
This individual contributor is primarily responsible for translating business requirements and functional specifications into software solutions, for assisting in contributing to and leveraging the technical direction for the development of integrated business and/or enterprise application solutions, and for providing consultation to help ensure new and existing software solutions are developed.
- Completes work assignments and supports business-specific projects by applying expertise in subject area; supporting the development of work plans to meet business priorities and deadlines; ensuring team follows all procedures and policies; coordinating and assigning resources to accomplish priorities and deadlines; collaborating cross-functionally to make effective business decisions; solving complex problems; escalating high priority issues or risks, as appropriate; and recognizing and capitalizing on improvement opportunities.
- Practices self-development and promotes learning in others by proactively providing information, resources, advice, and expertise with coworkers and customers; building relationships with cross-functional stakeholders; influencing others through technical explanations and examples; adapting to competing demands and new responsibilities; listening and responding to, seeking, and addressing performance feedback; providing feedback to others and managers; creating and executing plans to capitalize on strengths and develop weaknesses; supporting team collaboration; and adapting to and learning from change, difficulties, and feedback.
- As part of the IT Engineering job family, this position is responsible for leveraging DEVOPS, and both Waterfall and Agile practices, to design, develop, and deliver resilient, secure, multi-channel, high-volume, high-transaction, on/off-premise, cloud-based solutions.
- Provides insight into recommendations for technical solutions that meet design and functional needs.
- Translates business requirements and functional specifications into physical program designs, code modules, stable application systems, and software solutions by partnering withBusiness Analysts and other team members to understand business needs and functional specifications.
- Assists in identification of specific interfaces, methods, parameters, procedures, and functions, as required, to support technical solutions.
- Leads systems' incident support and troubleshooting for complex and non-complex issues.
- Supports component integration testing (CIT) and user acceptance testing (UAT) for application initiatives by providing triage, attending test team meetings, keeping the QC up-to-date, performing fixes and unit testing, providing insight to testing teams in order to ensure the appropriate depth of test coverage, and supporting the development of proper documentation.
- Builds and maintains trusting relationships with internal customers and third party vendors to ensure the alignment, buy-in, and support of diverse project stakeholders.
- Reviews and validates technical specifications and documentation.
- Identifies specific interfaces, methods, parameters, procedures, and functions to support technical solutions while incorporating architectural designs.
- Collaborates with architects and/or software consultants to ensure functional specifications are converted into flexible, scalable, and maintainable solution designs.
- Provides implementation and post-implementation triage and support of business software solutions by programming and/or configuring enhancements to new or packaged-based systems and applications.
- Reviews and makes changes to technical specifications and documentation.
- Develops and executes unit testing to identify application errors and ensure software solutions meet functional specifications.
- Writes technical specifications and documentation.
- Ensures new and existing software solutions are developed with insight into industry best practices, strategies, and architectures.
- Builds partnerships with IT teams and vendors to ensure written code adheres to company architectural standards, design patterns, and technical specifications.
- Leads, mentors, and trains other technical resources to develop software applications.
- Develops, configures, or modifies basic to moderately complex integrated business and/or enterprise application solutions within various computing environments by designing and coding component-based applications using programming languages.
- Participates and coaches others in all software development lifecycle phases by applying and sharing an in-depth understanding of company and industry methodologies, policies, standards, and controls.
- Assists with project estimation throughout the PLC.
- Develops and maintains specialist knowledge of database concepts, object and data modeling techniques and design principles, and a detailed knowledge of databasearchitectures, software, and facilities. Analyses data requirements to establish, modify, or maintain object/data models. Evaluates potential solutions, demonstrating, installing, and commissioning selected products.
- Takes responsibility for the accessibility, retrievability, and security of specific subsets of information. Provides advice on the transformation of information from one format/medium to another, where appropriate. Maintains and implements information handling procedures. Enables the availability, integrity, and search ability of information through the application of formal data structures and protection measures. Manipulates specific data from information services to satisfy local or specific information needs.
- Minimum three (3) years experience working on project(s) involving the implementation of solutions applying development life cycles (e.g., SDLC).
- Minimum one (1) year in a technical leadership role with or without direct reports.
- Bachelor's degree in Computer Science, CIS, or related field and Minimum six (6) years experience in software development or a related field. Additional equivalent work experience may be substituted for the degree requirement.
- One (1) year of work experience in a role requiring interaction with senior leadership (e.g., Director level and above)
- Three (3) years experience in systems analysis, including defining technical requirements and performing high level design for complex solutions.
- Three (3) years experience with engineering tools such as bug tracking and source code control systems.