POSITION TITLE: Data Engineer II

Supervised by: Data Engineer III - Team Lead

Supervises: N/A

FLSA Status: Full-time, Exempt/Salaried

POSITION SUMMARY:

The Data Engineer II is responsible for providing various database extracts of discrete and non-discrete data (i.e. documents, images, attachments, etc.) from various legacy applications. The DE II is also responsible for preparing and loading data into a Microsoft SQL Server environment. Additional activities for this position include transformation of discrete data, conversion of documents and images into formats supported by MediQuant software, and data analysis/quality assurance reviews of incoming data sets to ensure completeness and accuracy of the data.

In addition to the responsibilities of the DE I position, the DE II is responsible for mentoring the DE I team members, contributing to the improvement of extraction and load processes, and having a higher level of understanding of legacy system databases.

JOB DUTIES and ESSENTIAL FUNCTIONS. A qualified individual must be able to perform these essential functions of the job as listed, with or without accommodation. (An Asterisk (*) Identifies an Essential Function).

1. Extracts all data from legacy database systems. (*)
   1. Using existing software tools, can establish connectivity to client legacy database
   2. Kicks off extract process and monitors process during runtime
   3. Uses Structured Query Language (SQL) commands to browse and view database before extraction, as well as queries database-level statistics to help estimate the volume of data to be extracted
   4. Compares the layout of the legacy database with the layout of the extraction to ensure that all tables and expected row counts match
   5. Performs quality assurance checks on extractions to ensure work is accurate and reliable before it is passed on to the next team.
2. Loads various discrete and non-discrete data sources into a Microsoft SQL Server environment. (*)
   1. Uses existing tools to load flat file data into a SQL Server Database.
   2. Converts different file types, documents or images into PDF format.
3. Reviews incoming datasets to check for errors and ensure validity. (*)
   1. Acts as a second validator after the person who extracted the data; reviews the database layouts and verifies that no data is missing from the legacy system.
   2. Uses MQ scoping language from the contract to determine if the dataset delivered falls within the expected scope, or if the dataset delivered does not match the expected deliverable.
   3. Collaborates with both the Client/Extractor and the DM/IC to ensure a smooth transition of the data between the extraction and the modeling.
   4. Responsible for locating data and communicating to the team as the location of the data changes during the review process.
   5. Runs accurate, detailed comparisons between table lists from the legacy system and the extracted dataset. Works with other team members to compare to past extract projects throughout MediQuant history.
4. Troubleshooting and Problem Solving. (*) - Identifies problems and proactively intervenes to mitigate or eliminate potential for negative impact.
   1. Effectively checks work for accuracy, understanding where opportunities for errors exist. Takes ownership to ensure own work is error-free.
   2. Independently researches and problem-solves; recognizes when additional help is needed and pro-actively reaches out to mentors/leaders for help.
   3. Uses strong data analysis skills to debug data anomalies.
   4. Works collaboratively with project teams to innovate and find solutions to complex issues
   5. Documents issues and resolutions for and "lessons learned" to avoid repeating
   6. Applies technical knowledge and seeks to fully understand the client's expectations by asking questions.
   7. Liaise with application/tool developers, PMO, IT, or other sources to create tools needed to streamline workflows
5. Organization and Time Management. (*) - Handles multiple projects and tasks and prioritizes deadlines.
   1. Identifies and utilizes all resources available when priorities conflict or when external challenges are lining up against the deadline. Must be able to clearly communicate the issues to the PM and Lead
   2. Handles a fluctuating workload and can prioritize during times of peak demand and conflicting priorities. Knows when to seek assistance to ensure deadlines are met and quality is delivered
   3. Keeps organized notes and uses project management tools to keep project team updated with the latest statuses and roadblocks
   4. Attributes 90% of available time to billable initiatives for named clients, making sure that time is properly allocated on their timesheet.
6. Mentoring. (*)
   1. Actively mentors DE I team members, teaching the extract and load processes and assisting in issue resolution
   2. Works closely with Lead to understand training plan and goals for each DE I and supports plan execution
   3. Develops documentation during own work to explain steps for resolving errors and troubleshooting extracts; Makes documentation available to team, including DE I team members, to help them learn
7. Process Improvement. (*)
   1. Works alongside DE III to actively improve and refine the extraction and load processes
   2. Examines codebase of existing MediQuant tools and edits new functionalities or debugs discovered issues during tool use
   3. Assists DE III in developing standard procedures and processes; supports rolling out new procedures through mentoring

QUALIFICATIONS

Competencies

• Leadership Skills
• Collaboration Skills
• Organizational Skills
• Interpersonal and Communication Skills
• Critical Thinking
• Problem Solving/Analysis
• Team Player
• Self-Motivated/Self-Starter
• Time Management

Required Education and Experience

• Bachelor's degree in Computer Science, Computer Engineering, Software Engineering, preferred. High School Diploma or GED with 3-5 years of applicable experience in computer science related field required.
• Programming Skills
  • Advanced knowledge of SQL Scripting skills, able to query and transform data
  • Proficiency in one of the following programming languages: Python, Java, C#
  • Familiar with PowerShell/Bash scripting
• Excellent organizational and documentation skills
• At least 1 year of experience in Healthcare IT or clinical environments, familiar with healthcare terminology
• Advanced knowledge of healthcare data terminology used in both Clinical and Financial systems to help with scope review and understanding the datasets that are under review. Experience working with healthcare datasets preferred.

Additional Eligibility Qualifications

Preferred Experience with Legacy Systems (for example, but not limited to):

• Meditech (Magic, Client Server, 6.0, etc.)
• Cerner (MS4, Siemens Invision, Millennium, Soarian, etc.)
• McKesson (Star, Series, Paragon, etc.)
• Allscripts (Pro, Touchworks, etc.)
• GE Centricity
• NextGen
• CPSI
• eClinicalWorks
• SunQuest
• Medhost

Preferred Experience with Databases (for example, but not limited to):

• Intersystems Cache
• Sybase
• PostgreSQL
• IBM DB2
• Oracle
• MySQL
• Microsoft SQL Server
• Detailed knowledge of common database error messages, able to identify extract errors and derive a solution through either research or previous knowledge
• Able to understand database catalog, schema, and table layouts
• Able to work with large directories of files, extract metadata stored in the directory listing, and export resulting output in a tabular format, linking the metadata to the corresponding file path
• Knowledge of scripting tools (i.e. Python, Java, C#, PowerShell/Bash, etc.) and ability to write scripts to read, write, copy, rename, move, sort, or otherwise process many non-discrete data files

Proficient with ODBC drivers and is able to set up ODBC connectivity to various database types

Ability to solve problems using logical thought processes and devising creative solutions

High attention to accuracy and detail

Ability and desire to work in a strong team culture

Knowledge with HIPAA regulations involving PHI and HITECH

Strong customer service and information technology acumen

Other Duties. Please note this job description is not designed to cover or contain a comprehensive listing of activities, duties or responsibilities that are required of the employee for this job. Duties, responsibilities, and activities may change at any time with or without notice.

Work Environment. This job operates in a professional office environment. This role routinely uses standard office equipment such as computers, phones, photocopiers, filing cabinets and fax machines. While performing some duties of this job, the employee is exposed to dust or other airborne particles. The noise level on the job site is typical of an office environment.

Physical Demands. While performing the duties of this job, the employee is regularly required to talk or hear. The employee frequently is required to sit for long periods of time, stand; walk; use hands to finger, handle or feel; and reach with hands and arms. The employee is occasionally required to climb or balance; and stoop, kneel, crouch or crawl. The employee must occasionally lift and/or move up to 10 pounds and occasionally lift and/or move up to 25 pounds. Specific vision abilities required by this job include close vision, distance vision, color vision, peripheral vision, depth perception and ability to adjust focus.

Position Type and Expected Hours of Work. This is a full-time position, and hours of work and days are typically Monday through Friday, 8:30 a.m. to 5 p.m. Some flexibility in hours is allowed, but the employee must be available during the "core" work hours of 9:00 a.m. to 3:30 p.m. and must work 40 hours each week to maintain full-time status. Occasional evening and weekend work may be required as job duties demand.

Travel. Little to no travel is expected for this position.