Bachelor's degree or equivalent practical experience.
Expertise in distributed computing frameworks for large-scale data processing.
Familiarity with AWS or other cloud environments.
Experience with datastores like PostgreSQL, S3, Redshift, and MongoDB.
Proficient in Python and related libraries like pandas and PySpark.
Working knowledge of Docker, Kubernetes, and messaging systems like Kafka.
Experience with Agile development methodology.
Responsibilities
Interpret business needs and select appropriate technologies.
Create, maintain, and optimize data pipelines for seamless data flow.
Develop internal process improvements for scalability and automation.
Integrate data pipelines with AWS cloud services for insights extraction.
Work with stakeholders to support data infrastructure and troubleshoot issues.
Design robust dataflows to support both raw and expected data.
Provide Tier 3 technical support for deployed applications and dataflows.
Benefits
Opportunity to collaborate with key business stakeholders and IT experts.
Involvement in complex projects that push technology boundaries.
Chance to work in a fast-paced Agile environment.
Access to a team-focused work culture that prioritizes innovation.
Full Job Description
As a Data Engineer, you will be required to interpret business needs and select appropriate technologies and have experience in implementing data governance of shared and/or master sets of data. You will work with key business stakeholders, IT experts, and subject-matter experts to plan and deliver optimal data solutions. You will create, maintain, and optimize data pipelines as workloads move from development to production for specific use cases to ensure seamless data flow for the use case. You will perform technical and non-technical analyses on project issues and help to ensure technical implementations follow quality assurance metrics. You will analyze data and systems architecture, create designs, and implement information systems solutions.
Identify, design and implement internal process improvements including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes.
Develop and design data pipelines to support an end-to-end solution.
Develop and maintain artifacts (e.g. schemas, data dictionaries, and transforms related to ETL processes).
Integrate data pipelines with AWS cloud services to extract meaningful insights.
Work with stakeholders to support their data infrastructure needs and assist with data-related technical issues.
Design and develop robust and functional dataflows to support raw data and expected data.
Provide Tier 3 technical support for deployed applications and dataflows.
Define and communicate a clear product vision for our client's software products, aligning with user needs and business objectives.
Create and manage product roadmaps that reflect both innovation and growth strategies.
Partner with a government product owner and a product team of 7-8 FTEs.
Collaborate with the rest of data engineering team to design and launch new features.
Coordinate and document dataflows, capabilities, etc.?
Occasionally (as needed) support to off-hours deployment such as evening or weekends.
Requirements
Bachelor's degree or equivalent practical experience.
Expertise in distributed computing frameworks to handle large-scale data processing.
Familiarity working with Amazon Web Managed Services (AWS) or any other cloud environments.
Working experience with datastores like PostgreSQL, S3, Redshift, MongoDB/DynamoDB, Redis, Elasticsearch/OpenSearch and SQL.
Proficient utilizing Python with key libraries like pandas and PySark, NiFi, Airflow, AWS Lambda or similar technologies.
Working knowledge with software platforms and services, such as, Docker, Kubernetes, JMS/SQS, SNS and Kafka.
Familiar with Linux/Unix server environments.
Experience with Agile development methodology.
Publishing and/or presenting design reports.
Coordinating with other team members to reach project milestones and deadlines.
Working knowledge with Collaboration tools, such as, Jira and Confluence.
We provide a highly competitive benefits package to include: extensive medical/dental/vision, 7% of your annual salary toward 401(k), Paid Time Off (PTO), $5K annually toward ongoing education and training, and more. We also have monthly social and tech events!
Salary Description
$140,000 to $170,000
About OneGlobe
OneGlobe LLC is a software development and consulting company that provides innovative solutions to federal and commercial clients. The company specializes in cloud computing, big data analytics, and enterprise application development. OneGlobe has a team of experienced professionals who are dedicated to delivering high-quality services and products to their clients. The company is headquartered in Herndon, Virginia.
