Bachelor's degree in a relevant field; minimum two years of experience or equivalent education and training.
Strong T-SQL skills for complex queries and transformations.
Hands-on experience with data integrations across various sources.
Proficient in Python or similar scripting languages for data processing.
Ability to translate business requirements into technical models and communicate effectively.
Responsibilities
Design, build, and maintain data pipelines in Microsoft Fabric.
Develop data transformation logic using Python and PySpark.
Optimize T-SQL code for data processing and reporting.
Apply data modeling best practices for performance and usability.
Monitor and troubleshoot data pipeline runs to ensure timely delivery.
Assess and plan migration of SQL Server databases to Microsoft Fabric.
Build and maintain semantic models for Power BI reporting.
Benefits
Comprehensive insurance options including medical, dental, and vision.
401K plan with immediate matching contributions.
Annual discretionary bonus.
Generous paid time off and holidays.
Employee Assistance Program (EAP) for confidential support.
Wellness and company events throughout the year.
Full Job Description
Overview
WE ARE HIRING!
Data Engineer II
The Data Engineer II will be responsible for designing, building, and maintaining the organization's data platform on Microsoft Fabric. This includes developing data pipelines using Data Factory, Dataflows Gen2, and PySpark notebooks; writing and optimizing T-SQL. Assists with the migration of on-premises SQL Server databases and ETL/ELT processes to Fabric. Build semantic models that support Power BI reporting, ensuring data quality and governance, and managing changes through source control and deployment pipelines. Works closely with business users and stakeholders, communicating technical information in a clear and accessible way.
Responsibilities
Data Pipeline Development
Design, build, and maintain data pielines in Microsoft Fabric using Data Factory pipelines, Dataflows Gen2, and notebooks to move data from source systems into Lakehouses and Warehouses.
Develop data transformation and processing logic in Python and Pyspark within Fabric notebooks, including working with Delta tables and handling large or complex datasets.
Develop and optimize T-SQL code, including stored procedures, views, and transformation logic, to support data processing and reporting needs.
Apply data modeling best practice, such as medallion architecture and star schema design, to organize data in OneLake for performance and ease of use.
Monitor pipeline runs, troubleshoot failures, and put alerting and error handling in place so that data is delivered on time and accurately.
SQL Server to Fabric Migration
Assess existing on-premises SQL Server databases, SSIS packages, and scheduled jobs to plan their migration to Microsoft Fabric.
Rebuild or refactor legacy ETL processes as Fabric-native pipelines and PySpark notebooks, improving performance and maintainability.
Coordinate with the team and stakeholders on migration timelines to minimize disruption to reporting and operations.
Semantic Model Support
Build and maintain semantic models that serve as the foundation for PowerBI reports, including relationships, measures, ontology, and Direct Lake or Import configurations.
Partner with report developers and business users to understand reporting requirements and turn them into well-structured datasets.
Tune data models and queries to improve report performance and refresh reliability.
Communicate technical concepts, data limitations, and project status clearly to non-technical stakeholders, helping them understand and make effective use of available data.
Data Quality, Governance, and Operations
Implement data quality checks and validation rules to catch issues before they reach end users.
Manage workspace organization, access permissions, and security settings in Fabric in line with company data governance standards.
Use source control (Azure DevOps/Git) and deployment pipelines to manage changes across development, test, and production environments.
Create and maintain clear documentation of data sources, pipelines, models, dictionaries, and business logic.
Qualifications
Education and Experience
Bachelor's degree from a four-year college or university; minimum two years of related experience and/or training; or an equivalent combination of education and experience.
Work Location: Onsite - Candidates must be able to work full-time from our San Antonio office.
Required Technical Skills
Demonstrated ability to design relational and dimensional data models, including fact and dimension tables, keys, relationships, referential integrity, and historical changes.
Strong T-SQL skills for complex joins, transformations, query tuning, and troubleshooting.
Hands-on experience building and supporting integrations across databases, APIs, or file-based sources, including scheduling, incremental processing, and recovery.
Ability to validate data completeness and accuracy, reconcile source and target results, and investigate duplicates and broken relationships.
Experience using Python or a comparable scripting language for data processing or automation, and version control for maintainable development.
Ability to translate business requirements into documented models and explain technical decisions clearly; willingness to learn unfamiliar platforms.
Preferred Experience (Preferred)
Power BI semantic models, Power Query, DAX, or comparable business intelligence tools.
Microsoft SQL Server, SSIS, SSAS, or comparable database and integration technologies.
Microsoft Fabric, Azure Data Factory, Azure Data Lake Storage, Azure SQL, Synapse Analytics, Databricks, or Azure DevOps.
Oil and gas production data, SCADA, telemetry, time-series data, IoT or Streaming data.
Your benefits
Lewis Energy Group appreciates the hard work and dedication you bring to our team every day. To do our part, we are committed to keeping your benefits affordable and beneficial for you and your eligible family members. Lewis Energy Group strives to provide benefits that meet your needs, are easy to understand and use, and provide excellent value for affordable costs.
Insurance: Medical/Dental/Vision/ Basic Life and AD&D/and other benefit options to meet your personal needs
Retirement: 401K with immediate matching contribution
Annual Discretionary Bonus
Generous Paid Time Off and Paid Holidays
Employee Assistance Program (EAP) - Free, confidential support for you and your dependents
