Database Engineer - DatabricksLocation: Hybrid - Irvine, CAJob Type: Full-TimeOverview:Mutual of Omaha Mortgage is seeking a
Database Engineer to own the ongoing operation, reliability, and evolution of our Databricks Lakehouse platform following the completion of a major data migration initiative.
This is a hands-on engineering role focused on keeping our Databricks-based ingestion pipelines, transformation layers, and warehouse healthy, performant, cost-efficient, and compliant in a regulated lending environment.
You'll work on a small, senior platform team where you'll have direct ownership of the Databricks environment with high visibility and autonomy. This is a role for someone energized by being a primary owner rather than one specialist among many.
Key Responsibilities: - Own the day-to-day operation and reliability of Databricks jobs and pipelines, compute and cluster configuration, Delta Lake tables, and Databricks SQL warehouses.
- Develop and optimize Spark transformations in Databricks notebooks across bronze/silver/gold (medallion) layers, including incremental loads via Auto Loader and Structured Streaming where applicable.
- Maintain and extend orchestration across Databricks Workflows and Apache Airflow, ensuring freshness and reliability SLAs are met.
- Administer governance and access through Unity Catalog (catalogs, schemas, grants, lineage) consistent with PII containment in a regulated lending environment.
- Manage and tune Microsoft SQL Server (MSSQL) source and target databases for downstream consumer with a focus on query optimization, indexing, schema management, and performance diagnostics.
- Build monitors and control DBU consumption and cluster cost by implementing alerting and right-sizing to keep platform spend predictable.
- Diagnose and resolve pipeline failures, data quality issues, schema drift, and source authentication/connectivity problems end-to-end.
- Share responsibility for platform reliability and operational coverage with a small team, maintaining runbooks and documentation so that no single person is a point of failure.
- Document pipeline architecture, runbooks, and operational procedures to support team-wide maintainability.
Required Skills & Qualifications - 3+ years of hands-on data engineering experience in a production environment.
- Deep, demonstrable Databricks experience, including:
- Authoring and operating Databricks notebooks, Jobs, and Workflows
- Cluster and compute configuration, plus DBU cost awareness
- Databricks SQL / SQL Warehouses
- Unity Catalog governance
- Databricks Certification
- Strong Python, PySpark, SQL skills for building and debugging distributed data transformations.
- Solid MSSQL / SQL Server experience
- Production experience with Apache Airflow (authoring, scheduling, and debugging DAGs).
- Experience integrating data from SaaS/CRM source systems (e.g., Salesforce) via Bulk or streaming APIs.
- Comfort operating, monitoring, and troubleshooting pipelines in a live environment, not just building them.
- Strong communication skills and the ability to articulate and write clear technical documentation.
The annual salary for this position is $130,000-$150,000. This position offers eligibility for incentive compensation.
We offer an extensive compensation package. Benefits include:
- Vacation - Vacation hours accrue on a per pay period basis. Balance maximums are based on years of service.
- Paid Sick Time - 40 hours of paid sick time after 90 days of employment. State sick time requirements will be added as needed.
- 9 Paid Holidays + 1 Cultural Celebration Day (Floating Holiday)
- Multiple PPO Medical Plans, as well as HDHP eligible plan.
- Dental Coverage
- Vision Coverage
- Company Paid Life Insurance
- 401K with a generous employer match
- Additional Benefits including - Optional Life, FSA, Pet Insurance etc.
- Free Legal Services
- Employee Loan Program
