6+ years of overall experience, including 3+ years relevant to Databricks and Cloud Administration
Proficient in managing Databricks environments and user access, including Unity Catalog
Strong background in monitoring and optimizing Big Data resources
Hands-on experience with containerization technologies like Docker
Expertise in scripting and automation using Terraform, YAML, and cloud APIs
Responsibilities
Build and monitor complex Big Data platforms on Databricks and ElasticSearch.
Develop and optimize Data science hosting platforms based on Databricks technologies.
Define and implement technological strategies for our Data platform in the Cloud.
Design architectural frameworks and define strategic roadmaps for future developments.
Implement observability solutions including monitoring and alerting tools such as Zabbix and Datadog.
Enhance and manage CI/CD pipelines while staying current with industry advancements.
Automate infrastructure and deployment processes through various scripting languages.
Benefits
Collaborative work environment with opportunities to influence platform development
Access to cutting-edge tools and technologies for Big Data management
Opportunities for professional development and staying updated on industry trends
Impactful role in shaping cloud strategies and solutions
Flexible work arrangements supporting work-life balance
Full Job Description
Sr. Software Engineer MAHIN-JOB-37408 Location: Santa Clara, CA Job Description :- Databricks Administrator with Cloud Administration expertise
Overall Exp. :- 6+ Years
Relevant exp.:- 3+ Years
We are looking for a Databricks Administrator with Cloud Administration expertise inside and out.
Responsibilities
The candidate will work on building, scaling, and monitoring highly complex BigData platform on Databricks, Snowflake DB and ElasticSearch cloud.
The candidate will work on building, scaling, and monitoring Data science and Client hosting platform based on Databricks.
The candidate will be responsible for defining and implementing our Data platform, Data science workbench with Client hosting technology strategies in the Cloud.
The candidate will also involve to design our architecture and define our roadmap.
The candidate will be responsible to implement monitoring, alerting and observability solutions using CSP standard monitoring and analytical tools and also 3rd party tools like Zabbix and Datadog SaaS.
The candidate has to be an expert in Docker and Cloud based containerization technologies.
The candidate will also be responsible for building and improving our CI/CD pipelines and keeping themselves and their team up-to-date with the latest industry trends and technologies.
Develop automation scripts in Terraform, YAML, Helm charts, az cli, aws-cli, Powershell and other cloud APIs.
Participate in developer interaction calls and help.
Participate in Management update calls and provide precise, timely updates.
Skillset:
Operations or systems administration experience, particularly on Linux.
Hands-on administration experience, from setting up the Databricks environment to successfully administering it.
Experience as the Databricks account owner, managing workspaces, Unity Catalog, audit logs, and high-level usage monitoring
Experience as Databricks workspace admin, managing workspace users and groups including single sign-on, provisioning, access control, and workspace storage.
Experience Databricks administration configuring and installing libraries
Experience managing storage accounts access across a large user base
Experience managing cluster and jobs, policies, templates, pools configuration options
Experience with Databricks security and privacy setup
Experience troubleshooting end user and platform-level issues
Experience optimizing usage for performance and cost preferred
Experience in strategizing and implementing Disaster recovery for BigData platforms based on Databricks.
Ability to multitask and reprioritize tasking on the fly according to the needs of a growing platform and its stakeholders
Strong coding skills in SQL and Python (PySpark) with experience optimizing code
Assist in N/W, Data security, IAM initiatives. Work with n/w team to create PEs, troubleshoot network issues etc.
