Databricks Administrator - Birmingham, AL

Databricks Administrator · Job Summary: · Our client is growing and seeking to hire an experienced Azure Databricks Platform Administrator to join our talented Information Technology team. The position offers growth, flexibility and a collaborative work environment. · The Azure D ...

Job description

Databricks Administrator

Job Summary:
Our client is growing and seeking to hire an experienced Azure Databricks Platform Administrator to join our talented Information Technology team. The position offers growth, flexibility and a collaborative work environment.
The Azure Databricks Platform Administrator oversees the design, implementation, and maintenance of Databricks clusters, ensuring scalability, performance, and security. They collaborate with leadership to align platform capabilities with business goals and provide escalated support to users. Responsibilities also include enforcing policies, overseeing access controls, and automating admin tasks.
In this position you should have the following: excellent interpersonal skills with the ability to communicate at all levels. Strong problem solving and creative skills and the ability to exercise sound judgment. Most important, demonstrate a high level of integrity and dependability with a strong sense of urgency and results-orientation.
This role oversees the entire Databricks infrastructure, ensuring its scalability, performance, and security. They lead a team of administrators and align platform capabilities with business goals.
Responsibilities:

• Strong experience with Azure, Databricks and the enterprise tools they interact with (e.g Salesforce, Informatica)
• Oversee the design, implementation, and maintenance of Databricks clusters.
• Ensure the platform's scalability, performance, and security.
• Develop and enforce policies and procedures for platform administration.
• Collaborate with leadership to align platform capabilities with business goals.
• Provide escalated support and troubleshooting to users.
• Oversee maintenance of role-based access to data and features in the Databricks Platform using Unity Catalog.
• Oversee implementation of external access controls for outside teams using service principles, SQL warehouses and Delta Sharing
• Knowledge of Databricks development, including Azure Storage Technologies
• Support Application integration and security management
• Azure Cloud with Network security
• Excellent verbal/written communication, collaboration, analytical and presentation skills Job responsibility
• Provide operational support as Azure Databricks admin
• Responsible for user provisioning, access controls and group administration
• Manage –- Databricks clusters and permissions- Azure Data Storage, key vault, Tokens and secrets- Code deployment to production- Linked services, SQL endpoints
• Responsible for clusters health and Performance management
• Continuously improve processes and systems used to manage infrastructure, users and external access.
• Lead implementation of service principles and access tokens for external users.
• Oversee creation of queries and dashboards to monitor critical systems and processes.
• Review and maintain documentation for users and administrators.
• Automate common admin tasks using Databricks notebooks.
• Manage workflow tags, cluster tags, workflow naming convention enforcement.
• Review clusters health check and best practices implementation.
• Ensure regular backup and recovery.
• Oversee privilege reviews on users and resources.
• Stay current with industry trends and advancements in Databricks and cloud technologies.
• Lead and mentor a team of Databricks administrators.
• Provide training to junior members of the team and act as a point of escalation for complex questions and issues.

Experience:

• 9 years of relevant experience required.
  • Experience with Azure Databricks preferred.
• Bachelor's degree in related field required (combination of education and experience may be considered in lieu of degree)

Required Skills:

• Strong understanding of data engineering needs
• Strong understanding of Azure infrastructure required by Databricks
• Experience with compliance audits/audit documentation
• Experience using Databricks APIs with tools like Postman
• Understanding of the Databricks workspace and development environment
• Experience using Bitbucket to manage source code, and Understand Azure DevOps, managed code deployment and knowledge in CI/CD pipelines
• Strong SQL skills for creating admin related queries and dashboards
• Strong debugging skills
• Ability to do risk analysis for 3rd party integrations with Azure resources.
• Familiarity with Dynatrace.
• Proficiency in Spark, Python, SQL, and cloud platforms (AWS, Azure, or GCP).
• Strong knowledge with Python, PowerShell, pySpark
• Strong in Python and PowerShell scripting