DevOps Architect - Dallas-Fort Worth Metroplex
9 hours ago
Job description
- Candidates should be eligible to work for any employer in the United States without needing Visa sponsorship
- W2 ONLY position, cannot accept 1099 or C2C candidates for this role".
Job Title: Lead Azure Data Platform Operations Engineer/ DevOps Architect/ Databricks Admin
Duration: 11+ months contract
Fully remote, must be willing to work PST time zone.
Must have Skills: Azure, Terraform, IaC, and Databricks support.
Job Description
We are seeking an Azure Data Platform Operations leader who will be able to take strategic direction and implement the Ops parts that enable those strategies with a strong focus on automation and infrastructure as code to join our dynamic team. The successful candidate will be responsible for automating the operations, design, implementation, and maintenance of our Azure-based data platforms, including Databricks and other related Azure services. This role requires expertise in creating reproducible, scalable, and secure environments using infrastructure as code methodologies, particularly within the Azure cloud ecosystem.
Responsibilities:
Platform Automation and Infrastructure as Code:
- Design and implement infrastructure as code (IaC) using tools like Terraform to manage and provision Azure resources systematically and efficiently.
- Automate the setup, scaling, and management of Databricks workspaces and other Azure data services using CI/CD pipelines.
- Develop scripts and tools to automate operational tasks, reduce manual interventions and ensure consistent environments through code.
Platform Administration:
- Administer, configure, and optimize the Databricks platform and other Azure data services to support data analytics, machine learning, and data engineering activities.
- Manage Azure resources including Data Factories, Key Vaults, Storage Accounts, ML Workspaces, and Purview through automated scripts and IaC.
- Monitor, troubleshoot, and optimize resource utilization and platform costs using automated tools and techniques.
Operational Support:
- Ensure platform security, compliance, and reliability through automated patching, updates, and security configuration.
- Implement and manage access controls, security policies, and monitoring solutions to protect sensitive data across all Azure data platforms.
Collaboration and Technical Support:
- Work closely with data engineering, data science/ML, and application/integration teams to automate data workflows and optimize data delivery.
- Provide expert guidance on Azure best practices, especially regarding automation, scalability, and operational efficiency.
- Collaborate with DevOps and other technology teams to enhance infrastructure automation and continuous delivery efforts.
Data Management:
- Automate the management of schema data with Unity Catalog, including creation, configuration, cataloging, external storage, and access permissions.
- Develop and maintain automated solutions for data cataloging, metadata management, and governance to support data discovery and lineage tracking.
Skills and Qualifications:
Essential:
- 3+ years of hands-on experience with the Databricks platform and Azure PaaS.
- 3+ years of experience in infrastructure as code, preferably with Terraform, in a production environment.
- Strong knowledge of Azure cloud services and management of scalable cloud infrastructure.
- Proficient in scripting languages such as PowerShell, Python, or Bash for automation purposes.
- Solid understanding of security best practices, compliance frameworks, and risk management in the cloud.
- Proficiency in software development with Python
Preferred:
- Experience with CI/CD tools (e.g., Jenkins, Azure DevOps) for automating data and infrastructure workflows.
- Familiarity with containerization and orchestration technologies (e.g., Docker, Kubernetes) within Azure.
- Knowledge of real-time data processing technologies (e.g., Spark Streaming, Delta Live Tables, etc...).
Similar jobs
We're looking for an experienced Senior DevOps Engineer to lead infrastructure initiatives, optimize CI/CD systems, and mentor other engineers as we continue to scale our client's platform. · ...
1 month ago
We are seeking a senior Data Architect with deep Databricks expertise to join a global consulting firm. · Proven experience as a Data Architect or Senior Data Engineer with strong architectural ownership. · Deep hands-on expertise with Databricks, including Delta Lake, Delta Live ...
1 month ago
We are seeking a senior Solutions Architect with strong hands-on experience across modern frontend frameworks, · backend application frameworks, · and Azure cloud-native services. · This role requires a true polyglot architect who can design end-to-end digital solutions spanning ...
1 month ago
We are seeking a skilled Data Engineer with a minimum of 6 years of experience to join our team in Plano. · Pipeline Development: Design and deploy high-performance, scalable data pipelines to process large-scale datasets. · GCP Management: Architect solutions leveraging the full ...
1 week ago
The IT Engineer will be responsible for designing and developing modern data lakehouse solutions within the AWS ecosystem. · This role involves architecting scalable data platforms, building secure APIs, and ensuring performance, security, · and data integrity across all systems. ...
1 month ago
Optomi is building a new Site Reliability Engineering team focused on application domains and is seeking senior SRE engineers to ensure the reliability, performance, and availability of applications within each domain. · ...
1 month ago
Lead design and development of Java-based microservices using Spring Boot. · ...
2 weeks ago
We're seeking an experienced Director of Network Engineering to lead the architecture, design and transformation of our global network infrastructure. · We'll drive our evolution from traditional networking to a modern automation-first approach while building enterprise and hyper ...
3 weeks ago
We are seeking a versatile and highly skilled Full Stack Engineer to join our engineering organization. In this role the engineer will shape the traveler experience end-to-end building intuitive user interfaces while also crafting the backend services that power them. · Collabora ...
1 month ago
We are seeking a Senior Data Engineer to join our high-performing DataNext team, · blending deep data engineering, governance, and quality expertise to own mission-critical pipelines supporting healthcare financial data, · IDR processes, revenue cycle integrity, and enterprise an ...
3 weeks ago
We are looking for best-in-class talent to join our community and help us accomplish something extraordinary as we transform the payments industry. · ...
1 month ago
This is an opportunity to architect and run a world-class technology backbone that keeps design, manufacturing, and business systems humming 24 × 7. · Competitive executive base salary plus annual and long-term incentives. · Comprehensive medical, dental, vision, 401(k) with matc ...
1 month ago
This role is responsible for designing, implementing, and operationalizing security architecture and controls across multi-cloud environments. · Develop and maintain secure cloud architecture standards and reference designs for Azure, AWS, and GCP environments to support research ...
1 month ago
We are looking for a highly skilled Adobe AEM & Marketing Technology Architect to lead the design and implementation of scalable, secure, and high-performance digital experience solutions on the Adobe ecosystem. · Define and own the AEM solution architecture across author, publis ...
3 weeks ago
The organization seeks a Senior Java/AWS Developer to build secure enterprise APIs and cloud services that support authentication and common login capabilities. · Design develop test and deliver secure enterprise-grade RESTful APIs using Java and Spring Boot. · Collaborate within ...
1 month ago
We are seeking a Cloud Engineer with AWS expertise for a hands-on role focused on building and maintaining a highly reliable cloud platform. · ...
1 month ago
Aven Hospitality · is seeking a skilled · Principal Cloud Engineer · with expertise in Google Cloud Platform (GCP), Terraform, and Infrastructure as Code (IaC) to design, implement, · & manage scalable, secure, · & reliable cloud infrastructure. · The ideal candidate will have ha ...
3 weeks ago
We are hiring a hands‑on SharePoint Architect to own the design, build, and operational support of our SharePoint Online estate following a recent Microsoft 365 migration. · Remediate Power Automate flows: diagnose and rewrite broken flows and expressions migrated from legacy sys ...
1 week ago
The Technical Coach will elevate engineering capabilities, practices and architecture to enable faster and more reliable software delivery with a focus on measurable business outcomes. · ...
1 month ago
We're seeking an experienced Senior Staff Engineer, · DevOps to lead our cloud infrastructure and DevSecOps initiatives. · Lead technical and strategic initiatives for a line of business, · developing cohesive approaches across all projects · ,Mentor junior engineers and foster a ...
1 week ago