Databricks Lead/Architect - Denver - Merican

    Merican
    Merican Denver

    1 week ago

    Description
    Databricks Lead/Architect
    Denver, CO - Hybrid
    W2 role - Visa independent candidates only.
    Job Overview
    We are seeking highly skilled and experienced Hands-on Databricks Architect to design,
    build, and optimize our modern data streaming platform. This platform, built on Azure and
    leveraging the Databricks Lakehouse, is central to our Data Availability Initiative. It provides
    comprehensive data ingestion, real-time streaming, batch processing, analytics, and AI/ML
    capabilities. The ideal candidate will be a technical lead with deep expertise in Databricks,
    Azure cloud services, and a strong understanding of data architecture principles like the
    Medallion Architecture and Data Mesh. This role requires a hands-on approach to
    implementing solutions, driving robust data governance through Unity Catalog, and
    integrating advanced AI/ML models.
    Key Responsibilities
    • Lead the architectural design, development, and implementation of scalable and
    • robust data solutions on the Databricks Lakehouse Platform within an Azure
    • environment.
    • Design and implement a robust multi-workspace strategy by manually configuring
    • and automating the deployment of Development, QA, and Production Databricks
    • environments.
    • Standardize environment setup using Terraform, ensuring consistent, repeatable,
    • and version-controlled infrastructure across the entire SDLC.
    • Architect and implement data ingestion strategies for real-time streaming (using
    • Debezium, Apache Kafka, Confluent Platform) and batch processing, leveraging
    • Databricks Lake Flow and Auto Loader.
    • Design and develop data pipelines following the Medallion Architecture (Lead,
    • Bronze, Silver, Gold tiers) using Databricks Delta Live Tables (DLT) and Structured
    • Streaming, ensuring data quality, lineage, and governance.
    • Optimize Databricks clusters, Delta Lake tables, and Spark jobs for performance,
    • scalability, and cost efficiency, including leveraging auto-scaling, spot instances,
    • compression, and partitioning techniques.
    • Establish and enforce comprehensive data governance, security, and access
    • control policies utilizing Databricks Unity Catalog and Azure Purview across multi tenant workspaces and business units.
    • Develop and integrate MLOps pipelines using Databricks MLflow and Azure Machine
    • Learning for model training, serving, and monitoring, including custom Large
    • Language Model (LLM) integrations.
    • Collaborate with business stakeholders to translate complex data requirements into
    • technical designs and scalable solutions.
    • Implement CI/CD practices for Databricks solutions using Databricks Asset
    • Bundles, Terraform, GitHub, and GitHub Actions, ensuring automated deployment
    • and version control.
    • Integrate data from diverse sources including SQL databases (PostgreSQL, SQL
    • Server), MongoDB, Snowflake, and various flat/multimedia files into the data
    • platform.
    • Design and implement API-first data consumption patterns, serverless functions
    • (Azure Functions), and application integrations for various data products and real time services.
    • Provide technical leadership, mentorship, and best practices to data engineering
    • and development teams.
    • Contribute to the continuous evolution of the data platform, incorporating future
    • considerations such as the Databricks Ingestion Gateway, advanced AI integration,
    • and Data Mesh evolution.
    Required Skills
    Databricks Lakehouse, Apache Spark, Delta Live Tables (DLT), Auto Loader, Microsoft
    Azure, ADLS Gen2, Azure Functions, Azure Machine Learning, Apache Kafka, Confluent
    Platform, Debezium (CDC), Databricks Unity Catalog, Azure Purview, Python, SQL, Scala
    (Advantage), Databricks Asset Bundles, Terraform, GitHub Actions, Medallion Architecture,
    Data Mesh, PostgreSQL, MongoDB, Snowflake, MLOps, LLM Integration, Azure Networking
    Qualifications
    • Bachelor's or Master's degree in Computer Science, Data Engineering, or a related
    • quantitative field.
    • 8+ years of progressive experience in data architecture, data engineering, or a
    • related role within an enterprise environment.
    • 5+ years of hands-on experience specifically with the Databricks Lakehouse
    • Platform, including designing and implementing complex data solutions.
    • Proven track record of successfully designing and implementing large-scale, high performance, and cost-optimized data platforms on Microsoft Azure. Databricks Certified Data Engineer (Associate or Professional) and/or Azure Data
    • Engineer Associate (DP-203) certifications are highly desirable.
    • Excellent communication, presentation, and interpersonal skills with the ability to
    • influence technical and non-technical stakeholders effectively.

  • Databricks Lead/Architect

    3 weeks ago

    Only for registered members Denver

    We are seeking highly skilled and experienced Hands-on Databricks Architect to design, build, and optimize our modern data streaming platform. · This platform, built on Azure and leveraging the Databricks Lakehouse, is central to our Data Availability Initiative. It provides comp ...

  • Databricks Lead/Architect

    2 weeks ago

    Only for registered members Denver

    We are seeking highly skilled and experienced Hands-on Databricks Architect to design, · build, and optimize our modern data streaming platform. · Lead the architectural design, development, and implementation of scalable and · robust data solutions on the Databricks Lakehouse Pl ...

  • Databricks Lead/Architect

    1 month ago

    Only for registered members Denver

    We are seeking highly skilled and experienced Hands-on Databricks Architect to design, · build and optimize our modern data streaming platform. · This platform built on Azure leveraging the Databricks Lakehouse is central to our Data Availability Initiative It provides comprehens ...

  • Databricks Architect

    1 month ago

    Only for registered members Denver

    + Job summary: We are seeking highly skilled Databricks Architect to design, build, and optimize our modern data streaming platform. This platform, built on Azure and leveraging the Databricks Lakehouse, is central to our Data Availability Initiative. · + Qualifications:Bachelor ...

  • Databricks + AI Architect

    1 month ago

    Only for registered members Denver

    + AI/ML experience + Transform legacy platform into future-ready, scalable cloud-native Databricks-based architecture + Lead architectural modernization from on-prem/legacy platform to unified Databricks Lakehouse ecosystem · + Design and implement high-performance data pipelines ...

  • Azure Databricks Architect

    1 month ago

    Only for registered members Denver, CO

    We are seeking an Azure Databricks Architect to design and implement scalable data architectures using Azure Databricks. · Design and implement scalable data architectures using Azure Databricks · Build and optimize ETL/ELT pipelines using Azure Data Factory · ...

  • Databricks Architect with AI/ML

    1 month ago

    Only for registered members Denver, CO

    Databricks Architect with AI/ML experience required · ...

  • Databricks Architect with AI/ML

    1 month ago

    Only for registered members Denver

    We are seeking an experienced Databricks Architect with AI/ML enablement expertise to design and implement cloud-native data architectures. · Design and implement cloud-native data architectures. · Develop deep hands-on experience with Apache Spark and Delta Lake. · Pipeline inte ...

  • Azure Databricks Engineer/Architect

    4 days ago

    Only for registered members Denver

    The Azure Databricks Engineer implements and manages the operational aspects of cloud-native and hybrid data platform solutions built with Azure Databricks.Candidates for this role should understand all concepts covered in Azure Data Engineer Associate, Azure Database Administrat ...

  • Databricks Architect

    2 weeks ago

    Only for registered members Boulder Full time $159,100 - $238,700 (USD)

    The Databricks Architect will be responsible for technical decision-making, code quality, and the successful delivery of complex data projects using Delta Lake, Unity Catalog. · ...

  • Databricks Architect

    2 weeks ago

    Only for registered members Boulder, CO

    The Databricks Architect will be responsible for technical decision-making code quality and successful delivery of complex data projects using Delta Lake Unity Catalog and Mosaic AI. · The Databricks Architect/Lead owns end-to-end strategy for data ingestion storage and analytics ...

  • Lead Data Architect

    1 month ago

    Only for registered members Denver $115,000 - $200,000 (USD)

    +Job summary · CapTech Data Architects match clients' business goals with available technologies when developing data delivery implementation. · +Responsibilities · ...

  • Java Architect or Lead

    2 days ago

    Only for registered members Denver

    We are seeking a highly skilled Java Architect to design, build, and guide the development of scalable, · high-performance enterprise applications.The ideal candidate will have deep expertise in Java, · Spring Boot, Microservices architecture, · AWS cloud services, · REST APIs, · ...

  • Lead Solution Architect

    1 month ago

    Only for registered members Denver $118,450 - $236,900 (USD)

    We are looking for an outcome-driven Lead Solution Architect who will be working with design, research, product management and engineering teams to translate business requirements into architecture runways for application development plans which enables innovative digital solutio ...

  • Lead Solution Architect

    1 week ago

    Only for registered members Denver $118,450 - $236,900 (USD)

    We're building a world of health around every individual — shaping a more connected, · convenient and compassionate health experience.CVS Health is looking for an outcome-driven Lead Solution Architect who will be working with design, · research, product management and engineeri ...

  • Lead Platform Architect

    3 days ago

    Only for registered members Denver, Colorado, , United States

    The Lead Platform Architect will have a deep understanding of cloud technologies and experience in architecting scalable and resilient infrastructure solutions. · ...

  • Bigdata Lead/Architect

    4 days ago

    Only for registered members Denver, CO

    This job requires experience with Hadoop Cloudera HortonWorks MapR ecosystem components No SQL databases like HBase Mongo Cassandra etc Requirements Gathering designing developing scalable big data solutions with Hadoop HBase Hive Sqoop Flume Java Pig Python etc Good knowledge of ...

  • Lead Platform Architect

    4 days ago

    Only for registered members Denver, CO

    The Lead Platform Architect will have a deep understanding of cloud technologies DevOps principles and experience in architecting scalable resilient infrastructure solutions. · This role will be instrumental in designing implementing cutting-edge DevOps platforms to support our c ...

  • Test Lead/ Architect- NO C2C

    1 month ago

    Only for registered members Denver

    Test architect will serve · as the technical and quality authority · for assigned products and features guiding · testing strategy from requirements through release. · Champion testing best practices · Mentor testers · ...

  • Test Lead/ Architect- NO C2C

    1 month ago

    Only for registered members Denver, CO

    This is a Test Architect position that will serve as the technical and quality authority for assigned products and features. · Location - Denver/St Louis. · ...

  • SAP SD Lead/ Architect

    1 month ago

    Only for registered members Denver

    SAP SD Lead/Architect job opportunity at Biz Fusion Inc. with Dice as the career destination for tech experts. · System configuration · Create Functional Specifications for custom developments · Carry out Unit Test (Functional) and Feature Integration Test and Product Integration ...

Jobs
>
Denver