Lead Azure data engineer - Santa Clara, United States - Omega Solutions

    Omega Solutions
    Omega Solutions Santa Clara, United States

    Found in: Lensa US 4 C2 - 1 week ago

    Default job background
    Description

    Responsibilities:
    Design, develop, test, and deploy Data Engineering pipelines on Azure cloud platform.

    Provide technical leadership and guide the team.

    Analyze and profile the data to understand and resolve the data quality issues.

    Participate in design and code reviews and provide guidance.

    Create near real-time data pipelines and document the design.

    Requirements:

    10+ years of overall IT experience in designing and implementing Data Engineering solutions.


    6+ years of experience primarily on Azure cloud platform using Azure BLOB Storage / ADLS, Azure Data Factory, Databricks, Synapse, Logic Apps and Key Vault.

    Must have worked as a Lead on at least 2 projects.

    Experience in coding SQLs and Python.

    Must have experience in Data Warehousing.

    Informatica Cloud experience is a plus.

    Experience in Cosmos, HDInsight, Event Hubs, Azure Functions, APIs is a plus.

    Experience in streaming data ingestion and stream data processing is a plus.

    Experience in Spark (Python, Scala, or SQL) is a plus.

    DataOps, or DevOps experience is a plus.

    Azure Cloud certification preferred.
    #J-18808-Ljbffr