Jobs
>
Seattle

    Direct Client – Sr Lead Data Engineer Apache Spark, Spark Core, Spark SQL– Data Bricks - Seattle, United States - Chateck Inc

    Chateck Inc
    Chateck Inc Seattle, United States

    6 days ago

    Default job background
    Description
    Direct Client - Sr Lead Data Engineer Apache Spark, Spark Core, Spark SQL, Spark Streaming – Data Bricks

    Location:
    Hybrid (Seattle, WA)

    Length: 12+ Months

    Job description
    This position contributes to Client success by building enterprise data services for analytic solutions.

    This position is responsible for design, development, testing and support for data pipelines to enable continuous data processing for data exploration, data preparation and real-time business analytics.

    Models and acts in accordance with Client guiding principles.

    Tops 3 Skills Needed

    1

    Data Engineering

    5+ years

    2

    Data Bricks

    5+ years

    3

    Sparks/ Dev ops practices

    5+ years


    Years of Experience:
    5+ years


    Technology requirements:
    Proficiency in Apache Spark, including Spark Core, Spark SQL and Spark Streaming.
    Proficiency in languages such as Python for data processing and scripting.


    Basic Qualifications/ Experience:
    Experience in designing and implementing ETL processes using Databricks notebooks for efficient data extraction, transformation, and loading.
    In-depth knowledge of the Databricks Unified Analytics Platform and its features for collaborative big data analytics.
    Understanding of data modeling concepts for designing database structures.
    Proficiency in working with both relational databases and NoSQL databases.
    Integration of data from diverse sources, including APIs, databases, streaming data, and external data feeds.
    Implementation of processes to ensure data quality, including data validation, cleansing, and error handling.
    Knowledge of cluster management, optimization, and scaling for efficient data processing.
    Optimization of Spark jobs and Databricks clusters for better performance.
    Proficiency in cloud platforms such as Azure for building scalable and flexible data architectures.
    Use of tools like Apache Airflow, ADF, or Databricks to orchestrate and schedule data workflows.


    Degree or certifications required:
    Education (minimum education level, degree or certification necessary): Bachelor's degree in computer science, management information systems, or related discipline


    Skills (minimum skills required):
    5-7+ years Architect and design large scale high performance distributed systems
    5-7+ years SQL Platform
    2+ years Exposure No-SQL Platform is a plus
    5+ years Hadoop, YARN, MapReduce, Pig or Hive, Spark
    2+ years Data platform implementation on Azure or AWS is a plus


    Key Responsibilities:
    Responsibilities and essential job functions include but are not limited to the following:
    Demonstrate deep knowledge and ability to lead others in the data engineering team to build and support noninteractive (batch, distributed) & real-time, highly available data, data pipeline and technology capabilities
    Translate strategic requirements into business requirements to ensure solutions meet business needs
    Work with infrastructure provisioning & configuration tools to develop scripts to automate deployment of physical and virtual environments; to develop tools to monitor usage of virtual resources
    Assist in the definition of architecture that ensure that solutions are built within a consistent framework
    Lead resolution activities for complex data issues
    Define & implement data retention policies and procedures
    Define & implement data governance policies and procedures
    Identify improvements in team coding standards and help in implementation of the improvements
    Leverage subject matter expertise to coordinate issue resolution efforts across peer support groups, technical support teams, and vendors
    Develop and maintain documentation relating to all assigned systems and projects
    Perform systems and applications performance characterization and trade-off studies through analysis and simulation
    Perform root cause analysis to identify permanent resolutions to software or business process issues
    Lead by example by demonstrating the Client mission and value


    Nice-to-Haves:
    Knowledge of data security best practices and the implementation of measures to ensure data privacy and compliance.
    Implementation of monitoring and logging solutions to track the health and performance of pipelines.
    Familiarity with monitoring platforms like DataDog and New Relic
    Azure
  • Resource Logistics

    Azure Database Engineer

    4 days ago

    Resource Logistics Seattle, United States

    Clienture Database Engineer · Skill Set: Data Engineering, Clienture Data bricks, Data Bricks · Skill to Evaluate: Data Engineering, Clienture Data bricks, Data Bricks, Python, Clienture, Clienture Data, Clienture Data Factory, Clienture Databricks, Databricks, Clienture SQL,S ...

  • The AES Group

    Direct Client

    2 days ago

    The AES Group Seattle, United States

    Title: Direct Client - Data Engineer with Azure · Location: Hybrid (Seattle, WA) · Length: 12+ Months · Job description · This job contributes to the Client's success by planning, organizing, and executing engineering assignments and projects that develop technology and methods i ...

  • Georgia IT Inc

    Data engineer

    2 weeks ago

    Georgia IT Inc Seattle, United States

    Data engineer who has experience with Databricks and Unity Catalog . · Seattle WA · 10 months · Core Skills - · Experience · 10+ years · Azure Data bricks · Azure data piplelines · Azure data factory · Azure data lakes · Real time experience with migration to Unity Cata ...

  • Sonata Software

    Azure Databricks Architect

    3 days ago

    Sonata Software Seattle, United States

    Sonata Software Overview · Sonata is a global technology company specializing in platform-based digital transformation, supporting businesses to become connected, open, intelligent and scalable. Sonata's PlatformationTM methodology brings together industry expertise, platform tec ...

  • eTek IT Services, Inc.

    Sr Data Scientist with Machine Learning

    4 days ago

    eTek IT Services, Inc. Seattle, United States

    Job Description · Job DescriptionRole : Sr Machine Learning Engineer with AILocation : RemoteExperience : 10+years · W2 Contract · JD :Job Description · Title: Sr Engineer (Artificial Intelligence/Machine Learning) Location: Local Hybrid (3 days)Duration: 12 Months with potentia ...

  • Sonata Software

    Azure Databricks Architect

    2 weeks ago

    Sonata Software Seattle, United States

    Sonata Software Overview · Sonata is a global technology company specializing in platform-based digital transformation, supporting businesses to become connected, open, intelligent and scalable. Sonatas Platformation methodology brings together industry expertise, platform techn ...

  • Feuji

    Sr Data Engineer

    2 days ago

    Feuji Seattle, United States

    Feuji Inc is IT solutions company with operations in multiple geographies and specialization across domains. We believe in delivering solutions that will have a measurable positive impact on our client s business and bottom line. Feuji was born out of the desire to create a great ...

  • Synopsys

    D Engineering, Staff Engineer

    1 week ago

    Synopsys Seattle, United States

    VC Formal is a high capacity, high performance next-generation formal verification solution that includes best-in-class algorithms, methodologies, databases and user interfaces. Built from the ground up, this solution was architected to address today's most challenging verificati ...

  • Sonata Software

    Azure Databricks Architect

    3 days ago

    Sonata Software Seattle, United States

    Sonata Software Overview · Take the next step in your career now, scroll down to read the full role description and make your application. · Sonata is a global technology company specializing in platform-based digital transformation, supporting businesses to become connected, o ...

  • Lifetime Brands

    Assistant Category Manager

    1 week ago

    Lifetime Brands Seattle, United States

    Position Overview: Provides direct support to CMs with product development tasks and timelines throughout the entire product lifecycle · Essential Job Functions: · Support timelines, follow-up and track requests with proper individuals as needed to deliver tasks against timeline ...

  • adjust

    Senior Marketing Campaign Manager

    6 days ago

    adjust Seattle, United States

    Adjust, an AppLovin (NASDAQ: APP) company, is trusted by marketers around the world to measure and grow their apps across platforms, from mobile to CTV and beyond. Adjust works with companies at every stage of the app marketing journey, from fast-growing digital brands to brick-a ...

  • adjust

    Senior Marketing Campaign Manager

    1 week ago

    adjust Seattle, United States

    Adjust, an AppLovin (NASDAQ: APP) company, is trusted by marketers around the world to measure and grow their apps across platforms, from mobile to CTV and beyond. Adjust works with companies at every stage of the app marketing journey, from fast-growing digital brands to brick-a ...

  • HICC Pet

    E-Commerce Account Manager

    1 week ago

    HICC Pet Bellevue, United States

    About us: · Do you want to make a difference in the lives of people and their pets? What about having the opportunity to connect with others across the world, full of stimulating discussions, and making impactful contributions? · If this is how you see your career, HICC is the ...

  • Nutpods

    Vice President of Finance

    1 week ago

    Nutpods Bellevue, United States

    Who We Are: · Green Grass Foods, Inc is the manufacturer of the best-selling brand, nutpods, a plant based, dairy-free alternative to half and half made from almonds and coconuts. Awarded as "Editor's Pick" at the Natural Products Expo in its debut year, nutpods has grown tremen ...

  • CHOICE

    Head of Engineering

    2 weeks ago

    CHOICE Bellevue, United States

    Are you ready to join an organization that's investing in the team and technology to take the company (and you) to the next level? · As a fast-growing provider of payments technology and financial solutions, Choice has what it takes to get us there fast: · Great foundation: Stro ...

  • nutpods

    Regional Sales Associate, Natural

    2 days ago

    nutpods Bellevue, United States

    Job Description · Job DescriptionWho We Are: · Green Grass Foods, Inc is the manufacturer of the best-selling brand, nutpods, a plant based, dairy-free alternative to half and half made from almonds and coconuts. Awarded as "Editor's Pick" at the Natural Products Expo in its debu ...

  • Nutpods

    Regional Sales Associate, Natural

    4 days ago

    Nutpods Bellevue, United States

    Who We Are: · Green Grass Foods, Inc is the manufacturer of the best-selling brand, nutpods, a plant based, dairy-free alternative to half and half made from almonds and coconuts. Awarded as "Editor's Pick" at the Natural Products Expo in its debut year, nutpods has grown tremen ...

  • nutpods

    Vice President of Finance

    3 days ago

    nutpods Bellevue, United States

    Job Description · Job DescriptionWho We Are: · Green Grass Foods, Inc is the manufacturer of the best-selling brand, nutpods, a plant based, dairy-free alternative to half and half made from almonds and coconuts. Awarded as "Editor's Pick" at the Natural Products Expo in its debu ...

  • nutpods

    Deductions Specialist

    1 week ago

    nutpods Bellevue, United States

    Job Description · Job DescriptionWho We Are: · Green Grass Foods, Inc is the manufacturer of the best-selling brand, nutpods, a plant based, dairy-free alternative to half and half made from almonds and coconuts. Awarded as "Editor's Pick" at the Natural Products Expo in its debu ...

  • Diverse Lynx

    Azure Data Engineer

    1 week ago

    Diverse Lynx Redmond, United States

    Azure Data Engineer · Redmond, WA (Day one Onsite) · Full Time · Description: Hands on experience as an Azure data engineer which includes. · Design and implement highly available and scalable Azure cloud data solutions that meet business requirements · Developing and maintain ...