Data Engineer - Oldwick - A.M. Best

    A.M. Best
    A.M. Best Oldwick

    23 hours ago

    $80,000 - $140,000 (USD) per year *
    Description
    Benefits Summary
    • Flexible and hybrid work arrangements
    • Paid time off/Paid company holidays
    • Medical plan options/prescription drug plan
    • Dental plan/vision plan options
    • Flexible spending and health savings accounts
    • 401(k) retirement savings plan with a Roth savings option and company matching contributions
    • Educational assistance program
    Overview
    The Data Engineer is responsible for designing, building, and optimizing scalable data solutions to support a wide range of business needs. This role requires a strong ability to work both independently and collaboratively in a fast-paced, agile environment. The ideal candidate will engage with cross-functional teams to gather data requirements, propose enhancements to existing data pipelines and structures, and ensure the reliability and efficiency of data processes.
    Responsibilities

    • Assist with leading the team's transition to the Databricks platform and utilize the newer features of Delta Live Tables, Workflows etc

    • Design and develop data pipelines that extract data from Oracle, load it into the data lake, transform it into the desired format, and load it into Databricks data lakehouse

    • Optimize data pipelines and data processing workflows for performance, scalability, and efficiency

    • Implement data quality checks and validations within data pipelines to ensure the accuracy, consistency, and completeness of data

    • Help create and maintain documentation for data mappings, data definitions, architecture and data flow diagrams

    • Build proof-of-concepts to determine viability of possible new processes and technologies

    • Deploy and manage code in non-prod and prod environments

    • Investigate and troubleshoot data related issues and fix or provide solutions to fix defects

    • Identify and resolve performance bottlenecks, which could include suggesting ways to optimize and performance tune databases and queries to enhance query performance
    Qualifications

    • Bachelor's Degree in Computer Science, Data Science, Software Engineering, Information Systems, or related quantitative field

    • 4 plus years of experience working as a Data Engineer, ETL Engineer, Data/ETL Architect or similar roles

    • Must hold a current/active Databricks Data Engineer/Analyst certification
    Skills

    • 4+ years of solid continuous experience in Python

    • 3+ years working with Databricks with knowledge and expertise of data structures, data storage and change data capture gained from prior production implementations of data pipelines, optimizations, and best practices

    • 3+ years of experience in Kimball dimensional modeling (star-schema comprising of facts, type1 and type2 dimensions, aggregates, etc.) with solid understanding of ELT/ETL

    • 3+ years of solid experience writing SQL and PL/SQL code

    • 2+ years of experience with Airflow

    • 3+ years of experience working with relational databases (Oracle preferred)

    • 2+ years of experience working with NoSQL databases: MongoDB, Cosmos DB, DocumentDB or similar

    • 2+ years of cloud experience (Azure preferred)

    • Experience with CI/CD utilizing git/Azure DevOps

    • Experience with storage formats including Parquet/Arrow/Avro

    • Effectively collaborate with team members while being able to work independently with minimal supervision

    • Must have a creative mindset, knack to solve complex problems, passion to work with data, and a positive attitude

    • Ability to collaborate within and across teams of different technical knowledge to support delivery and educate end users on data products

    • Expert problem-solving skills, including debugging skills, allowing the determination of sources of issues in unfamiliar code or systems
    Pluses, but not required: Any work experience in the following:
    ETL / ELT tools: Spark, Kafka, Azure Data Factory (ADF)
    Languages: R, Java, Scala
    * This salary range is an estimation made by beBee
  • Work in company

    Data Engineer

    Only for registered members

    The Data Engineer is responsible for designing, building, and optimizing scalable data solutions to support a wide range of business needs. · This role requires a strong ability to work both independently and collaboratively in a fast-paced, agile environment. ...

    Oldwick

    1 month ago

  • Work in company

    Data Engineer

    Only for registered members

    The Data Engineer will be responsible for designing, building, and optimizing scalable data solutions to support a wide range of business needs. · This role requires a strong ability to work both independently and collaboratively in a fast-paced, agile environment. · ...

    Oldwick

    1 month ago

  • Work in company

    Data Engineer

    Only for registered members

    The Data Engineer is responsible for designing, building, and optimizing scalable data solutions to support a wide range of business needs. · Fo... · ...

    Oldwick

    1 month ago

  • Work in company

    Data Engineer

    Only for registered members

    The Data Engineer role focuses on designing, building, and optimizing scalable data solutions that support diverse business needs. · ...

    Tewksbury Township

    1 month ago

  • Work in company

    Data Engineer

    Only for registered members

    The Data Engineer will be responsible for designing, building, and optimizing scalable data solutions to support a wide range of business needs. · This role requires a strong ability to work both independently and collaboratively in a fast-paced, agile environment. · ...

    Tewksbury Township

    1 month ago

  • Work in company

    Data Engineer

    Only for registered members

    We are looking for a skilled Data Engineer to design and enhance scalable data solutions that meet diverse business objectives.This role involves collaborating with cross-functional teams to identify data requirements, · improve existing pipelines, and ensure efficient data proce ...

    Tewksbury Township

    1 month ago

  • Work in company

    Data Engineer

    Only for registered members

    The Data Engineer will be responsible for designing, building, and optimizing scalable data solutions to support a wide range of business needs. · Develop and maintain scripts and tools using Python, PowerShell, and R · ...

    Tewksbury, NJ

    1 month ago

  • Work in company

    DataBricks Data Engineer

    Only for registered members

    DataBricks Data Engineer position at Oldwick NJ to lead the transition to Databricks platform utilizing Delta Live Tables Workflows etc and design develop data pipelines extracting data from Oracle loading into Databricks lake house optimizing performance scalability efficiency i ...

    Oldwick

    1 month ago

  • Work in company

    Data Engineer

    Only for registered members

    Data Engineer. Dice is the leading career destination for tech experts at every stage of their careers. · ...

    Bedminster

    1 month ago

  • Work in company

    Data Engineer

    Only for registered members

    The Data Engineer will be responsible for designing, building, and optimizing scalable data solutions to support a wide range of business needs. · ...

    Bedminster

    1 month ago

  • Work in company

    Data Engineer

    Only for registered members

    The Data Engineer is responsible for designing, building, and optimizing scalable data solutions to support a wide range of business needs. · This role requires a strong ability to work both independently and collaboratively in a fast-paced, agile environment. · The ideal candida ...

    Bedminster

    1 month ago

  • Work in company

    Data Engineer

    Only for registered members

    The Data Engineer will be responsible for designing, building, and optimizing scalable data solutions to support a wide range of business needs. · This role requires a strong ability to work both independently and collaboratively in a fast-paced, agile environment. · ...

    Bedminster

    3 weeks ago

  • Work in company

    Data Engineer

    Only for registered members

    +We're looking for problem solvers, innovators and collaborators who want to be part of the mission to help us further grow our company. Lead end-to-end data projects within complex client environments ensuring timely delivery of high-quality Microsoft Cloud data solutions (Azure ...

    Bedminster

    1 month ago

  • Work in company

    Data Engineer

    Only for registered members

    The Data Engineer is responsible for designing, building and optimizing scalable data solutions to support business needs.This role requires strong ability to work independently and collaboratively in an agile environment. · ...

    Bedminster

    1 month ago

  • Work in company

    Databricks Data Engineer

    Only for registered members

    Databricks Data Engineer at ProCorp Systems Inc. · Lead team transition to Databricks platform and utilize Delta Live Tables features. · ...

    Tewksbury Township Full time

    1 month ago

  • Work in company

    Data Engineer – Databricks

    Only for registered members

    The Data Engineer is responsible for designing, building, and optimizing scalable data solutions to support a wide range of business needs. · ...

    Tewksbury Township

    1 month ago

  • Work in company

    Databricks Data engineer @ Onsite

    Only for registered members

    Databricks Data Engineer Full time role · Assist with leading the team's transition to the Databricks platform · Design and develop data pipelines that extract data from Oracle · ...

    Oldwick

    1 month ago

  • Work in company

    Data Engineer

    Only for registered members

    The Data Engineer is responsible for designing, building, and optimizing scalable data solutions to support a wide range of business needs. · ...

    Bedminster, NJ

    1 month ago

  • Work in company

    Data Engineer

    Only for registered members

    The Data Engineer is responsible for designing, building and optimizing scalable data solutions to support business needs. · Focus on continued build out/improvement of the current data stores (Oracle) · Requirement of prior knowledge/experience with database concepts such as SQL ...

    Bedminster, NJ

    1 month ago

  • Work in company

    Databricks Data Engineer

    Only for registered members

    Databricks Data Engineer. · Assist with leading the team's transition to the Databricks platform. · Design and develop data pipelines that extract data from Oracle, load it into the data lake, transform it into the desired format, and load it into Databricks data lakehouse. · ...

    Bedminster

    1 month ago

  • Work in company

    Data Engineer

    Only for registered members

    Bright Vision Technologies is a forward-thinking software development company dedicated to building innovative solutions that help businesses automate and optimize their operations. · We leverage cutting-edge data engineering technologies to build scalable, reliable, and high-per ...

    Bridgewater $80,000 - $140,000 (USD) per year

    1 week ago

Jobs
>
Data engineer
>
Jobs for Data engineer in Oldwick