Senior Data Engineer - San Francisco, United States - Nightfall

Description

We're looking for a

Senior Data Engineer

to enhance Nightfall's core data models and infrastructure supporting our real-time and data annotation pipelines, analytics warehouse, MLOps, real-time monitoring, and business intelligence reporting.

At Nightfall, we use ML/NLP to detect and protect sensitive personal, health, financial, and credential information in real time. You will contribute directly to the core product.

Working with Engineers, Data Scientists, and Data Analysts, you will ensure high technical standards for data management and drive Nightfall's short and long-term data strategies.

This is a hybrid role based out of our San Francisco office.

Responsibilities

Design, build, and own the core data models and infrastructure to manage data across Nightfall's analytics stack - AWS, Kafka, Glue, S3, Snowflake, Cassandra, Cockroach DB, Tableau, and Datadog.

Collaborate closely with Data Analysts to unlock the data needed to power dashboards, ad-hoc analyses, monitoring, and high-quality decision-making across the company.

Collaborate closely with ML Engineering and Data Science to scale our data pipelines for machine learning, analytics, and data annotation.

Design and implement a data quality monitoring system with source-to-target data validations and anomaly detection.

Collaborate with Security and Engineering to meet data governance requirements.

Qualifications

SQL Expert. Solid experience in Python or similar.

Experience (5+ years) with analytical data warehouses/lakes. AWS/Snowflake preferred.

Experience (5+ years) developing data pipelines supporting ML/NLP projects. Experience with popular MLOps and data annotation platforms is a plus.

Experience (5+ years) analyzing complex datasets using SQL, optimizing queries, working with Data Analysts to automate dashboards (Tableau preferred), and performing ad-hoc analyses that lead to actionable business insights.

Bachelor's degree in Computer Science or related field. MS/PhD a plus.

Experience with source code repositories and cloud monitoring tools (Datadog preferred).

Possess excellent communication skills, strong analytical skills, and the ability to work independently and as part of a team.

The ability to thrive in a dynamic environment, being flexible and willing to jump in and do whatever it takes to be successful.

#J-18808-Ljbffr