Jobs
>
San Francisco

    Senior Data Engineer - San Francisco, United States - People Data Labs

    People Data Labs
    People Data Labs San Francisco, United States

    1 week ago

    Default job background
    Description

    Job Description

    Job Description

    About Us

    At People Data Labs, we're committed to democratizing access to high-quality B2B data and leading the emerging DaaS economy. We empower developers, engineers, and data scientists to create innovative, compliant data products at scale with our clean, easy-to-use datasets of resume, company, location, and education data consumed through our suite of APIs.

    PDL is an innovative, fast-growing, global team backed by world-class investors, including Craft Ventures, Flex Capital, and Founders Fund. We scour the world for people hungry to improve, curious about how things work, and willing to challenge the status quo to build something new and better.

    Roles & Responsibilities:

    • Build infrastructure for ingestion, transformation, and loading an exponentially increasing volume of data from a variety of sources using Spark, SQL, AWS, and Databricks
    • Building an organic entity resolution framework capable of correctly merging hundreds of billions of individual entities into a number of clean, consumable datasets.
    • Developing CI/CD pipelines and anomaly detection systems capable of continuously improving the quality of data we're pushing into production.
    • Devising solutions to largely-undefined data engineering and data science problems.
    • Work with stakeholders in Engineering and Product to assist with data-related technical issues and support their infrastructure needs

    Technical Requirements

    • 5-7+ years industry experience with clear examples of strategic technical problem solving and implementation
    • Strong software development fundamentals.
    • Experience withPython Expertise with Apache Spark (Java, Scala, and/or Python-based)
    • Experience with SQL
    • Experience building scalable data processing systems (e.g., cleaning, transformation) from the ground up.
    • Experience using developer-oriented data pipeline and workflow orchestration (e.g., Airflow (preferred), dbt, dagster or similar)
    • Knowledge of modern data design and storage patterns (e.g., incremental updating, partitioning and segmentation, rebuilds and backfills)
    • Experience working in Databricks (including delta live tables, data lakehouse patterns, etc.)
    • Experience with cloud computing services (AWS (preferred), GCP, Azure or similar)
    • Experience with data warehousing (e.g., Databricks, Snowflake, Redshift, BigQuery, or similar)
    • Understanding of modern data storage formats and tools (e.g., parquet, ORC, Avro, Delta Lake)

    Professional Requirements

    • Must thrive in a fast paced environment and be able to work independently
    • Can work effectively remotely (able to be proactive about managing blockers, proactive on reaching out and asking questions, and participating in team activities)
    • Strong written communication skills on Slack/Chat and in documents
    • You are experienced in writing data design docs (pipeline design, dataflow, schema design)
    • You can scope and breakdown projects, communicate and collaborate progress and blockers effectively with your manager, team, and stakeholders

    Nice To Haves:

    • Degree in a quantitative discipline such as computer science, mathematics, statistics, or engineering
    • Experience working with entity data (entity resolution / record linkage)
    • Experience working with data acquisition / data integration
    • Expertise with Python and the Python data stack (e.g., numpy, pandas)
    • Experience with streaming platforms (e.g., Kafka)
    • Experience evaluating data quality and maintaining consistently high data standards across new feature releases (e.g., consistency, accuracy, validity, completeness)

    Our Benefits

    • Stock
    • Competitive Salaries
    • Unlimited paid time off
    • Medical, dental, & vision insurance
    • Health, fitness, and office stipends
    • The permanent ability to work wherever and however you want

    No C2C, 1099, or Contract-to-Hire. Recruiters need not apply.

    People Data Labs does not discriminate on the basis of race, sex, color, religion, age, national origin, marital status, disability, veteran status, genetic information, sexual orientation, gender identity or any other reason prohibited by law in provision of employment opportunities and benefits.

  • WPRO TALENTS

    Data Engineer

    1 week ago

    WPRO TALENTS San Francisco, United States

    This is a remote position. · Our client is at the forefront of web3 innovation. Our mission is to establish The Graph as the unbreakable foundation of open data. Our pioneering sub graphs set the industry standard and solidify The Graph as the premier solution for organizing and ...

  • WPRO TALENTS

    Data Engineer

    1 week ago

    WPRO TALENTS San Francisco, United States

    This is a remote position. · Our client is at the forefront of web3 innovation. Our mission is to establish The Graph as the unbreakable foundation of open data. Our pioneering sub graphs set the industry standard and solidify The Graph as the premier solution for organizing and ...

  • Nexus Solutions

    Data Engineer

    1 week ago

    Nexus Solutions Emeryville, United States

    Soda is teaming up with a leading E-commerce platform based in Germany that specializes in creating lottery platforms. With 20 years of experience, they have continuously grown and innovated across Europe. In 2022, they raised an impressive €286 million for social projects and ch ...

  • Infinity Ventures

    Data Engineer

    1 week ago

    Infinity Ventures Oakland, United States

    Oakland Infomotion GmbH is looking for a Data Engineer (m/w/d) to join our team. We are passionate about extracting the maximum value from data and are looking for someone who shares our enthusiasm and has a strong understanding of digital technology. · As a Data Engineer, you wi ...

  • Triune Infomatics Inc

    Data Engineer

    1 week ago

    Triune Infomatics Inc San Francisco, United States

    Role: Data Engineer · Location: Hybrid in Pleasanton, CA · Duration: 6 months · Areas of Responsibility Include: · • Build test and implement Data Engineering - ETL, Data Integrations, Data Governance, Data Quality. · • Build reusable Data Engineering assets and frameworks. ...

  • BayOne Solutions

    Data Engineer

    1 week ago

    BayOne Solutions San Francisco, United States Full time

    Senior Data Engineer · Location: (100% Remote) · Duration: Full time · The Opportunity: · Technology · Our technology team works fast and smart. With San Francisco as our home, we take bringing new tech to market seriously, developing the latest in mobile technologies, scalable a ...

  • Tekfortune Inc

    Data Engineer

    1 week ago

    Tekfortune Inc San Francisco, United States

    Tekfortune is a fast-growing consulting firm specialized in permanent, contract & project-based staffing services for world's leading organizations in a broad range of industries. In this quickly changing economic landscape, virtual recruiting and remote work are critical for the ...

  • VRChat Inc

    Data Engineer

    1 week ago

    VRChat Inc San Francisco, United States

    Join the VRChat team · VRChat offers a first-of-its kind, game-changing platform that provides an endless collection of social immersive experiences and gives the power of creation to its robust community. With over 250,000 worlds and growing, VRChat's vision is to allow users t ...

  • Asana

    Data Engineer

    1 week ago

    Asana San Francisco, United States

    As part of our dynamic Data Engineering team, you will play a pivotal role in aiding company-wide, data-informed decision making. Product and Business teams leverage our data assets to optimize user adoption, growth, and experience. We partner with other Data Infrastructure teams ...

  • Psychic Services

    Data Engineer

    2 weeks ago

    Psychic Services San Francisco, United States

    [Full Time] Data Engineer at Psychic (United States) | BEAMSTART Jobs · Data Engineer · Psychic United States · Date Posted · 19 May, 2023 · Work Location · San Francisco, United States · Salary Offered · $120000 — $200000 yearly · Job Type · Full Time · Experience Required · 3 ...

  • Rigil Corporation

    Data Engineer

    1 week ago

    Rigil Corporation San Francisco, United States

    Job Description · Job DescriptionRole: Data Engineer · About Rigil: · Rigil is an award-winning, woman-owned, small business that specializes in technology consulting, strategy consulting and product development. We value teamwork and strive to build strong leaders. · Location: S ...

  • TEKsystems

    Data Engineer

    2 weeks ago

    TEKsystems San Francisco, United States

    : · The data engineer will collaborate closely with the growth and data foundation teams to define and create foundational data tables, migrate test tables while adhering to EDW standards, and build self-serve ETL frameworks capable of handling streaming and batch processing. Th ...

  • University of California

    Data Engineer

    2 weeks ago

    University of California San Francisco, United States

    Data Engineer · Neuro-Memory and Aging · Full Time · 77338BR · Job Summary · **This is an onsite position @ Mission Bay; Local SF · The Brain Aging Network for Cognitive Health (BRANCH) study aims to understand the complex biological, genetic, and lifestyle factors that unde ...

  • ConsultNet

    Data Engineer

    1 week ago

    ConsultNet San Francisco, United States

    Data Engineer · Direct Hire · 100% Remote · Salary: $130, ,000 a year + company equity · Job Description: · Seeking Data Software Engineer who is well versed in Python and SQL and who loves to code. No specific industry experience required but Healthcare background is a plus. Mus ...

  • Speak LLC

    Data Engineer

    1 week ago

    Speak LLC San Francisco, United States

    About us · Our mission is to become the de facto way people learn foreign languages. We begin by teaching the next billion people English and Spanish. · English is the global language of business, culture, and communication, and over 1.5 billion people around the world are activ ...

  • FocusKPI Inc.

    Data Engineer

    1 week ago

    FocusKPI Inc. San Francisco, United States

    Job Description · Job DescriptionFocusKPI is looking for a Data Engineer to join one of our client's team and to support them. As a member of the Data Engineering, you own the ETL/ELT pipelines and data warehouse that are used to analyze trends and activity in our client's produc ...

  • Hazel Health, Inc.

    Data Engineer

    1 week ago

    Hazel Health, Inc. San Francisco, United States

    Hazel Health, the national leader in school-based telehealth, was founded in 2015 to address systemic inequities in healthcare access, and ensure all children can get the quality care they need and deserve. We leverage digital health technology to provide on-demand physical and m ...

  • Bayone

    Data Engineer

    1 week ago

    Bayone San Francisco, United States

    Your role at Sephora: · As a Sr. Software Engineer you will design and implement innovative analytical solutions and work alongside the product engineering team, evaluating new features and architecture. Reporting to the Engineering Manager, Data Platform, you will work closely ...

  • Speak

    Data Engineer

    6 days ago

    Speak San Francisco, United States

    Job Description · Job DescriptionAbout usOur mission is to become the de facto way people learn foreign languages. We begin by teaching the next billion people English and Spanish. · English is the global language of business, culture, and communication, and over 1.5 billion peop ...

  • Speak

    Data Engineer

    1 week ago

    Speak San Francisco, United States

    Job Description · Job DescriptionAbout usOur mission is to become the de facto way people learn foreign languages. We begin by teaching the next billion people English and Spanish. · English is the global language of business, culture, and communication, and over 1.5 billion peop ...