Jobs
>
San Francisco

    Data Engineer - San Francisco, United States - Pathway Vet Alliance

    Pathway Vet Alliance
    Pathway Vet Alliance San Francisco, United States

    4 weeks ago

    Default job background
    Description
    About Pathway

    Deeptech start-up, founded in March 2020.
    • Our primary developer offering is an ultra-performant Data Processing Framework (unified streaming + batch) with a Python API, distributed Rust engine, and capabilities for data source integration & transformation at scale (Kafka, S3, databases/CDC,...).
    • The single-machine version is provided on a free-to-use license (`pip install pathway`).
    • Major data use cases are around event-stream data (including real-world data such as IoT), and graph data that changes over time.
    • Our enterprise offering is currently used by leaders of the logistics industry, such as DB Schenker or La Poste, and tested across multiple industries. Pathway has been featured in Gartner's market guide for Event Stream Processing.
    • Learn more at and
    Pathway is VC-funded, with amazing BAs from the AI space and industry. We have operations across Europe and in the US. We are headquartered in Paris, with significant support from the French ecosystem (BPI, Agoranov, WILCO,...).

    The Team

    Pathway is built by and for overachievers. Its co-founders and employees have worked in the best AI labs in the world (Microsoft Research, Google Brain, ETH Zurich), worked at Google, and graduated from top universities (Polytechnique, ENSAE, Sciences Po, HEC Paris, PhD obtained at the age of 20, etc...). Pathway's CTO is a co-author with Goeff Hinton and Yoshua Bengio. The management team also includes the co-founder of (1M+ developer users) and (13.5M+ users) and experienced growth leader who has scaled companies with multiple exits.

    The opportunity

    We are searching for a person with a Data Processing or Data Engineering profile, willing to work with live client datasets, and to test, benchmark, and showcase our brand-new stream data processing technology.

    The end-user of our product are mostly developers and data engineers working in a corporate environment. Our development framework is one day expected to become for them a part of their preferred development stack for analytics projects at work - their daily bread & butter.

    You Will

    You will be working closely with our CTO, Head of Product, as well as key developers. You will be expected to:
    • Implement the flow of data from their location in client's warehouses up to Pathway's ingress.
    • Set up CDC interfaces for change streams between client data stores and i/o data processed by Pathway; ensuring data persistence for Pathway outputs.
    • Design ETL pipelines within Pathway.
    • Contribute to benchmark framework design (throughput / latency / memory footprint; consistency), including in a distributed system setup.
    • Contribute to building open-source test frameworks for simulated streaming data scenarios on public datasets.
    Requirements
    • Inside-out understanding of at least one major distributed data processing framework (Spark, Dask, Ray,...)
    • 6 months+ experience working with a streaming dataflow framework (e.g.: Flink, Kafka Streams or ksqldb, Spark in streaming mode, Beam/Dataflow)
    • Ability to set up distributed dataflows independently.
    • Experience with data streams: message queues, message brokers (Kafka), CDC.
    • Working familiarity with data schema and schema versioning concepts; Avro, Protobuf, or others.
    • Familiarities with Kubernetes.
    • Familiarity with deployments in both Azure and AWS clouds.
    • Good working knowledge of Python.
    • Good working knowledge of SQL.
    • Experienced in working for an innovative tech company (SaaS, IT infrastructure or similar preferred), with a long-term vision.
    • Warmly disposed towards open-source and open-core software, but pragmatic about licensing.
    Bonus Points
    • Know the ways of developers in a corporate environment.
    • Passionate about trends in data.
    • Proficiency in Rust.
    • Experience with Machine Learning pipelines or MLOps.
    • Familiarity with any modern data transformation workflow tooling (dbt, Airflow, Dagster, Prefect,...)
    • Familiarity with Databricks Data Lakehouse architecture.
    • Familiarity with Snowflake's data product vision
    • Experience in a startup environment.
    Benefits
    Why You Should Apply
    • Intellectually stimulating work environment. Be a pioneer: you get to work with a new type of stream processing framework.
    • Work in one of the hottest data startups in France, with exciting career prospects
    • Responsibilities and ability to make significant contribution to the company' success
    • Compensation: contract of up to $150k (full-time-equivalent) + Employee stock option plan.
    • Inclusive workplace culture
    Further details
    • Type of contract: Flexible / remote
    • Preferable joining date: early 2023.
    • Compensation: contract of up to $150k (full-time-equivalent) + Employee stock option plan.
    • Location: Remote work from home. Possibility to meet with other team members in one of our offices:
      • Paris - Agoranov (where Doctolib, Alan, and Criteo were born) near Saint-Placide Metro
      • Paris Area - Drahi X-Novation Center, Ecole Polytechnique, Palaiseau.
      • Wroclaw - University area.
    Candidates based anywhere in the United States and Canada will be considered.
  • WPRO TALENTS

    Data Engineer

    3 weeks ago

    WPRO TALENTS San Francisco, United States

    This is a remote position. · Our client is at the forefront of web3 innovation. Our mission is to establish The Graph as the unbreakable foundation of open data. Our pioneering sub graphs set the industry standard and solidify The Graph as the premier solution for organizing and ...

  • WPRO TALENTS

    Data Engineer

    2 weeks ago

    WPRO TALENTS San Francisco, United States

    This is a remote position. · Our client is at the forefront of web3 innovation. Our mission is to establish The Graph as the unbreakable foundation of open data. Our pioneering sub graphs set the industry standard and solidify The Graph as the premier solution for organizing and ...

  • Meta

    Data Engineering Leader

    6 days ago

    Meta San Francisco, United States

    **Data Engineering Leader (Generative AI) Responsibilities**: · - Proactively drive the vision for BI and Data Warehousing across a product vertical, and define and execute on a plan to achieve that vision. · - Define the processes needed to achieve operational excellence in all ...

  • Unreal Gigs

    Data Engineer

    5 hours ago

    Unreal Gigs San Francisco, United States Full time

    Company Overview: Welcome to the forefront of data-driven innovation At our company, we're revolutionizing how businesses leverage data to drive insights and make informed decisions. Our mission is to develop scalable and robust data infrastructure that empowers organizations to ...

  • Sia Partners

    Data Engineer

    2 days ago

    Sia Partners San Francisco, United States

    Sia Partners is a next-generation management consulting firm. We offer a unique blend of AI and design capabilities, augmenting traditional consulting to deliver superior value to our clients. Counting 3,000 consultants in 19 countries, we expect to achieve USD 420 million in tur ...

  • Work & Co

    Data Engineer

    6 days ago

    Work & Co San Francisco, United States Freelance

    Sr. Data Engineer (Contractor) · Presence is a San-Francisco based consultancy specializing in digital product development and innovation for global brands. We've been delivering cutting-edge projects for ten years– and we're growing · We're currently seeking a Sr Data Engineer ...

  • ConsultNet

    Data Engineer

    3 weeks ago

    ConsultNet San Francisco, United States

    Data Engineer · Direct Hire · 100% Remote · Salary: $130, ,000 a year + company equity · Job Description: · Seeking Data Software Engineer who is well versed in Python and SQL and who loves to code. No specific industry experience required but Healthcare background is a plus ...

  • Infinity Ventures

    Data Engineer

    3 weeks ago

    Infinity Ventures Oakland, United States

    Oakland Infomotion GmbH is looking for a Data Engineer (m/w/d) to join our team. We are passionate about extracting the maximum value from data and are looking for someone who shares our enthusiasm and has a strong understanding of digital technology. · As a Data Engineer, you wi ...

  • Nexus Solutions

    Data Engineer

    3 weeks ago

    Nexus Solutions Emeryville, United States

    Soda is teaming up with a leading E-commerce platform based in Germany that specializes in creating lottery platforms. With 20 years of experience, they have continuously grown and innovated across Europe. In 2022, they raised an impressive €286 million for social projects and ch ...

  • CriticalRiver Inc

    Data Engineer DBT

    3 days ago

    CriticalRiver Inc San Francisco, United States

    Job Description · Job DescriptionJob Title: Data Engineer DBT · Employment Type: Fulltime with CR · Onsite: Pleasanton/Santa Clara/SFO (3-4 days based on need) · Years of Experience: 10-15 Years · Technology · SQL & DBT, Snowflake, AWS, Data Bricks · Skill Set · Data Modeling, Da ...

  • Unreal Gigs

    Data Warehouse Engineer

    1 week ago

    Unreal Gigs San Francisco, United States

    Job Description · Job DescriptionCompany Overview: Welcome to the forefront of data-driven decision-making At our company, we're committed to harnessing the power of data to drive insights and innovation. Our mission is to develop scalable and efficient data warehouse solutions t ...

  • Claritas Rx

    Analytics Data Engineer

    5 hours ago

    Claritas Rx San Francisco, United States

    Who We Are · Claritas Rx is a venture-backed digital health startup that brings clarity to the challenges of specialty biopharmaceutical products in the marketplace. In today's highly complex specialty networks, our mission is to illuminate the patient experience beyond the clin ...

  • BayOne Solutions

    Data Engineer

    3 weeks ago

    BayOne Solutions San Francisco, United States Full time

    Senior Data Engineer · Location: (100% Remote) · Duration: Full time · The Opportunity: · Technology · Our technology team works fast and smart. With San Francisco as our home, we take bringing new tech to market seriously, developing the latest in mobile technologies, scalable a ...

  • VRChat Inc

    Data Engineer

    3 weeks ago

    VRChat Inc San Francisco, United States

    Join the VRChat team · VRChat offers a first-of-its kind, game-changing platform that provides an endless collection of social immersive experiences and gives the power of creation to its robust community. With over 250,000 worlds and growing, VRChat's vision is to allow users t ...

  • FocusKPI Inc.

    Data Engineer

    2 weeks ago

    FocusKPI Inc. San Francisco, United States

    Job Description · Job DescriptionFocusKPI is looking for a Data Engineer to join one of our client's team and to support them. As a member of the Data Engineering, you own the ETL/ELT pipelines and data warehouse that are used to analyze trends and activity in our client's produc ...

  • Saxon Global

    Data Engineer

    3 weeks ago

    Saxon Global San Francisco, United States

    Client: T he County of Sacramento · Position: Data Engineer - ETL, Spark, Airflow Specialist · Location: Sacramento, Ca ONLY CANDIDATES LOCAL TO SACRAMENTO WILL BE CONSIDERED. 100% ONSITE WORK · Rate:$65/hr. on C2C (Some Flex) · Visa: Open · Duration: 12+ Months · Looking f ...

  • Tekfortune Inc

    Data Engineer

    3 weeks ago

    Tekfortune Inc San Francisco, United States

    Tekfortune is a fast-growing consulting firm specialized in permanent, contract & project-based staffing services for world's leading organizations in a broad range of industries. In this quickly changing economic landscape, virtual recruiting and remote work are critical for the ...

  • Speak LLC

    Data Engineer

    3 weeks ago

    Speak LLC San Francisco, United States

    About us · Our mission is to become the de facto way people learn foreign languages. We begin by teaching the next billion people English and Spanish. · English is the global language of business, culture, and communication, and over 1.5 billion people around the world are activ ...

  • Stefanini, Inc

    Data Engineer

    2 weeks ago

    Stefanini, Inc San Francisco, United States

    Job Description · Job DescriptionReference #57151 · Stefanini Group is hiringStefanini is looking for Cloud Data Engineer in San Francisco, CA(Hybrid)For quick Apply, please reach out to Ayush Dwivedi · - call: / email: Open to W2 Candidates only (US Citizens & Green Card Holders ...

  • Sia Partners

    Data Engineer

    2 weeks ago

    Sia Partners San Francisco, United States

    Job Description · Job DescriptionCompany Description · Sia Partners is a next-generation management consulting firm. We offer a unique blend of AI and design capabilities, augmenting traditional consulting to deliver superior value to our clients. Counting 3,000 consultants in 19 ...