Lead Data Infrastructure Engineer - Cupertino

Only for registered members Cupertino, United States

1 month ago

Default job background
$180,000 - $250,000 (USD)

Job summary

We're looking for an engineer to help lead the scaling and reliability of our data infrastructure, which is core to the ML work we do at Gridmatic.

Forecasting energy prices is challenging. We have very effective price forecasting models, but we'd like to go much further — scaling the amount of data we can use in our ML models by a factor of 10-100x by incorporating petabyte-scale weather data, increasing spatial granularity of our price forecasting, and more.

We'd also like someone who can tackle the challenge of scaling and improving reliability of our data platform. We deal with a lot of real-world problems when ingesting data from external sources — downtime, late-arriving data, changing schemas.

Responsibilities

  • Experience building the infrastructure for large-scale data processing pipelines (both batch and streaming) using tools like Spark,Kafka,Apollo Flink,and Apache Beam.

  • Experience designing and implementing large-scale data storage systems (feature store,timeseries DBs) for ML use cases.Strong familiarity with relational databases,data warehouses(object storage,timeseries

  • Experience building,data pipelines for external ,sources that are observable,,debuggable,and verifiably correct.Have dealt with challenges
    like ,data versioning point-in-time correctness,evolving schemas.

  • Owning ,and scalingour dataninfrastructureby several ordersof magnitude.Thisincludesour datapipelines,distributed
    data processing,and datastorage.Builidinga unified featurestorefor allourMLmodels.Efficiently storingalandloading hundredsOf terabytes Ofweatherdatavforusein AI-basedweathermodels.Processingandstoringpredictionsandevaluationmetricsforlarge-scalegforecastingmodels.Interviewprocessusuallyhaschatwiththehiringsmanagerorsomeonetontheteamaboutyourbackgroundandexperiencenextdependingontherole,you'llhaveatechnicalphone screenwithanengineer orworkona take-homeproject



    Lorem ipsum dolor sit amet
    , consectetur adipiscing elit. Nullam tempor vestibulum ex, eget consequat quam pellentesque vel. Etiam congue sed elit nec elementum. Morbi diam metus, rutrum id eleifend ac, porta in lectus. Sed scelerisque a augue et ornare.

    Donec lacinia nisi nec odio ultricies imperdiet.
    Morbi a dolor dignissim, tristique enim et, semper lacus. Morbi laoreet sollicitudin justo eget eleifend. Donec felis augue, accumsan in dapibus a, mattis sed ligula.

    Vestibulum at aliquet erat. Curabitur rhoncus urna vitae quam suscipit
    , at pulvinar turpis lacinia. Mauris magna sem, dignissim finibus fermentum ac, placerat at ex. Pellentesque aliquet, lorem pulvinar mollis ornare, orci turpis fermentum urna, non ullamcorper ligula enim a ante. Duis dolor est, consectetur ut sapien lacinia, tempor condimentum purus.
    Get full access

    Access all high-level positions and get the job of your dreams.



Similar jobs

  • Only for registered members Sunnyvale

    We are looking for a hands-on Infrastructure Engineer to join our team and support our high-performance, on-premise server and networking infrastructure. · Maintain, provision, and troubleshoot hardware and Linux systems. · Physically install rack cable maintain blade servers and ...

  • Only for registered members Sunnyvale

    A summary of the job is not available in this data · ...

  • Only for registered members Cupertino $181,100 - $272,100 (USD)

    Are you passionate about Generative AI? · We are the Intelligence System Experience (ISE) team within Apple's software organization. · ...

  • Only for registered members Cupertino

    Etched is seeking an Design Infrastructure Engineer to join our rapidly growing team. · ...

  • Only for registered members Cupertino $181,100 - $318,400 (USD)

    Embedded within Apple's Security Engineering u0026amp; Architecture organization, this role is on an engineering team that builds security infrastructure into Apple's production build system and provides security engineering expertise and leadership across Apple to create securit ...

  • Only for registered members Sunnyvale, CA

    We're looking for an Infrastructure Engineer to help us design, build, and scale the foundational architecture that powers our next-generation AI systems. · ...

  • Only for registered members Cupertino $181,100 - $272,100 (USD)

    We are seeking engineers experienced in building tools for training, adapting. · ...

  • Only for registered members Cupertino $181,100 - $318,400 (USD)

    We design and build software that implements security features into Apple's build systems that affect the software that is shipped to Apple's customers. · In addition to writing software, we perform security analysis on these internal systems to assess threats and security risks. ...

  • Only for registered members Cupertino, CA

    Designing and writing software for new ASICs is hard. · ...

  • Only for registered members Cupertino, CA

    +We are seeking engineers experienced in building tools for training, adapting and deploying large-scale generative models. · +Contributing towards tools for large generative model training including diffusion & autoregressive workflowsBenchmarking, Analysing and Improving traini ...

  • Only for registered members Cupertino, CA

    This is a software engineer role within Apple's Security Engineering & Architecture organization. · We design and build software that implements security features into Apple's build systems that affect the software that is shipped to our customers. · We perform security analysis ...

  • Only for registered members Cupertino, CA

    This is a full-time opportunity for an experienced Infrastructure Lead Engineer to join our team in Cupertino, CA. The ideal candidate will have at least 7 years of experience in Business Process Consulting and Architecture/Design/Detailing of Processes. · • At least 7 years of e ...

  • Only for registered members Mountain View

    We're hiring an Infrastructure & Embedded Software Engineer to help own and scale the embedded infrastructure stack behind next-generation radio and networking systems. · Develop and maintain embedded Linux and bare-metal firmware for radio and networking hardware. · Ownand impro ...

  • Only for registered members Mountain View, CA

    +We're hiring an Infrastructure & Embedded Software Engineer to help own and scale the embedded infrastructure stack behind next-generation radio and networking systems. · +Develop and maintain embedded Linux and bare-metal firmware for radio and networking hardware. · Own and im ...

  • Only for registered members Cupertino Full time

    We're looking for an engineer to evolve our performance monitoring and testing infrastructure.You'll own and improve the systems that power our performance testing and analysis workflows, · and your work will directly impact how WebKit engineers detect and diagnose regressions. · ...

  • Only for registered members Cupertino Full time

    Imagine being at the forefront of an evolution where modern AI meets the elegance of Apple silicon. · ...

  • Only for registered members Cupertino $181,100 - $318,400 (USD)

    The On-Device Machine Learning team transforms groundbreaking research into practical applications. · ...

  • Only for registered members Cupertino $181,100 - $318,400 (USD)

    We are seeking engineers experienced in building infrastructure for training, adapting and deploying large-scale generative models.We are the Intelligence System Experience (ISE) team within Apple's software organization. · ...

  • Only for registered members Cupertino $181,100 - $318,400 (USD)

    The On-Device Machine Learning team transforms groundbreaking research into practical applications, enabling billions of Apple devices to run powerful AI models locally, privately, and efficiently. · ...

  • Only for registered members Cupertino $139,500 - $258,100 (USD)

    This is your opportunity to join a hands-on team that cultivates innovation and helps define the future of quality for Apple's entire enterprise ecosystem. · Design, build, and operate internal infrastructure that simulates core enterprise device management and identity protocols ...

  • Only for registered members Cupertino $147,400 - $272,100 (USD)

    +Job summary · The Apple Services Engineering(ASE) team builds and provides systems and infrastructure that power Apple's services (such as iCloud, iTunes, Siri, and Maps). We are the foundation on which Apple's software developers build the products that our customers love. Our ...