Jobs
>
San Francisco

    Web Crawler Engineer - San Francisco, United States - Acceler8 Talent

    Acceler8 Talent
    Acceler8 Talent San Francisco, United States

    1 week ago

    Default job background
    Description

    Web Crawler Engineer

    In the rapidly evolving landscape of artificial intelligence, the role of a Web Crawler Engineer is becoming increasingly critical. As technology progresses, the demand for high-quality data to fuel innovative models and applications grows significantly. This pivotal role supports the development of cutting-edge AI by ensuring a steady supply of well-curated data.

    Our organization stands at the forefront of AI technology, backed by industry leaders and pioneering venture capital firms. With a mission to enhance human-computer interaction, we are committed to solving complex AI challenges through a synergy of technology and human creativity. Our team consists of dedicated professionals who bring their diverse expertise to tackle unique problems, driving forward the boundaries of what's possible with AI.

    As a Web Crawler Engineer, you will play a fundamental role in sculpting the landscape of data that our AI models rely on. You'll be tasked with architecting and implementing robust web crawler systems that are scalable and efficient. This involves crafting high-performance data acquisition pipelines and ensuring the integrity and reliability of the data collected.

    What we can offer you:

    • A collaborative environment with driven industry experts.
    • Opportunities to work on groundbreaking projects that shape the future of AI.
    • Competitive salary and comprehensive benefits.
    • Support for professional development and continuous learning.
    • Relocation assistance for qualified candidates.

    Key responsibilities include:

    • Designing and developing large-scale distributed web crawler systems.
    • Implementing and maintaining web crawlers and scrapers for dynamic content handling.
    • Creating and managing data acquisition pipelines to process large data volumes efficiently.
    • Optimizing the performance and scalability of web crawlers.
    • Collaborating with teams across the organization to enhance data quality and utility.

    This role is designed for innovators and problem solvers ready to make a significant impact in the AI field. If you are driven by challenging technical problems and have a passion for data and its capabilities in AI, we would love to hear from you.

    Relevant technologies and skills for the role include proficiency in high-performance programming languages like Go, Rust, or C++, expertise in Docker/Kubernetes for orchestration, and experience with cloud services like GCP or AWS. A strong background in web crawling and a curiosity about the impact of data quality on large language models (LLMs) are also highly valued.

  • Acceler8 Talent

    Web Crawler Engineer

    2 weeks ago

    Acceler8 Talent San Francisco, United States

    Web Crawler Engineer · Do you have the right skills and experience for this role Read on to find out, and make your application. · In the rapidly evolving landscape of artificial intelligence, the role of a Web Crawler Engineer is becoming increasingly critical. As technology p ...

  • essential AI

    Member of Technical Staff: Data Acquisition

    1 week ago

    essential AI San Francisco, United States

    Job Description · Job DescriptionEssential AI's mission is to deepen the partnership between humans and computers, unlocking collaborative capabilities that far exceed what could be achieved today. We believe that building delightful end-user experiences requires innovating acros ...

  • Sigmaways Inc

    DevOps Engineer

    4 days ago

    Sigmaways Inc San Francisco, United States

    In this role, you will get an opportunity to broadly apply your engineering skills across various technology solutions, as well as build your skills in other areas by being exposed to various aspects of product delivery from inception, through design, build, and deployment. · Res ...

  • Shimento, Inc.

    Senior Infrastructure Automation Engineer

    1 week ago

    Shimento, Inc. San Francisco, CA, United States

    Senior Infrastructure Automation Engineer · 9 Months · Location:- San Francisco, CA · Qualifications: · Location-this is a hybrid position, but we can consider remote candidates with exceptional technical expertise · This role will be subject to level 2 screening. · Demonstrated ...

  • Varite

    Sr Data Engineer

    1 week ago

    Varite San Francisco, United States

    Pay rate range: $80-90/hr. · True Job Title-Senior Infrastructure Automation Engineer · Location-this is a hybrid position but we can consider remote candidates with exceptional technical expertise. · This role will be subject to level 2 screening. · In this role, you will ge ...

  • Varite

    Sr Data Engineer

    5 days ago

    Varite San Francisco, United States

    Pay rate range: $70-90/hr. · True Job Title-Senior Infrastructure Automation Engineer · Location-this is a hybrid position but we can consider remote candidates with exceptional technical expertise. · This role will be subject to level 2 screening. · In this role, you will ge ...

  • Invisible Technologies

    Automation Engineer

    12 hours ago

    Invisible Technologies San Francisco, United States Full time

    Company Description · Invisible Technologies offers unusual services (a combination of outsourcing and automation) to fast-scaling, innovative companies. Each company understands and uses our services in a variety of ways. It's critical we can communicate to segments of users wit ...

  • SGA

    Senior Infrastructure Automation Engineer

    9 hours ago

    SGA San Francisco, United States

    Software Guidance & Assistance, Inc., (SGA), is searching for a · Senior Infrastructure Automation Engineer · for a · contract · assignment with one of our premier financial services clients in · San Francisco, CA . · Responsibilities · : · In this role, you will get an o ...

  • Stefanini Group

    Infrastructure Automation Engineer

    5 days ago

    Stefanini Group San Francisco, United States

    Details: · Stefanini Group is hiring · Stefanini is looking for Infrastructure Automation Engineer for San Francisco, CA location (Hybrid work) · For quick Apply, please reach out to Ravi Singh- call: / email: · Open to W2 Candidates only · In this role, you will get an opp ...

  • Intelliswift

    Senior Infrastructure Automation Engineer

    1 week ago

    Intelliswift San Francisco, United States

    Job ID: · Job Title: Senior Infrastructure Automation Engineer · Location: San Francisco highly preferred, · Job Type: 12+ Months Contract · Pay Range: $85.00-$92.00/hr. on W2 · Intelliswift Software Inc. conceptualizes, builds, and supports the world's most amazing technology p ...

  • Stefanini

    Infrastructure Automation Engineer

    1 week ago

    Stefanini San Francisco, United States

    Job Details · Information Technology · Infrastructure Automation Engineer · San Francisco, CA Posted: 5/3/2024 · Job Description · Job ID#: · 56981 · Job Category: · Information Technology · Position Type: · Contract · Duration: · Long Term · Details: · Stefanini Group ...

  • Stefanini Group

    Infrastructure Automation Engineer

    1 week ago

    Stefanini Group San Francisco, United States

    Details: · Stefanini Group is hiring · Stefanini is looking for Infrastructure Automation Engineer for San Francisco, CA location (Hybrid work) · For quick Apply, please reach out to Ravi Singh- call: / email: · Open to W2 Candidates only · In this role, you will get an opport ...

  • Federal Reserve Bank (NY)

    Sr Site Reliability Engineer

    1 week ago

    Federal Reserve Bank (NY) San Francisco, United States

    Sr Site Reliability Engineer page is loaded · Sr Site Reliability Engineer · Apply · locations · San Francisco, CA · time type · Full time · posted on · Posted 3 Days Ago · job requisition id · R · Company · Federal Reserve Bank of San FranciscoWhen you join the SF Fe ...

  • Federal Reserve System

    Sr Site Reliability Engineer

    1 week ago

    Federal Reserve System San Francisco, United States

    **Company** · Federal Reserve Bank of San Francisco · We are the Federal Reserve Bank of San Francisco—public servants with a mission to advance the nation's monetary, financial, and payment systems to build a stronger economy for all Americans. We are a community-engaged bank, a ...

  • PG&E Corporation

    Expert, Nondestructive Examination Technical Specialist

    6 days ago

    PG&E Corporation San Ramon, United States Full time

    Requisition ID # · Job Category: Engineering / Science · Job Level: Individual Contributor · Business Unit: Electric Engineering · Work Type: Onsite · Job Location: San Ramon · Department Overview · Applied Technology Services (ATS) is an internal service provider performi ...

  • PG&E Corporation

    Expert, Nondestructive Examination Technical Specialist

    2 days ago

    PG&E Corporation San Ramon, United States

    Requisition ID# · Job Category: Engineering / Science · Job Level: Individual Contributor · Business Unit: Electric Engineering · Work Type: Onsite · Job Location: San Ramon · Department Overview · Applied Technology Services (ATS) is an internal service provider performin ...

  • PG&E Corporation

    Expert, Nondestructive Examination Technical Specialist

    3 days ago

    PG&E Corporation San Ramon, United States

    Requisition ID# 153124Job Category: Engineering / ScienceJob Level: Individual ContributorBusiness Unit: Electric EngineeringWork Type: OnsiteJob Location: San Ramon Department OverviewApplied Technology Services (ATS) is an internal service provider performing technical services ...