- Design robust data processing pipelines handling large amounts of textual data, ensuring robustness and timeliness.
- Ensure secure handling of data (e2e encryption, data isolation, etc.)
- Manage cloud orchestration and data infrastructure using various relevant data processing tools
- Curate and manage large-scale data ingestion and indexing pipelines, ensure data quality and error handling
- Strong programming and engineering skills, versatility with Python and cloud data infrastructure.
- Experience scaling up to data pipelines to handle large amounts of data on an ongoing basis.
- Independence and willingness to take ownership of tasks
- Passionate about data and AI innovation
- Must currently reside in (or able to relocate quickly to) the SF Bay Area or London, UK and able to work onsite minimum of 3 days/week in the office.
- Must have current US Work Authorization (Green Card/Permanent Resident or US Citizenship).
- Base Salary: $190K - $300K (or competitive offer)
- Hybrid work environment (typically spend 3 days in the office, earlier in the week)
- Full health insurance (medical, dental, vision, and short term disability)
- Unlimited PTO
- 401K (US) /Pension (UK)
- Travel budget & regular in person team building events
- At home office equipment allowance
- Pre-IPO Equity/Stock Options
Senior Data Engineer - Palo Alto, United States - Kismet Search
Description
Exclusive Client of Kismet Search is an exciting, well-funded Bay Area Startup building knowledge discovery tools for domain experts using AI and Large Language Models (LLMs). They've already secured partnerships with major financial institutions and banking customers, making it an opportune time to join as they expand
Co-founders are the CEO who's a former Research Lead at Google Brain and the CTO coming from Meta AI. They're building a world-class team of talent firepower across AI/ML researchers, data scientists, software engineers, and web developers with a breadth of AI experience from Stanford, Meta AI, Google, Microsoft, and more major leaders in the space
We are seeking an experienced Data Engineer to join their team and help deliver high quality data to their retrieval system, large language models, and users. As a member of their team, you will be responsible for designing, optimizing, and maintaining robust data pipelines on public and private data sources. High quality data is crucial to their mission and we're looking for candidates passionate about fast, robust, scalable and AI-native data processing pipelines.
Responsibilities:
Requirements:
Benefits & Compensation Package: