- Python
- Pyspark
- Bigdata
- SQL
- Data engineering
-
Senior Data Engineer
1 week ago
HS Solutions Inc Texas, United StatesWe are looking for Data Engineer in Austin ,TX. ( I'm not looking for Banking or Insurance co. experienced profiles. Please submit only product base co. experienced profiles) · We're seeking a Data Engineer to take the lead in implementing and scaling data · collection, storage, ...
-
AI Engineer/Developer
2 weeks ago
Synergy Consulting Group, Inc. Texas, United StatesWe are seeking a highly motivated and skilled AI Engineer/Developer to join our team and play a crucial role in developing and implementing cutting-edge Generative AI solutions. You will be responsible for the entire AI development life cycle, from understanding business needs an ...
-
Machine Learning Engineer
1 week ago
Canopus IT Solutions LLC Texas, United StatesML Engineer · Remote · Job Description: · We are seeking a talented Machine Learning Engineer with expertise in software engineering to join our team. As a Machine Learning Engineer, your primary responsibility will be to develop machine learning (ML) solutions that focus on tec ...
-
Full Stack Engineer
6 days ago
Fractal Texas, United StatesFractal is a strategic AI partner to Fortune 500 companies with a vision to power every human decision in the enterprise. Fractal is building a world where individual choices, freedom, and diversity are the greatest assets; an ecosystem where human imagination is at the heart of ...
-
Data Scientist II
1 day ago
Ringer Sciences Texas, United StatesAbout Ringer: · Ringer Sciences is a data science and analytics consulting firm that works with companies of all sizes and across industries to help them leverage data to make informed business decisions backed by research. Using a combination of real-world, social, and third-par ...
AI Data Engineer with LLM - Texas, United States - Nexwave
Description
Position : Sr Data Engineer
Location : Austin, TX ( Day1 onsite )
Exp Req : 9+
Duration : Full Time & 12 Months
Locals are priority
Full Time Consultants First Priority
Visa/GC
MUST have Skills -
We're seeking a Data Engineer to take the lead in implementing and scaling data
collection, storage, processing, and filtering for fine-tuning large language models (LLMs) within Conversational Engineering.
These data pipelines are crucial for powering our cutting-edge research, safety systems, and product development.
If you're passionate about working with data and are eager to create solutions that directly impact the advancement of LLMs, we'd love to hear from you.
This role provides the exciting opportunity to collaborate closely with applied ML engineers, software engineers, and data scientists that create our AI systems today.
In this role, you will:
• Design, build, and manage scalable data pipelines for collecting, storing, processing, and filtering large volumes of text data for fine-tuning LLMs.
• Develop and optimize data storage architectures to handle the massive scale of data required for training state-of-the-art language models.
• Implement efficient data preprocessing, cleaning, and feature extraction techniques to ensure high-quality data for model training.
• Collaborate with machine learning engineers and researchers to understand their data requirements and provide tailored solutions for LLM fine-tuning.
• Design and implement robust and fault-tolerant systems for data ingestion, processing, and delivery.
• Optimize data pipelines for performance, scalability, and cost-efficiency, leveraging distributed computing frameworks and cloud platforms.
• Ensure the security, privacy, and compliance of data according to industry best practices and regulatory requirements.
You might thrive in this role if you:
• Have 8+ years of experience as a data engineer, with a strong background in designing and building large-scale data pipelines.
• Possess deep expertise in distributed computing frameworks such as Apache Spark, Hadoop, or Flink, and have hands-on experience optimizing data processing at scale.
• Are proficient in programming languages commonly used in data engineering, such as Python, and have a solid understanding of data structures and algorithms.
• Have extensive experience with cloud platforms like AWS, Google Cloud, or Azure for data storage, processing, and management.
• Are well-versed in various data storage technologies, including distributed file systems (e.g., HDFS, S3),databases (e.g., Cassandra, HBase), and data warehouses (e.g., Redshift, Big Query ).
• Have hands-on experience with ETL orchestration tools such as Apache Airflow, Dagster, or Prefect for managing complex data workflows.
• Possess knowledge of natural language processing (NLP) techniques and have worked with text data preprocessing, normalization, and feature extraction.
• Are passionate about staying up to date with the latest advancements in data engineering and NLP, and are eager to apply innovative techniques to solve challenging problems.
• Have strong problem-solving skills, are detail-oriented, and can effective