Data Engineer - Crockett, United States - IntagHire

Description

Data Engineer with Python, SQL, Slurm, and Kubernetes - Onsite in Houston, TX. Fulltime role with a benefits package

The experience expected from applicants, as well as additional skills and qualifications needed for this job are listed below.

About the Role:
We are seeking a skilled mid to senior level

Python Data Engineer

to join our clients' team in Houston, TX. We are seeking a talented and experienced Data Engineer to join our dynamic team.

If you are passionate about transforming data into actionable insights and have a strong background in data engineering, we want to hear from you.

The ideal candidate will have 5+ years of experience in data engineering with a focus on , Python, SQL, and Java, and will work closely with our clients' Data Scientists and Business Analysts to ensure that data is properly collected, processed, and analyzed to generate insights and drive business decisions.

Experience with Slurm and Kubernetes are highly preferred. The candidate will be required to work on-site at our clients' office in Downtown Houston.


Responsibilities:

Data Pipeline Development:
Build and maintain efficient data pipelines for collecting, processing, and storing large volumes of data

Data Integration:
Collaborate with cross-functional teams to integrate data from various sources into our systems

Database Management:
Design, optimize, and maintain databases using SQL and NoSQL technologies

Software Development:
Write clean and efficient code using tech stacks such as , Python, Java, and more (the more, the better)

Linux Job Scheduling:
Utilize Slurm or similar Linux job scheduling systems for resource management and job scheduling

Containerization:
Experience with Kubernetes (Kube) is preferred for container orchestration

Streaming Data:
Work with real-time data processing and streaming technologies, including MQTT and Kafka

Data Quality:
Ensure data quality, accuracy, and consistency by implementing data validation and cleansing processes

Performance Optimization:
Identify and resolve bottlenecks in data processing, storage, and retrieval

Documentation:
Maintain comprehensive documentation for data pipelines, systems, and processes

Requirements:

Bachelor's degree in Computer Science, Information Systems, or a related fieldAt least 5 years of data engineering experience, with a strong track record of designing and implementing data solutions.

Proficiency in data engineering technologies, including , Python, SQL, and Java (additional programming languages are a plus).Experience with Linux job scheduling, particularly with Slurm, or similar job scheduling systems.

Familiarity with containerization technologies, with a preference for Kubernetes experience.
Excellent knowledge of streaming data technologies, including MQTT and Kafka.
Strong problem-solving skills and the ability to work in a collaborative team environment.
Strong communication skills and the ability to translate complex technical concepts to non-technical stakeholders.
A commitment to best practices in data engineering, including data security and privacy considerations.

Strong problem-solving and analytical skillsExcellent communication and collaboration skillsThis is an exciting opportunity to be part of a startup organization, as well as to work directly with a Fortune 200 company.

If you are a strategic thinker with a passion for delivering innovative products, we would love to hear from you.

Benefits:
Profit Sharing401k with match100% paid medical premiumsCareer development/training opportunities

#J-18808-Ljbffr