Data Engineer - Oakbrook Terrace
3 days ago
Job description
Title:
Data Engineer
Location:
Onsite in Oakbrook Terrace, IL
Terms of Employment
- W2 Contract, 12 months
- The office is located in Oakbrook Terrace, IL.
Our Client is seeking a mid-level Data Engineer is responsible for designing, building, and maintaining the data infrastructure and pipelines that enable efficient data processing, storage, and analysis within a hybrid computing environment.
This role bridges the gap between data collection and data consumption across both high-performance computing (HPC) servers and Azure cloud platforms, ensuring that data is accessible, reliable, and optimized for analytics, machine learning, and business intelligence purposes.
Responsibilities- Design, develop, and implement scalable data pipelines using modern ETL/ELT frameworks and tools in hybrid environments
- Create and maintain data architectures that span high-performance computing servers and Azure cloud services
- Build robust data integration solutions to connect various data sources across on-premises and cloud systems
- Configure and optimize databases and data stores in both UNIX/Linux environments and cloud platforms
- Implement MLOps practices to support the machine learning lifecycle from development to production
- Design and implement modern data flow architectures that support both batch and real-time processing needs
- Apply performance tuning techniques for high-volume data processing in HPC environments
- Collaborate with data scientists, ML engineers, and business stakeholders to understand data requirements
- Develop and maintain documentation for data infrastructure, pipelines, and workflows
- Implement data quality monitoring and validation across hybrid environments
- Mentor junior data engineers on best practices for hybrid cloud/on-premises architectures
- Bachelor's degree in Computer Science, Information Systems, or related technical field
- Relevant certifications in Azure, data technologies, or HPC administration
- 3-5 years of experience in data engineering or related field
- Strong UNIX/Linux administration skills and command-line proficiency
- Experience with high-performance computing environments and workload management
- Proficiency in SQL and experience with major database systems (PostgreSQL, SQL Server, Oracle, MySQL)
- Hands-on experience with Azure cloud data services (Azure Data Factory, Synapse Analytics, Azure SQL)
- Knowledge of modern data flow architectures (Lambda, Kappa, Delta) and implementation patterns
- Experience with ML flow frameworks and ML lifecycle management tools
- Programming skills in Python and shell scripting for automation and data processing
- Familiarity with containerization (Docker) and orchestration (Kubernetes)
- Experience with data processing frameworks (Spark, Databricks, Azure Batch)
- Understanding of networking concepts for data movement between on-premises and cloud
- Experience with real-time data processing and streaming technologies (Kafka, Event Hubs)
- Knowledge of infrastructure-as-code tools (Terraform, ARM templates)
- Experience with distributed file systems (HDFS, Lustre) and object storage
- Familiarity with graph databases and specialized analytics datastores
- Understanding of data governance, security, and compliance requirements across hybrid environments
- Experience with optimization techniques for large-scale data transfers between systems
- Knowledge of GPU acceleration for data processing pipelines
- Background in implementing CI/CD for data solutions
Similar jobs
Title: Data Engineer · Location: Onsite in Oakbrook Terrace, IL · Terms of Employment · • W2 Contract, 12 months · • The office is located in Oakbrook Terrace, IL. · Overview · Our Client is seeking a mid-level Data Engineer is responsible for designing, building, and maintaining ...
2 days ago
nTech Workforce has an immediate 23128 Data Engineer · Terms of Employment · W2 Contract, 12 months · The office is located in Oakbrook Terrace, IL. · Overview · Our Client is seeking a mid-level Data Engineer is responsible for designing, building, and maintaining the data inf ...
4 hours ago
Hiring: Mid-Level Data Engineer (Hybrid – HPC + Azure) · We're looking for a talented Data Engineer who thrives at the intersection of high-performance computing and cloud data platforms. If you enjoy building scalable data pipelines and working in hybrid environments, this role ...
3 days ago
Hiring: Mid-Level Data Engineer (Hybrid – HPC + Azure) · We're looking for a talented · Data Engineer · who thrives at the intersection of · high-performance computing and cloud data platforms · . If you enjoy building scalable data pipelines and working in hybrid environments, t ...
2 days ago
The position is completely remote but may eventually be on Hybrid arrangement. Analyze complex data elements and systems, data flows, dependencies, and relationships to troubleshoot data issues across the business and presents solutions to development team. · ...
1 month ago
Drives the design development and optimization of enterprise data pipelines and platforms. · ...
1 month ago
We are seeking an experienced Data Engineer with expertise in the Microsoft Business Intelligence stack to design, build, and maintain robust data pipelines and analytical models. · This role focuses on backend data engineering using strong SQL skills, SQL Server Analysis Service ...
1 week ago
We are seeking an experienced Data Engineer with expertise in the Microsoft Business Intelligence stack to design build and maintain robust data pipelines and analytical models. · This role focuses on backend data engineering using strong SQL skills SQL Server Analysis Services S ...
1 month ago
Drive scope definition requirements analysis data and technical design pipeline build product configuration unit testing and production deployment. · Design scalable ingestion processes to bring on prem API drive 3rd party end user generated data sources to integrate in common cl ...
1 month ago
We're Hiring: · Sr Data Engineer (Remote)We foster a collaborative, inclusive culture that values diversity of thought and encourages creativity. In this role, you'll have direct opportunities to shape the future of our healthcare data platforms by designing scalable systems, opt ...
1 month ago
The IT Data Engineering Manager is expected to use their strong business acumen and leadership skills to effectively guide the Data Engineering team to work with business partners to understand their data needs and develop technical designs and requirements to meet those needs. · ...
1 week ago
Drives the design, development and optimization of enterprise data pipelines and platforms. Focused on building robust scalable secure data solutions using modern cloud technologies. · ...
1 month ago
The IT Data Engineering Manager is expected to use their strong business acumen and leadership skills to effectively guide the Data Engineering team. · This role manages the Data Engineering team to take the lead in helping to identify, organize and report on the Firm's valuable ...
1 week ago
Drive scope definition, requirements analysis, data and technical design, pipeline build product configuration unit testing and production deployment. Design scalable ingestion processes to bring on prem API drive third party end user generated data sources to integrate in common ...
1 month ago
Drive scope definition, requirements analysis, data and technical design ...
1 month ago
We are looking for a contract Data Engineer to assist our client in hiring a talented individual to work on their team. · Design, develop and implement components to migrate data from legacy platforms into SAP ERP platforms. · Evaluate and select tools and technologies for data e ...
1 week ago
The Data Engineering team builds data solutions that powers strategic and tactical business decisions and supports the Analytics and Artificial Intelligence operations. · ...
1 month ago
Data Engineer Lead responsible for ensuring data accuracy and reliability. · Lead data conversion team. · Select tools for data extraction. · Design and implement components to migrate data. · ...
6 days ago
· Position at Parts Town · See What We're All About · As the fastest-growing distributor of restaurant equipment, HVAC and residential appliance parts, we like to do things a little differently. First, you need to understand and demonstrate our Core Values with safety being your ...
2 days ago
This data engineer role involves building scalable data pipelines, data modeling, and performance optimization. The candidate must be local to the Chicagoland area and have strong hands-on experience in data engineering. · Strong hands-on data engineering experience · Solid SQL a ...
3 weeks ago