Data Engineer - Edison, United States - Diverse Lynx

Diverse Lynx Edison, United States

3 days ago

Description

Role:
Data Engineer

Location:
Edison NJ(Onsite)

Duration:
Full-time

8-10 Years

Skill:
Data Engineer

Design, develop, monitor, and operate data pipelines and maintenance of existing data pipelines.
Collaborating with product owner and analysts to provide data-driven solutions and insights.
Implementing data transformations and aggregations for large datasets.
Ensuring data security and compliance with data governance policies.
Developing in languages like Pyspark and Python.
Preprocess of structured and unstructured data.
nalyze, transform, and create SQL queries.
Developing and optimizing Spark-based data processing applications on the AWS Databricks platform.
Troubleshooting and debugging data-related issues on the AWS Databricks platform.
Integrating Databricks with other data storage and processing systems.

What we are looking for:

Expertise in Apache Spark along with Spark streaming & Spark SQL.
Highly proficient with Apache Spark, Scala/ Python and PySpark.
Good understanding & hands-on experience Gitlab with CI/CD.
Hands-on experience in AWS Services like S3, Glue is required.
Working knowledge of AWS Databricks, Cluster Management.
Design, build, and test data pipelines and workflows using AWS Databricks.
Having the knowledge about Optimizing AWS Databricks Streaming jobs.
Have a deep understanding of AWS Cloud platforms and data engineering technologies.
Strong SQL skills to perform data analysis and understanding of source data.
Familiarity with NoSQL databases and JSON formats.
Experience in building ETL / data warehouse transformation processes.
Strong knowledge of Data Management principles.
Good to have experience in analytical tool.
Good to have experience in Alteryx/Tableau.

Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will receive due consideration for employment without any discrimination.

All applicants will be evaluated solely on the basis of their ability, competence and their proven capability to perform the functions outlined in the corresponding role.

We promote and support a diverse workforce across all levels in the company.

data engineers

3 days ago

Corus Soft Inc Edison, United States

Design and develop data warehouse and data pipelines to ingest, translate, and analyze large sets of data using Java, MongoDB, Tableau and SQL Server. Create HBase, Hive tables and HDFS directories. Develop and build various graphs, dashboards, and automated framework for data ex ...
Data Engineer

5 days ago

BHIRED Edison, United States

Full Time Data Engineer - 800 · We are looking for a full-time Data Engineer to join our team. The ideal candidate will have experience in data management, data accessing, and SOPs. · The primary responsibility of the Data Engineer will be to design, develop, and maintain the c ...
Data Engineer

3 days ago

The Dignify Solutions LLC Edison, United States

7+ Years of Experience · 2+ years of hands-on experience in ETL development for a data warehouse · 2+ years of hands-on experience in development, maintenance, and enhancements of Informatica Mappings, Workflows, and processes · Proficient in programming against large data assets ...
Data Engineer

3 weeks ago

QData Piscataway, United States contract

Mandatory Skills Data Engineer Data Integration Required skills Informatica Unix Informatica DIH. Candidates should have overall 10+ year of experience. Designs and builds solutions to move data from operational and external environments to the business intelligence environment u ...
Data Engineer

5 days ago

LanceSoft New Brunswick, United States

Job Description: Data Engineer--Devens, MA · Local Candidates only · This role is 100% onsite · Work Schedule: Mon - Fri, Business Hours · We are seeking a talented and highly motivated data engineer to join the Quantitative Sciences & Digital Transformation group within the ...
Azure Data Engineer

6 days ago

Dice Edison, United States

Dice is the leading career destination for tech experts at every stage of their careers. Our client, eSolutionsFirst, LLC, is seeking the following. Apply via Dice today · Needed Locals only *** · Azure Data Engineer- ADF @ Edison ,NJ · 12 Months Contract (Possible extension) ...
GCP Data Engineer

5 days ago

Diverse Lynx Edison, United States

Onsite Role · Job Title:GCP Data Engineer · Duration: 6+ Months · Location: Atlanta, GA · Roles & Responsibilities : · * Strong Python Development Experience is mandatory · * 8 + years of Data Engineering experience working with both distributed architectures, ETL, EDW an ...
Big Data Engineer

4 days ago

Tekfortune Inc Piscataway, United States

Tekfortune is a fast-growing consulting firm specialized in permanent, contract & project-based staffing services for world's leading organizations in a broad range of industries. In this quickly changing economic landscape, virtual recruiting and remote work are critical for the ...
Data Engineer

3 days ago

SHI International Somerset, United States

Job Summary · We are seeking a highly motivated and experienced Data Engineer design and implement enterprise-scale data analytics solutions on Microsoft Azure and Microsoft Fabric. This role will play a key part in building and maintaining our data platform, ensuring data qualit ...
Data Engineer

4 hours ago

SHI GmbH Somerset, United States

Job Summary · We areseekinga highly motivated and experiencedData Engineerdesign and implement enterprise-scale data analytics solutions on Microsoft Azure andMicrosoftFabric. This role will play a key part in building andmaintainingour data platform, ensuring data quality and ac ...
Systems Data Engineer

3 hours ago

Kaizen Technologies Edison, United States

Analyze, Design, develop, integrate, configure, test, implement & support s/w applications using Java, JIRA, SQL, Python, AWS, Agile, Postgres SQL, PySpark, PowerBI, Visio, DataBricks, Azure. Must be willing to travel & reloc to unanticipated client locations throughout the US. · ...
Architect Data Engineering AWS

3 days ago

ClifyX Edison, United States

Job Description (Please provide summary of the position): · Provide Data Engineering & platform architecture solutions/support in line with the identified needs and expectations of TCS's NA LS clients and NA LS Data & Analytics practice. · Responsibilities: · • s a Senior Data ...
Senior Data Engineer

1 week ago

Bayer Industry Services (Currenta GmbH & Co. OHG) New York, United States

Das kannst Du bei uns bewegen · Große Datenmengen · sind Dein Ding – Du unterstützt den Data Steward bei der Anbindung immer neuer Datenquellen an unseren Data Lake, sorgst für die · Transformation der Rohdaten zu "Trusted Data" · und unterstützt die Data Scientists bei der D ...
Data Engineer II

3 days ago

Nuvance Health South Plainfield, United States

ONLY EXPERIENCED MEDICAL/HEALTHCARE DATA ENGINEERS WILL BE CONSIDERED Minimum of 4 years of experience in a Data Engineering role in healthcare · Experience with EMR/Electronic Medical Records · BS Degree in Computer Science, Information Systems, or related discipline · 100% R ...
Data Engineer

6 days ago

Diverse Lynx Iselin, United States

Data Engineer · Location - Plano TX & Mclean VA · We are looking for a strong Data Engineer with skills as listed below: · Must have: Java, AWS, Data Processing experience, SQL · Diverse Lynx LLC is an Equal Employment Opportunity employer. All qualified applicants will recei ...
Senior Electronic Data Capture Engineer

3 days ago

Takeda Pharmaceutical Trenton, United States Full time

By clicking the "Apply" button, I understand that my employment application process with Takeda will commence and that the information I provide in my application will be processed in line with Takeda'sPrivacy Notice and Terms of Use. I further attest that all information I subm ...
Data Center Engineer

2 days ago

NESCO Inc Somerset, United States

Nesco Data Center & Infrastructure is seeking an Data Center Engineer in the greater Somerset/Piscataway NJ Area. · Shift:2:00pm to 12:00am Wednesday - Saturday · Direct Hire · Job Summary: · We are looking for self-motivated, dedicated individuals to join our team of highly ...
Data Engineer, Analytics

3 days ago

BEUMER Group Somerset, United States

Company Description · BEUMER has operated in the North American market for more than 40 years and has completed over 160 airport baggage handling projects; with fully integrated design and manufacturing systems in the United States. We are a family owned intra-logistics leader w ...
Software or Network or Test or Data Engineers, or Business/Data/QA Analysts

1 week ago

IP Heights LLC Edison, United States

IP Heights LLC seeks Software or Network or Test or Data Engineers, or Business/Data/QA Analysts. Primary worksite is Edison, NJ, but relocation is possible. Contact · ...
Data Engineer – Spark, Hadoop

2 days ago

QData Piscataway, United States contract

Role Duties Designs and builds solutions to move data from operational and external environments to the business intelligence environment using Ab Initio software and DataStage (formerly Ascential) - IBM's WebSphere Data Integration Suite. Skills include designing and developing ...