- Experience with PySpark or Python
- DataBricks experience is mandatory
- Experience with big data tools Spark, Hadoop Kafka would be good to have
- Hands on experience in Microsoft Azure Data Factory Azure Logic apps service Data Bricks
- Experience integrating different data sources
- Should have experience in Azure services e.g., Azure Data factory, Azure Data Lake (Gen 2) , Azure SQL, Blob Storage and Azure DevOps, ARM
- Should have experience in Azure Databricks PySpark , Databricks Delta, n ice to have experience in Scala, Python
- Experience building data pipeline by using Azure Big data stack
- Extensive ingesting, cleaning, transformation, and aggregation of massive amounts of data from multiple internal and external sources (Salesfo rce, Google Analytics, etc.) by using Azure Services
- Create data pipeline that enable various systems within ecosystem to stream high volume data from multiple sources into central repository for processing
- Reporting at scale enabled by Databrick s Delta for incremental processing
- Understanding of spark framework and tuning of spark applications Extensive experience with horizontally scalable and highly available system design and implementation, with focus on performance and resiliency
- Coordinate with all stakeholders and understand the requirement and meet the delivery deadlines
-
Data Engineering
2 weeks ago
Understood New York, United States**About Understood**: · Understood is a nonprofit focused on shaping the world for difference. We raise awareness of the challenges and strengths of people who learn and think differently. Our resources help people navigate challenges, gain confidence, and find support and commun ...
-
Specialist - Data Engineering
1 week ago
LTIMindtree New York, United States**Job Req Id**:1287937 · - **About Us**: · - **Job Title**:Specialist - Data Engineering (Geneva Developer) · - **Work Location**: New York, NY days onsite/week) · experience as developer. · - Strong functional experience in financial industry and Portfolio Accounting System Gene ...
-
Engineering Contractor, Data
1 week ago
EQUINOX New York, United States Freelance**Engineering Contractor, Data**: · Overview · **OUR STORY** · Equinox Group is a high growth collective of the world's most influential, experiential, and differentiated lifestyle brands. We restlessly seek what is next for maximizing life - and boldly grow the lifestyle brands ...
-
Data Engineering Director
1 week ago
FanDuel New York, United States**ABOUT FANDUEL** · FanDuel Group ("FanDuel") is an innovative sports-tech entertainment company that is changing the way consumers engage with their favorite sports, teams, and leagues. The premier gaming destination in the United States, FanDuel consists of a portfolio of leadi ...
-
Junior Data Engineering Specialist
6 days ago
STV Group New York, United StatesWe are seeking a talented and motivated Junior Data Engineering Specialist to join our dynamic Digital Advisory team. As a Junior Data Engineering Specialist, you will play a key role in designing, developing, and maintaining data pipelines and systems to support our data-driven ...
-
Senior Data Engineer
3 weeks ago
Bayer Industry Services (Currenta GmbH & Co. OHG) New York, United StatesDas kannst Du bei uns bewegen · Große Datenmengen · sind Dein Ding – Du unterstützt den Data Steward bei der Anbindung immer neuer Datenquellen an unseren Data Lake, sorgst für die · Transformation der Rohdaten zu "Trusted Data" · und unterstützt die Data Scientists bei der D ...
-
IT Data Center Engineer
2 weeks ago
CBRE New York, United StatesTeilzeit (20 Wochenstunden) | unbefristet · Die Technische Universität Wien ist Österreichs größte Forschungs- und Bildungseinrichtung im technisch- naturwissenschaftlichen Bereich und leistet seit mehr als 200 Jahren einen unverzichtbaren Beitrag zur Sicherung der international ...
-
Data Engineer
5 days ago
YASH Technologies Jersey City, United StatesHi, · We at Yash Technologies are looking for Data Engineer, if you are looking for new opportunity, please share your updated resume. · Requirement Details: · Location: Jersey City, NJ · Work Authorizations: Citizen, GC EAD, Green Card, H4 EAD · Position Type: Contract · Job Ti ...
-
Data Engineer
2 weeks ago
Diverse Lynx Jersey City, United StatesJob Title: Data Engineer ( Python + AWS) · Location : Jersey City , NJ (Onsite, Local s only) · Job responsibilities : Executes creative software solutions, design, development, and technical troubleshooting with ability to think beyond routine or conventional approaches to bui ...
-
Data Engineer
3 weeks ago
Tiger Analytics Jersey City, United StatesTiger Analytics is a fast-growing advanced analytics consulting firm. Our consultants bring deep expertise in Data Science, Machine Learning and AI. We are the trusted analytics partner for multiple Fortune 500 companies, enabling them to generate business value from data. Our bu ...
-
Data Engineer
1 day ago
YASH Technologies Jersey City, United StatesLooking for 10+ yrs Senior Resource · Mandatory Skills · Oracle · Mongo DB · Data Engineer · NO SQL database · Data Modeling · The Senior Database Engineer(MongoDB) will lead complex or critical database management initiatives that typically involve multiple disciplines and busin ...
-
Data Engineer
2 weeks ago
Tradeweb Markets LLC Jersey City, United StatesThe Billing Technology team is responsible for the overall ownership of our internal billing and invoicing systems including the development, quality assurance and operations of the platform. · As a member of this team, you will be able to take in a set of complex requirements a ...
-
Data Engineer
3 weeks ago
Lord Abbett & Co. Jersey City, United StatesLord Abbett is looking for a talented and experienced developer to join our development team to support the continued development of our analytics platforms. Besides regular individual contributions, Data Engineers are expected to be deep technical e Data Engineer, Data Architect ...
-
Data Engineer
2 weeks ago
Jarvis Walker Jersey City, United StatesData Engineer · We are seeking an enthusiastic and experienced Data Engineer to join our team. As a Data Engineer, you will be a key member of our product development team. You will play a pivotal role in turning data into actionable models that our company uses to create new lin ...
-
Data Engineer
1 week ago
SANS Jersey City, United StatesJob Title - Data Engineer - Snowflake · Location - Jersey City, NJ (Hybrid) · Duration - 12+ Months · Requirements: · 8 to 10 Yrs IT experience on Data driven initiatives with recent 2-3 Yrs experience in snowflake is mandatory · SQL, Data Warehousing, Data Modeling, Cloud based ...
-
Data Engineer
2 weeks ago
Expedite Technology Solutions LLC Jersey City, United StatesProvide technical design and develop Extract/Transform/Load (ETL), ELT applications that interface with Datametica's customer applications. · Lead and implement data driven solutions and collaborate with larger development teams · Lead and Develop Test Strategy, Test Plan, Test C ...
-
Data Engineer
2 weeks ago
iPivot, LLC Jersey City, United StatesRole: Data Engineer · Fulltime/Contract · Location: New Jersey City, NJ · Hybrid · ...
-
Data Engineer
2 weeks ago
A.M. Best Jersey City, United StatesOverview · The Data Engineer will build data pipelines / data orchestrator and data structures for existing operational data stores, Azure data lake & data warehouse. They will develop proof-of-concepts to evaluate new technologies and suggest improved and better ways to re-arch ...
-
Data Engineer
2 weeks ago
Hire Talent Jersey City, United StatesPGIM Fixed Income - Data Engineer · PGIM is the diversified asset management business of Prudential Financial, Inc. (NYSE: PRU). With $1.4 trillion in assets under management, and 38 offices spanning 16 countries, PGIM is among the world's top-10 largest asset managers. Comprised ...
-
Data Engineer
2 weeks ago
ICONMA Jersey City, United StatesAbility to find the root cause of the issue and come up with a solution and to use business requirements to document clear and concise technical design. · Hands-on experience in integration development, testing, and deployment experience · Exposure to disparate databases like T ...
Data Engineer - Jersey City, United States - ClifyX
Description
Job title :
Data Engineer - Databricks
Location :
New Jersey (Remote)
Duration :
Contract / Fulltime
Health Care Company
$75/Hr (Contract)
salary K/Annum (Fulltime)
Job Description: