Databricks Architect - Jersey City, United States - Inizio Partners Corp

Inizio Partners Corp Jersey City, United States

2 weeks ago

Description

Job Description

Role & Responsibilities Overview:

Develop and optimize ETL pipelines from various data sources using Databricks on cloud (AWS, Azure, etc.)
Experienced in implementing standardized pipelines with automated testing, Airflow scheduling, Azure DevOps for CI/CD, Terraform for infrastructure as code, and Splunk for monitoring
Continuously improve systems through performance enhancements and cost reductions in compute and storage
Data Processing and API Integration: Utilize Spark Structured Streaming for real-time data processing and integrate data outputs with REST APIs
Lead Data Engineering Projects to manage and implement data-driven communication systems
Experienced with Scrum and Agile Methodologies to coordinate global delivery teams, run scrum ceremonies, manage backlog items, and handle escalations
Integrate data across different systems and platforms
Strong verbal and written communication skills to manage client discussions

Candidate Profile:

8+ years' experience in developing and implementing ETL pipelines from various data sources using Databricks on cloud
Based out of US
Some experience in insurance domain/ data is must
Programming Languages SQL, Python
Technologies - IaaS (AWS or Azure or GCP), Databricks platform, Delta Lake storage, Spark (PySpark, Spark SQL).
- Good to have - Airflow, Splunk, Kubernetes, Power BI, Git, Azure DevOps
Project Management using Agile, Scrum
B.S. Degree in a data-centric field (Mathematics, Economics, Computer Science, Engineering or other science field), Information Systems, Information Processing or engineering.
Excellent communication & leadership skills, with the ability to lead and motivate team members
Ability to work independently with some level of ambiguity and juggle multiple demands

Databricks Architect

3 weeks ago

Inizio Partners Jersey City, United States

Role & Responsibilities Overview: · Develop and optimize ETL pipelines from various data sources using Databricks on cloud (AWS, Azure, etc.) · Experienced in implementing standardized pipelines with automated testing, Airflow scheduling, Azure DevOps for CI/CD, Terraform for inf ...
Databricks Architect

3 weeks ago

Inizio Partners Jersey City, United States

Role & Responsibilities Overview: · Develop and optimize ETL pipelines from various data sources using Databricks on cloud (AWS, Azure, etc.) · Experienced in implementing standardized pipelines with automated testing, Airflow scheduling, Azure DevOps for CI/CD, Terraform for inf ...
Databricks Architect

1 week ago

Inizio Partners Jersey City, United States

About the job Databricks Architect · Role & Responsibilities Overview: · Develop and optimize ETL pipelines from various data sources using Databricks on cloud (AWS, Azure, etc.)Experienced in implementing standardized pipelines with automated testing, Airflow scheduling, Azure ...
Databricks Architect

1 week ago

AppLab Systems Inc Jersey City, United States

Hi, · Hope you are doing good... · I am · Baiju · represented to Applab Systems and I have an excellent job opportunity for you. Please find the below job description and if you are comfortable, please reply or contact me for further discussion. · Job Title: · Sr Databricks ...
Databricks Architect

2 weeks ago

ishare Inc Jersey City, United States

Hiring on behalf of a client for full time , direct hire role for · Databricks Architect / · 1) Hybrid in Jersey City · 2) Dublin , OH (Hybrid) | requres 2 days in month onstie to Dublin ( Travel Expenses Paid) · Role & Responsibilities Overview: · Develop and optimize ETL pipe ...
Databricks Architect

3 weeks ago

Zortech Solutions Jersey City, United States

Job Description · Job DescriptionRole: Databricks Architect · Location: NJ, New york · Experience: 8 to 15 · Must Have: · Azure Databricks. Pyspark, SQL, CI/CD Pipelines for Databricks, ETL Processes, Project Management, Delta Lake Storage · Nice to Have: · Airflow, Splunk, Kube ...
Databricks Architect

1 day ago

BSL consulting LLC Jersey City, United States

**Job Role: Databricks Architect** · **Location: Dublin, OH/Jersey City, NJ (Hybrid Role)** · **Job Type: Full Time(Permanent)** · **Skillsets/knowledge required**: · - 8+ years experience in developing and implementing ETL pipelines from various data sources using Databricks on ...
Databricks Architect

3 weeks ago

Acunor New Jersey, United States

Acunor is looking for Databricks Architect for one of its premier clients. it is fulltime and hybrid opportunity at NJ / OH. interested candidates are encouraged to apply with the most recent resume to · Position: Databricks Architect · Location: New Jersey / Dublin, OH (Hybrid) ...
Senior Data Architect, Databricks Expert

3 weeks ago

Harnham New York, United States

Senior Data Architect · Fully Remote · 6-Month Contract, possible extensions · $95-$115/hr · It's time to utilize your skills in software and data engineering to drive scientific and technological life-changing solutions within the pharmaceutical industry. This global company ...
Manager, Delivery Solutions Architect

1 week ago

Databricks New York, United States

FEQ225R31 · Reporting to the Sr Manager, Field Engineering for Healthcare & Life Sciences. You'll be responsible for building and leading a team of Delivery Solution Architects. DSA's lead the post sales technical strategy for our largest Enterprise customers, with a focus on ens ...
Data Governance Manager

4 days ago

MultiPlan Inc. New York, United States

JOB SUMMARY: · Reporting to the Director of Enterprise Data Management, this position is responsible for mobilization, operationalization, and execution of the data maturity program and co-architecting of the technical foundations of the new self-service, store front analytics & ...
Director Full Stack Engineering/architecture

1 day ago

S&P Global New York, United States

**About the Role**: · **Grade Level (for internal use)**: 13 · **About the Role**: · **The Team**: · S&P Global Ratings' data services group is a team of data and technology professionals who define and execute the strategic data roadmap. This team builds and manages a series of ...
Technical Architect

3 weeks ago

Tiger Analytics Jersey City, United States

Tiger Analytics is advanced analytics consulting firm. We are the trusted analytics partner for several Fortune 100 companies, enabling them to generate business value from data. Our consultants bring deep expertise in Data Science, Machine Learning and AI. Our business value and ...
Data Engineer

3 weeks ago

A.M. Best Jersey City, United States

Overview · The Data Engineer will build data pipelines / data orchestrator and data structures for existing operational data stores, Azure data lake & data warehouse. They will develop proof-of-concepts to evaluate new technologies and suggest improved and better ways to re-arch ...
Data Architect with Mongo DB

3 weeks ago

Tanisha Systems Jersey City, United States

Role : Data Architect with Mongo DB · Location : Jersey City, NJ Onsite · Hire Type : Full Time & Contract W2 · Data Architect with Mongo DB. · Expertise in Mongo DB Hands-on, · Performance Tuning on systems built on MongoDB · Experience in large highly scalable and resilient sy ...
Senior Data Engineer

3 weeks ago

Sumitomo Mitsui Banking Corp Jersey City, United States

SMBC Group is a top-tier global financial group. Headquartered in Tokyo and with a 400-year history, SMBC Group offers a diverse range of financial services, including banking, leasing, securities, credit cards, and consumer finance. The Group has more than 130 offices and 80,000 ...
Azure Data Architect

3 weeks ago

Visionet Systems Inc. New Jersey, United States

Visionet is seeking an experienced Azure Data Architect to design and implement end-to-end data solutions on the Azure platform. The ideal candidate will have a strong background in data architecture, modeling, Azure services, and data governance. They will collaborate with cross ...
AWS/Databricks Software Engineer

1 day ago

Capgemini New York, United States

Job Title: AWS/Databricks Software Engineer · Job location: New York NY. · Job Description: · Design and implement Azure cloud-based Data Warehousing and Governance architecture with Lakehouse paradigm Integrating technical functionality ensuring data accessibility accuracy and s ...
Onsite With Hybrid

6 days ago

Quantum World Technologies Inc. New York, United States

Role: DevOps Architect · Location: NYC, NY · Duration: Long Term Contract · Need experience in: · Terraform · CI/CD · Web App · AWS · Strong Containers exp like Kubernetes, Docker, etc. · Java or Python Programming Language exp . · Responsibilities: · Responsible for building ...
Lead Dotnet Developer

3 days ago

Amtex Systems Inc. New York, United States

.NET Lead · Location: Onsite at New York office (Monday-Friday) - Local Candidates only · Interview Process: 1 internal video interview & 1 potential client round · Lead software engineer eager to lead a team working on product development using dotnet/C#, MSSQL, AWS, containeriz ...