Databricks Engineer Lead - Dublin, United States - Prodware Solutions

Prodware Solutions Dublin, United States

3 weeks ago

Technology / Internet

Description

Job Details:

Position: Sr. Databricks Architect / Engineer Lead

Location: Columbus Metro area, OH

Duration: Full Time Opportunity

Need to travel every other week (twice a month) 3-4 days each trip --- Columbus Metro area, OH

Travel expenses will be paid

Required Skills:

Azure Databricks, Pyspark, SQL, CI/CD Pipelines for Databricks, ETL Processes, Project Management, Delta Lake Storage

Strong hands on SQL query writing/Coding experience.

Very strong on SQL coding - SQL is a must have, hands on able to write complex queries during interview.

Very Strong on Databricks – must have

Role & Responsibilities Overview:

· Develop and optimize ETL pipelines from various data sources using Databricks on the cloud (AWS, Azure, MicroStrategy, etc.)

· Experienced in implementing standardized pipelines with automated testing, Airflow scheduling, Azure DevOps for CI/CD, Terraform for infrastructure as code, and Splunk for monitoring

· Continuously improve systems through performance enhancements and cost reductions in compute and storage

· Data Processing and API Integration: Utilize Spark Structured Streaming for real-time data processing and integrate data outputs with REST APIs

· Lead Data Engineering Projects to manage and implement data-driven communication systems

· Experienced with Scrum and Agile Methodologies to coordinate global delivery teams, run scrum ceremonies, manage backlog items, and handle escalations

· Integrate data across different systems and platforms

· Strong verbal and written communication skills to manage client discussions

Candidate Profile:

· 8+ years experience in developing and implementing ETL pipelines from various data sources using Databricks on cloud

· Some experience in the insurance domain/ data

· Programming Languages – SQL, Python

· Technologies - IaaS (AWS or Azure or GCP), Databricks platform, Delta Lake storage, Spark (PySpark, Spark SQL).

o Good to have - Airflow, Splunk, Kubernetes, Power BI, Git, Azure Devops

· Project Management using Agile, Scrum

· B.S. Degree in a data-centric field (Mathematics, Economics, Computer Science, Engineering or other science field), Information Systems, Information Processing or engineering.

· Excellent communication & leadership skills, with the ability to lead and motivate team members

Lead Software Engineer

5 days ago

OCLC Dublin, United States Full time

You have a life. We like that about you. · At OCLC, we believe you'll do the best work of your life when you're living the best life possible. · We work hard to build the technology that connects thousands of today's libraries. But we also work hard to make a job at OCLC a meanin ...
Lead Software Engineer

3 weeks ago

OCLC Dublin, United States

You have a life. We like that about you. · At OCLC, we believe you'll do the best work of your life when you're living the best life possible. · We work hard to build the technology that connects thousands of today's libraries. But we also work hard to make a job at OCLC a meanin ...
Lead Engineer

3 weeks ago

Wendy's Dublin, United States

Why Wendy's · When our square shaped burgers made their first sizzle on the scene more than 50 years ago, people knew our approach wasn't like any other. Same goes for the way we support our employees. Our culture of openness, flexibility, and inclusiveness allows everybody to fl ...
Software Engineer Tech Lead

3 days ago

Central Mutual Insurance Company Dublin, United States

Hybrid Work Model- Van Wert or Dublin, OH · Technology plays a crucial role in our ability to deliver best-in-class insurance products. We are currently seeking a Tech Lead to join our Core Policy & Underwriting team. This mission-critical system offers complex technical challen ...
Lead DevOps Engineer

3 weeks ago

OCLC Dublin, United States

You have a life. We like that about you. · At OCLC, we believe you'll do the best work of your life when you're living the best life possible. · We work hard to build the technology that connects thousands of today's libraries. But we also work hard to make a job at OCLC a mean ...
Lead Network Engineer

2 weeks ago

Apex Systems Dublin, United States

Job#: · Job Description: · Title: Lead Network Engineer · How many openings? · 1 opening · Contract Length: 6 months with plan to convert to full time- if they are on the fence, they will extend to 12 months. · Hours: (Ask about OT)This person must embrace afterhours and weekend ...
Lead IT Service Engineer

6 days ago

OCLC Dublin, United States

You have a life. We like that about you. · At OCLC, we believe you'll do the best work of your life when you're living the best life possible. · We work hard to build the technology that connects thousands of today's libraries. But we also work hard to make a job at OCLC a mean ...
Lead Systems Engineer

4 weeks ago

OCLC Dublin, United States

You have a life. We like that about you. · At OCLC, we believe you'll do the best work of your life when you're living the best life possible. · We work hard to build the technology that connects thousands of today's libraries. But we also work hard to make a job at OCLC a mean ...
Lead Systems Engineer

3 weeks ago

OCLC Dublin, United States Full time

You have a life. We like that about you. · At OCLC, we believe you'll do the best work of your life when you're living the best life possible. · We work hard to build the technology that connects thousands of today's libraries. But we also work hard to make a job at OCLC a meanin ...
Lead IT Service Engineer

5 days ago

OCLC Dublin, United States

You have a life. We like that about you. · At OCLC, we believe you'll do the best work of your life when you're living the best life possible. · We work hard to build the technology that connects thousands of today's libraries. But we also work hard to make a job at OCLC a mean ...
Lead Systems Engineer

1 week ago

OCLC Dublin, United States

You have a life. We like that about you. · Interested in this role You can find all the relevant information in the description below. · At OCLC, we believe you'll do the best work of your life when you're living the best life possible. · We work hard to build the technology that ...
Lead Software Engineer

1 week ago

OCLC Dublin, United States

You have a life. We like that about you. · A high number of candidates may make applications for this position, so make sure to send your CV and application through as soon as possible. · At OCLC, we believe you'll do the best work of your life when you're living the best life po ...
Plant Services Engineering Lead

1 week ago

Mill Run Hilliard, United States

· Overview · At ADS, no idea is off limits, and we celebrate creativity and bold moves. For more than 50 years ADS has been manufacturing a variety of innovative and environmentally friendly alternatives to traditional materials. Headquartered in Hilliard, Ohio, we are a multi- ...
Plant Services Engineering Lead

1 week ago

Mill Run Hilliard, United States

Overview · At ADS, no idea is off limits, and we celebrate creativity and bold moves. For more than 50 years ADS has been manufacturing a variety of innovative and environmentally friendly alternatives to traditional materials. Headquartered in Hilliard, Ohio, we are a multi-bill ...
CyberArk Lead Engineer

2 weeks ago

Western Alliance Bank Columbus, United States Full time

Job Title: · CyberArk Lead Engineer Location: · CityScape What you'll do: · We are seeking a motivated and skilled CyberArk Engineer to join our Cyber Security team. In this role, you will be responsible for the day-to-day operations and maintenance of our CyberArk Privilege ...
Lead DevOps Engineer

5 days ago

NICE Columbus, United States

At NICE, we don't limit our challenges. We challenge our limits. Constantly. We're relentless. We're ambitious. And we make an impact. Our NICErs bring their A game and spend each day turning it into an A+. And if you're like us, we can offer you the kind of challenge that will l ...
Lead Engineer

4 hours ago

Newmark ltd Columbus, United States

Job Description · Operates and maintains all building systems. Responsibilities will becarried out in a manner consistent with high quality maintenance and economical operation of the building's equipment and systems. Ability to work independently most times with noon-site super ...
Engineering Team Lead

1 week ago

Cyient Columbus, United States Full time

Description · Lead a team of Manufacturing Engineers and support staff. Develop and maintain strong lines of communication with customers. · Responsibilities: · Successfully manage multiple tasks across several different job locations. · Organize daily delegation of job assignm ...
Production Engineer

1 week ago

BlueSky Innovations Dublin, United States

Heating with a clear conscience - wouldn't that be great? That's where we come in. With over 90 years of experience, we, as the Japanese Daikin Group, develop innovative and environmentally friendly heating and climate solutions and distribute them worldwide. We are the global ma ...
Plant Services Engineering Lead

3 weeks ago

Mill Run Hilliard, United States

Job Description · Overview · At ADS, no idea is off limits, and we celebrate creativity and bold moves. For more than 50 years ADS has been manufacturing a variety of innovative and environmentally friendly alternatives to traditional materials. Headquartered in Hilliard, Ohio, w ...