Lead Data Engineer - Jersey City, United States - Chubb

    Chubb
    Chubb Jersey City, United States

    3 weeks ago

    Default job background
    Description
    Job Description

    A Lead Data Engineer generally works on implementing complex big data projects with a focus on collecting, parsing, managing, analyzing, and visualizing large sets of data to turn information into insights using multiple platforms. He or she should be able to decide on the needed hardware and software design needs and act according to the decisions. The big data engineer should be able to develop prototypes and proof of concepts for the selected solutions.

    Ideal candidate for this role is someone with a strong background in computer programming, statistics, and data science who is eager to tackle problems with large, complex datasets using the latest python, R, and/or Spark. You are a self-starter who will take ownership of your projects and deliver high-quality data-driven analytics solutions. You are adept at solving diverse business problems by utilizing a variety of different tools, strategies, algorithms, and programming languages.

    Specific responsibilities are as follows:
    • Strong analytical & logical skills
    • Utilize the data engineering skills within and outside of the developing Chubb information ecosystem for discovery, analytics and data management.
    • Work with data scientists, architects, business partners and business analysts to understand requirements, design and build effective solutions.
    • Understanding of P&C insurance processes, risk data attributes and concepts such as Limits, Exposure bases, Coverages, Packaged, Rating Factors, etc. Experienced in identifying, capturing, profiling & analyzing data from multiple sources (internal and external).
    • Skilled in identifying remediation solutions for addressing data issues. Experience in documenting data capture requirements, source to target mappings, data flow diagrams, entity relationships and complex data models. Experienced in translating business needs into systems requirements including rating systems, reports, dashboards and scorecards with minimal or no supervision.
    • Work with various relational and non-relational data sources with the target being Azure based SQL Data Warehouse & Cosmos DB repositories.
    • Work closely with the Data Science team to perform complex analytics and data preparation tasks.
    • Sourcing data from multiple applications, profiling, cleansing, and conforming to create master data sets for analytics use.
    • Experience with Complex Data Parsing (Big Data Parser) and Natural Language Processing (NLP) Transforms on Azure a plus.
    • Design solutions for managing highly complex business rules within the Azure ecosystem.
    Skills Required
    • Knowledge of Azure, Hadoop 2.0 ecosystems, HDFS, MapReduce, Hive, Pig, Sqoop, Mahout, Spark etc. a must
    • Experience with Web Scraping frameworks (Scrappy or Beautiful Soup or similar)
    • Extensive experience working with Data APIs (Working with RESTful endpoints and/or SOAP)
    • Knowledge of any commercial distribution like Horton Works, Cloudera, MapR, DataBricks etc. a must
    • Excellent working knowledge of relational databases, MySQL, Oracle etc.
    • Hands on experience of ETL tools like Informatica and SSIS is preferred.
    • Experience with Complex Data Parsing (Big Data Parser) a must. Should have worked on XML, JSON and other custom Complex Data Parsing formats.
    • Natural Language Processing (NLP) skills
    • Good knowledge of Python libraries like Pandas, NumPy, scikit-learn etc.
    • Ready to learn new technologies and tools.
    Qualifications
    • Bachelors in computer science or related educational background.
    • Prior experience of Insurance domain a huge plus
    • 10+ years of experience
    About Us

    Chubb is a world leader in insurance. With operations in 54 countries, Chubb provides commercial and personal property and casualty insurance, personal accident and supplemental health insurance, reinsurance, and life insurance to a diverse group of clients. The company is distinguished by its extensive product and service offerings, broad distribution capabilities, exceptional financial strength, underwriting excellence, superior claims handling expertise and local operations globally.

    At Chubb, we are committed to equal employment opportunity and compliance with all laws and regulations pertaining to it. Our policy is to provide employment, training, compensation, promotion, and other conditions or opportunities of employment, without regard to race, color, religious creed, sex, gender, gender identity, gender expression, sexual orientation, marital status, national origin, ancestry, mental and physical disability, medical condition, genetic information, military and veteran status, age, and pregnancy or any other characteristic protected by law. Performance and qualifications are the only basis upon which we hire, assign, promote, compensate, develop and retain employees. Chubb prohibits all unlawful discrimination, harassment and retaliation against any individual who reports discrimination or harassment.