Jobs
>
Newark

    Lead Data Engineer - Newark, United States - Hire Talent

    Hire Talent
    Hire Talent Newark, United States

    4 weeks ago

    Default job background
    Description
    Lead Data Engineer VMS Prudential
    Level 3 (potentially level 4)
    Right to hire (conversion is possible)
    Long term requirement (Initial couple months will want to join them FT)
    Narayana Murthy is HM
    Interview process: about 3 rounds of interviews

    Notes:
    • Chief Data Office
    • Provided data to entire CEO offices.
    • EDP corporate team
    • AWS platform, Pyspark, Hadoop
    • Understanding technical requirements and give guidelines to the team.
    • Help implement the development structure for the team.
    • Acts as a bridge for development.
    • 2-3 members he will be leading.
    • AWS Solution architect certified is helpful. (nice to have but very useful)
    • Candidates need to understand why the solution is built and how does it solve the problem.
    • Sourcing the problem, creating a solution, looking to the market to create a solution (industry standard/new tech), implement the solution and look at.
    • Need to build end to end solutions with Pyspark, AWS glue, API Calls (Fullstack dev comes into play here)
    • They already have a pre-built data platform.
    • No coding at all, needs to know what needs to be done. Will work with the business and work as a bridge.
    • Bridge for EDP corporate team
    • Need to be able to explain the how they build pipeline and why they used such technology
    Struggles
    • Candidates do not understand the business problem, what is the solution.
    • Candidates have never done solutions themselves.
    9p Manager (Data Engineering Lead, Data Team) - Engineering lead to help the design/build of the CDO enterprise data projects.
    Responsibilities
    Implement and support
    • end-to-end data lake/warehousing/mart/business intelligence/ analytics/ services solutions (ingest, storage, integration, processing, services, access) in AWS
    • data lake data intake/request/onboarding services and service documentation
    • data lake ingestion services for batch/real time data ingest and service documentation
    • data processing services (ETL/ELT) for batch/real time (Glue/Kinesis/EMR) and service documentation
    • data storage services for data lake (S3)/ data warehouses (RDS/Redshift)/ data marts and service documentation
    • data services layer including Athena, Redshift, RDS, microservices and APIs
    • pipeline orchestration services including lambda, step functions, MWAA (optional)
    • data security services (IAM/KMS/SM/encryption/anonymization/RBAC) and service documentation
    • data access provisioning services (Accounts, IAM Roles RBAC), processes, documentation and education
    • data provisioning services for data consumption patterns including microservices, APIs and extracts
    • metadata capture and catalog services for data lake(S3/Athena), data warehouses (RDS/Redshift), Microservices/APIs
    • metadata capture and catalog services for pipeline/log data for monitoring /support
    • Implement CI/CD pipelines
    • Prepare documentation for data projects utilizing AWS based enterprise data platform
    • Implement high velocity streaming solutions using Amazon Kinesis, SQS, and SMS
    • Migrate data from traditional relational database systems to AWS relational databases such as Amazon RDS, Aurora, and Redshift
    • Migrate data from traditional file systems and NAS shares to AWS data lake (S3) and relational databases such as Amazon RDS, Aurora, and Redshift
    • Migrate data from APIs to AWS data lake (S3) and relational databases such as Amazon RDS, Aurora, and Redshift
    • Provide cost/spend monitoring reporting for AWS based data platform data initiatives
    • Provide governance/audit reporting for access of AWS based data platform
    • Lead the implementation of a data lake strategy to enable LOBs and Corporate Functions with a robust, holistic view of data - driven decision making
    • Serve as delivery lead for EDP data initiatives product owner
    • Partner with immediate engineering team, product owner, IT, partners on EDP agenda
    • Provide technology thought leadership, consulting, and coaching/mentoring
    • Establish development, qa, stage and production migration/support processes
    • Establish best practices for development and support teams
    • Deliver end-end data initiatives from ingest-consume via microservices/ apis, jdbc/ odbc, file extracts etc.
    • Work with scrum master to develop and own backlog, stories, epics, sprints
    Qualifications
    • Bachelor's degree in Computer Science, Software Engineering, MIS or equivalent combination of education and experience
    • Experience implementing, supporting data lakes, data warehouses and data applications on AWS for large enterprises
    • Programming experience with Java, Python/ Scala, Shell scripting
    • Solid experience of AWS services such as CloudFormation, S3, Glue, EMR/ Spark, RDS, Redshift, DynamoDB, Lambda, Step Functions, IAM, KMS, SM etc.
    • Solid experience implementing solutions on AWS based data lakes
    • Experience implementing metadata solutions leveraging AWS non-relational data solutions such as ElastiCache and DynamoDB
    • AWS Solutions Architect or AWS Big Data Certification preferred
    • Experience in AWS data lake/data warehouse/business analytics
    • Experience and understanding of various core AWS services such as IAM, Cloud Formation, EC2, S3, EMR/Spark, Glue, Datasync, CloudHealth, CloudWatch, Lambda, Athena, and Redshift
    • Experience in system analysis, design, development, and implementation of data ingestion pipeline in AWS
    • Experience with DevOps and Continuous Integration/ Delivery (CI/ CD) concepts and tools
    • Experience with business intelligence tools such as Tableau, Power BI or equivalent
    • Knowledge of ETL/ ELT
    • Experience in production support from Level 1 to Level 3
    • Awareness of Data Management & Governance tools
    All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender, identity, national origin, disability, or protected veteran status.
  • Peer Consulting Resources Inc.

    AWS Data Engineer

    1 week ago

    Peer Consulting Resources Inc. Newark, United States

    Contact Details · Name: Savitha Chinnappa · Email: savitha.- · Cell: · Job Title: AWS Data Engineer (On W2) · Location: Newark, NJ (Onsite) · Duration: Months (Possibility of Extension) · Years of Experience: 7+ Yrs · Job Description: · AWS Certifications · Data pipeline (inges ...

  • JSR Tech Consulting

    Sr. Data Engineer

    4 weeks ago

    JSR Tech Consulting Newark, United States

    Job Description : · 8p - Engineering lead to help the design/build of the CDO enterprise data platforms · Design, Develop and ImplementEnd-to-end data solutions (ingest, storage, integration, processing, access) on AWS. · Data intake/ request/ onboarding services and service d ...

  • Phaxis

    Azure Data Engineer

    2 weeks ago

    Phaxis Newark, United States

    Qualifications: · 3+ yrs of experience in building data and analytics platform focused on Azure data and analytics solutions · Proficiency in Azure suite of services, including Azure SQL Database, Azure Data Factory, Azure Synapse Analytics, and Azure Data Lake · Experience imple ...

  • Jarvis Walker

    Data Engineer

    1 week ago

    Jarvis Walker Jersey City, United States

    Data Engineer · We are seeking an enthusiastic and experienced Data Engineer to join our team. As a Data Engineer, you will be a key member of our product development team. You will play a pivotal role in turning data into actionable models that our company uses to create new lin ...

  • YASH Technologies

    Data Engineer

    2 weeks ago

    YASH Technologies Jersey City, United States

    Hi, · We at Yash Technologies are looking for Data Engineer, if you are looking for new opportunity, please share your updated resume. · Requirement Details: · Location: Jersey City, NJ · Work Authorizations: Citizen, GC EAD, Green Card, H4 EAD · Position Type: Contract · Job Ti ...

  • YASH Technologies

    Data Engineer

    2 weeks ago

    YASH Technologies Jersey City, United States

    Looking for 10+ yrs Senior Resource · Mandatory Skills · Oracle · Mongo DB · Data Engineer · NO SQL database · Data Modeling · The Senior Database Engineer(MongoDB) will lead complex or critical database management initiatives that typically involve multiple disciplines and busin ...

  • Iris Software Inc.

    AWS Data Engineer

    4 weeks ago

    Iris Software Inc. Newark, United States

    Iris's client, a large insurance company, is currently searching for a strong for AWS Data Engineer. · Our client has offices all over the world and has the global reach, advisory services, and distribution power to meet the needs of issuers and investors worldwide. · Job title: ...

  • Insight Global

    Intl Data Engineer

    16 hours ago

    Insight Global Newark, United States

    Design new and modernize existing applications in support of the Actuarial Transformation initiative · Work with development teams and product owners/managers to build software solutions. · Implement solutions using AWS cloud services. · Write effective APIs. · Work with large v ...

  • Epsilon

    Senior Data Engineer

    3 days ago

    Epsilon Harrison, United States

    Job Description · About Epsilon PeopleCloud Discovery · Epsilon PeopleCloud Discovery empowers users with a deep understanding of millions of people-on a marketing platform that unites audience insights, segmentation, creative analysis, media planning and digital activation. It ...

  • MDMS Recruiting LLC

    Lead Data Engineer

    1 week ago

    MDMS Recruiting LLC Newark, United States

    Job Description · Job Description***W2 or Self Incorporated consultants Only*** No C2C/No Corp to Corp*** · Lead Data Engineer for AWS data platform (to engineer applications integrating with enterprise data platform using AWS) · Responsibilities: · Designing, building and mainta ...

  • Phaxis LLC

    Azure Data Engineer

    1 week ago

    Phaxis LLC Newark, United States

    Pay rate is $70 to $80 per hour · 3 days remote and 2 days onsite · Must Have Qualifications: · " 5+ years of experience in building data and analytics platform focused on Azure data and analytics solutions · " 5+ years of programming experience in Python/Pyspark · " Good experie ...

  • Exiger

    Data Engineer

    16 hours ago

    Exiger Jersey City, United States

    Product is seeking a Data Engineer to join our team. You will work closely with product management, sprint managers, and other members of the engineering team to design, develop, test, and deliver data pipelines and bespoke data flow integrations for clients; create product based ...

  • Spectraforce Technologies

    Data Engineer

    4 weeks ago

    Spectraforce Technologies Newark, United States

    Job Title: Data Engineer · Location: Newark, NJ (Hybrid) · Duration: 6 Months (Temp to Hire) · Pay Rate starts from: $55/hr. on W2 · Job Description: · Data Tech team is looking for a Data Engineer to join a diverse team dedicated to providing best in class data services to ou ...

  • Tiger Analytics

    Data Engineer

    3 days ago

    Tiger Analytics Jersey City, United States

    Job Description · Job DescriptionTiger Analytics is a fast-growing advanced analytics consulting firm. Our consultants bring deep expertise in Data Science, Machine Learning and AI. We are the trusted analytics partner for multiple Fortune 500 companies, enabling them to generate ...

  • Phaxis

    Data Engineer

    2 weeks ago

    Phaxis Newark, United States

    Junior Data Engineer (3+ years): Salary is $110k-$140k + bonus · Senior Data Engineer (5+ years): Salary is $135k-$155k + bonus · 3 days remote and 2 days onsite in Newark, NJ · All candidates will need to complete a Python assessment · Must Have Qualifications: · 3+ experience i ...

  • Proit

    Data Engineer

    4 weeks ago

    Proit Newark, United States

    Job DescriptionSkill: AWS Data Engineer · * Must have AWS Glue, Lambda, Athena, Pyspark, Redshift, EMR, AWS RDB, AWS DynamoDB, AWS Glacier, Python. · Experience Required: · * Minimum 5 years of professional experience in data integration and management. · * 3+ years of hands ...

  • Dyania Health

    Data Integration Engineer

    1 week ago

    Dyania Health Jersey City, United States Permanent

    · About Dyania Health · Dyania Health is a venture-backed company founded in 2019 which has developed Synapsis AI, an end-to-end system that combines a medically pre-adapted LLM with a physician-driven algorithmic reasoning engine to understand and assess clinical characteristic ...

  • Stratacent

    Big Data Engineer

    6 days ago

    Stratacent Jersey City, United States Full time / Contract

    We seek an experienced Senior Big Data Engineer to lead and enhance our data engineering initiatives. The ideal candidate will possess advanced expertise in Impala, Spark, Scala, Hive, Hadoop, and other Big Data technologies. You will be responsible for designing, implementing, a ...

  • NAM Info Inc

    Senior Data Engineer

    3 weeks ago

    NAM Info Inc Jersey City, United States

    Lead Data Engineer with Salesforce (CRM), Payments · Jersey City, NJ · 12+ Months · Candidate must be on our payroll (W2) · Must have Investment Banking Exp · Skillset: · AWS, Java, Python, Pyspark, Big Data, Databricks · Must be hands on and an excellent communicator · Must be ...

  • Rose International

    Data Architect, Engineer

    1 week ago

    Rose International Jersey City, United States

    Date Posted: 05/31/2024 · Hiring Organization: Rose International · Position Number: 465057 · Job Title: Data Architect, Engineer · Job Location: Jersey City, NJ, USA, 07310 · Work Model: Hybrid · Shift: 3 Days in Office · Employment Type: Temporary · Estimated Duration (In month ...