Jobs
>
Plano

    Bigdata / Pyspark Developer - Plano, United States - Virtusa

    Virtusa
    Virtusa Plano, United States

    3 weeks ago

    Default job background
    Description
    Experience in building SparkStreaming process. Proficient in understanding distributed computing principles. Experience in managing Hadoop cluster with all services.
    Proficiency with Hadoop MapReduce HDFS Pig Hive and Impala. Experience with Nosql Databases and Messaging systems like Kafka. Designing building installing configuring and supporting Hadoop Perform analysis of vast data stores. Good understanding of cloud technology. Must have strong technical experience in Design Mapping specifications HLD LLD.

    Must have the ability to relate to both business and technical members of the team and possess excellent communication skills.

    Leverage internal tools and SDKs, utilize AWS services such as S3, Athena, and Glue, and integrate with our internal Archival Service Platform for efficient data purging.

    Lead the integration efforts with the internal Archival Service Platform for seamless data purging and lifecycle management. Collaborate with the data engineering team to continuously improve data integration pipelines, ensuring adaptability to evolving business needs.
    Performed database health checks and tuned the databases using Teradata Manager.
    Develop and maintain data platforms using Python
    Work with AWS and Big Data, design and implement data pipelines, and ensure data quality and integrity
    Collaborate with crossfunctional teams to understand data requirements and design solutions that meet business needs
    Implement and manage agents for monitoring, logging, and automation within AWS environments
    Handling migration from PySpark to AWS

    Experience in building SparkStreaming process. Proficient in understanding distributed computing principles. Experience in managing Hadoop cluster with all services.
    Proficiency with Hadoop MapReduce HDFS Pig Hive and Impala. Experience with Nosql Databases and Messaging systems like Kafka. Designing building installing configuring and supporting Hadoop Perform analysis of vast data stores. Good understanding of cloud technology. Must have strong technical experience in Design Mapping specifications HLD LLD.

    Must have the ability to relate to both business and technical members of the team and possess excellent communication skills.

    Leverage internal tools and SDKs, utilize AWS services such as S3, Athena, and Glue, and integrate with our internal Archival Service Platform for efficient data purging.

    Lead the integration efforts with the internal Archival Service Platform for seamless data purging and lifecycle management. Collaborate with the data engineering team to continuously improve data integration pipelines, ensuring adaptability to evolving business needs.
    Performed database health checks and tuned the databases using Teradata Manager.
    Develop and maintain data platforms using Python
    Work with AWS and Big Data, design and implement data pipelines, and ensure data quality and integrity
    Collaborate with crossfunctional teams to understand data requirements and design solutions that meet business needs
    Implement and manage agents for monitoring, logging, and automation within AWS environments
    Handling migration from PySpark to AWS
  • VirtusaPolaris - Virtusa Corporation

    Bigdata / Pyspark Developer

    3 weeks ago

    VirtusaPolaris - Virtusa Corporation Plano, United States

    Experience in building SparkStreaming process. Proficient in understanding distributed computing principles. Experience in managing Hadoop cluster with all services.Proficiency with Hadoop MapReduce HDFS Pig Hive and Impala. Experience with Nosql Databases and Messaging systems l ...

  • IT Vision Group

    Senior Pyspark Developer

    3 weeks ago

    IT Vision Group Dallas, United States

    Job Description · Job DescriptionTitle : Senior Pyspark Developer · Location: Remote · Duration: 12 Months+ · Job Description : · One of our clients is looking for Senior Python Pyspark Developer ...

  • Ascentt

    Mid-Senior Data Engineer

    3 weeks ago

    Ascentt Plano, United States

    Job Summary: We are seeking an experienced Mid-Senior Data Engineer (PySpark Engineer) to join our team of data professionals. In this role, you will collaborate closely with Data Scientists to prepare and transform large-scale datasets using PySpark, a popular open-source Python ...

  • Ascentt

    Senior PySpark Engineer

    4 weeks ago

    Ascentt Plano, United States

    Job Summary: We are seeking an experienced Senior PySpark Engineer to join our team of data professionals. In this role, you will collaborate closely with Data Scientists to prepare and transform large-scale datasets using PySpark, a popular open-source Python library for Apache ...

  • CrackaJack Digital Solutions

    Databricks DLT Lead Engineer

    1 week ago

    CrackaJack Digital Solutions Plano, United States

    Databricks DLT Lead Engineer · Plano, TX (Hybrid, 3 Days a week) · 12 Months · *** Local Candidates Highly Preferred** · Job Summary: · We are seeking an experienced Senior Databricks Engineer to join our team of data professionals. In this role, you will collaborate closely wit ...

  • Cognizant

    AWS PySpark Snowflake Engineer

    3 weeks ago

    Cognizant Plano, United States Full time

    We are Cognizant Artificial Intelligence · Digital technologies, including analytics and AI, give companies a once-in-a-generation opportunity to perform orders of magnitude better than ever before. However, clients need new business models built from analyzing customers and busi ...

  • ApTask

    PySpark SME

    1 week ago

    ApTask Plano, United States

    Job Title: PySpark SME · Location: Plano, Tx ( Candidates within 55 Miles) · Pay Rate: $65/hr C2C · Day 1 Hybrid · Job Description · Hands-on Pyspark SME with multiple project experience with Data planforms comprising of Hadoop, Teradata Data Warehouse, Ab Initio, Informatica, Ja ...

  • The AES Group

    Big Data Developer

    1 week ago

    The AES Group Plano, United States

    Title: Big Data Developer · Location: Plano/Irving, TX (Must be local) · Duration: Contract to Hire · Note: We reviewed multiple resumes in Q1 and every candidate we received was either a Data Engineer or an Analytics consultant. We need someone in a Hybrid role with superior pr ...

  • The AES Group

    Big Data Developer

    1 week ago

    The AES Group Plano, United States

    Absolute W2 Role · Job Title: Big Data Developer · Location: Plano/Irving, TX (Must be local) · C2H role · Only GC EADs, GC, and USC are acceptable · We need someone in a Hybrid role with superior programming skills · Job Description: · As a Big Data Developer, you will play a cr ...

  • Accord Technologies Inc.

    Sr Data Engineer

    1 week ago

    Accord Technologies Inc. Plano, United States

    Job Description · Job DescriptionSr Data Engineer (Pyspark/AWS/Databricks) · Plano TX (Hybrid Role) · Relocation can be considered · Pre-screen must be on Pyspark & AWS, Databricks. · I need someone with React, AWS & Pyspark with Databricks experience - THESE ARE MUST HAVE SKILL ...

  • HexaQuEST Global

    Data Engineer

    4 weeks ago

    HexaQuEST Global Plano, United States

    Treasury Data Platform is collection of tools and processes that manage and interact with data to orchestrate, source, transform, and make data available to consumers. This role requires an experienced and highly skilled software engineer to be part of the design and implementati ...

  • Wizcom Consulting

    Sr Data Engineer

    1 week ago

    Wizcom Consulting Plano, United States

    Job Description · Job DescriptionSr Data Engineer (Pyspark/AWS/Databricks) · Plano TX (Hybrid Role) · Relocation can be considered · W2 only (No C2C) · Pre-screen must be on Pyspark & AWS, Databricks. · I need someone with React, AWS & Pyspark with Databricks experience - THESE ...

  • The Judge Group

    Software Engineer

    1 week ago

    The Judge Group Plano, United States

    Title: React JS – Advanced · Location : Plano, TX Hybrid · need someone with AWS & Pyspark with Databricks experience · Description: · Expert in React JS and design technique as well as experience working across large environments with multiple operating systems/infrastructure f ...

  • Avacend Inc.

    Python Developer

    2 weeks ago

    Avacend Inc. Plano, United States

    Responsibilities · 2-4 years of experience developing Data engineering, and ad-hoc transformation of unstructured raw data · Use of orchestration tools · Design, build, and maintain workflows/pipelines to process a continuous stream of data with experience in end-to-end design an ...

  • AA SOFTWARE & NETWORKING PRIVATE LIMITED

    ETL/ Data Warehousing Developer

    2 weeks ago

    AA SOFTWARE & NETWORKING PRIVATE LIMITED Plano, United States

    Job Title: ETL/Data Warehousing Developer · Job Location: Plano, TX · Job Type: W2 Only · Job responsibilities: · Executes software solutions, design, development, and technical troubleshooting with ability to think beyond routine or conventional approaches to build solutions or ...

  • Avacend Inc.

    Python Developer

    3 weeks ago

    Avacend Inc. Plano, United States

    Responsibilities · 2-4 years of experience developing Data engineering, and ad-hoc transformation of unstructured raw data · Use of orchestration tools · Design, build, and maintain workflows/pipelines to process a continuous stream of data with experience in end-to-end design ...

  • Diverse Lynx

    AWS and Python and Java

    2 weeks ago

    Diverse Lynx Plano, United States

    Role: AWS and Python and Java (Microservices) Developer · Location - Plano TX, Northern VA (3 Days Office 2 Days remote) · Experience: 10+ Year · Duration: Long Term · Mandatory Skills: AWS and Python and Java (Microservices) · Job Description: · • Strong working Python, AW ...

  • Pinnacle Group, Inc.

    Developer (W2 Only)

    3 weeks ago

    Pinnacle Group, Inc. Plano, United States

    Role: React Developer · Location: Plano, TX- Hybrid 2/3 days in office · Duration: 6/12 Month Contract (Possible Extension or Hire) · W2 ONLY NO C2C · USC/GC only · Must have: · Must Have: (1) React JS (2) AWS red shift and data bricks (3) PySpark · React and AWS Cloud · Job Desc ...

  • Teleworld Solutions

    Software Engineer/Developer

    2 weeks ago

    Teleworld Solutions Plano, United States

    Overview · TeleWorld Solutions is seeking a experienced Software engineer with focus on Data Engineering, ETL processes, preferably with exposure to both batch and streaming data. The candidate should have familiarity with use of Databases and DataLake infrastructure and associat ...

  • Tata Consultancy Services

    Business Analyst

    2 weeks ago

    Tata Consultancy Services Plano, United States

    Job Title Data Engineering Tech LeadRelevant Experience (in Yrs yearsMust Have Technical/Functional Skills1. Strong communication2. Has closely worked with Business users in understanding their requirements and processes3. Working knowledge of python, Pyspark , ETLs , Data loadin ...