Jobs
>
Pittsburgh

    Data Engineer - Pittsburgh, United States - InfoGravity

    InfoGravity
    InfoGravity Pittsburgh, United States

    3 weeks ago

    Default job background
    Technology / Internet
    Description

    Data Engineer - ETL with Python, Spark & Hadoop

    Location: Pittsburgh only (Hybrid 2-3 days a week).

    Citizens, GC, or L2S

    Note: This is a Hyrbid role. The Hiring Manager requires the individual to be onsite for 2-3 days a week. And they must be willing to go fulltime after a year, salary can be negotiated with hiring manager while converting to their fulltime from our payroll. There are no sponsorships available for this role.

    Responsibilities:

    • Organize business needs into ETL/ELT logical models and ensure data structures are designed for flexibility to support the scalability of business solutions.
    • Craft and implement data pipelines utilizing Spark and Python
    • Define and deliver reusable components for the ETL/ELT framework.
    • Define optimal data flow for system integration and data migration.
    • Integrate new data management technologies and software engineering tools into existing structures.
    • Design, build, and maintain CI/CD pipelines in multiple integration and test environments.
    • Install, configure, and manage automated testing tools in the environment.

    Qualifications:

    • Experienced in the Design, Development, and Implementation of large-scale projects in financial industries using Data Warehousing ETL tools(Spark)
    • Experience in creating ETL transformations and jobs using PySpark and automating workflows using Orchestration tools like Airflow, Control-M
    • Strong knowledge and experience in SQL, Python, and Spark
    • Experience with Big Data/distributed frameworks such as Spark, Kubernetes, Hadoop, and Hive
    • Ability to design ETL/ELT solutions based on user reporting and archival requirements
    • Strong sense of customer service to consistently and effectively address client needs
    • Self-motivated; comfortable working independently under general direction
    • Hands-on experience in building and managing CI/CD pipelines
    • Basic knowledge of Azure Cloud Components
  • Govini

    Data Engineer

    1 week ago

    Govini Pittsburgh, United States Full time

    Company Description · Govini is the leading Commercial Data company in the Defense Technology space. Built by Govini, is used at scale across the National Security sector of the federal government. This platform enables government analysts, program managers, and decision-makers ...

  • Software Engineering Institute

    Data Engineer

    1 week ago

    Software Engineering Institute Pittsburgh, United States

    What We Do · The CERT Cyber Mission Readiness (CMR) directorate fortifies the U.S. and its partners against cyber threats. We design large and small-scale exercises, facilitate tabletop scenarios, run cybersecurity contests, and create open-source tools to bolster cyber defense ...

  • Accion Labs

    Data Engineer

    2 weeks ago

    Accion Labs Pittsburgh, United States

    Those authorized to work in the United States without sponsorship are encouraged to apply.We are unable to sponsor visas for this position. No H1B's. · Our direct client is an American chain of convenience stores started interviewing Data Engineering Pittsburgh, PA (Hybrid – 2 da ...

  • Innovation Works, Inc.

    Data Engineer

    4 weeks ago

    Innovation Works, Inc. Pittsburgh, United States

    Job Description · Job DescriptionABOUT IDELIC · Idelic uses cutting-edge technology and beautifully designed interfaces to help predict and prevent trucking accidents and reduce driver turnover to ensure drivers get home safely each and every night. Our SaaS solutions are radical ...

  • Tekfortune Inc

    Data Engineer

    2 days ago

    Tekfortune Inc Pittsburgh, United States

    Python(pyspark), jupyter hub, airflow, Java · Database knowledge - Oracle / SQL / Data Analytics · Regulatory reporting knowledge preferred · Good communication skills · Open / Ability to learn new technologies · Very good Analytical Skills · Experience in ETL is preferable · ...

  • Synechron

    Data Engineering Architect

    5 days ago

    Synechron Pittsburgh, United States

    Title: Data Engineering Architect (Neo4j and PySpark) · Location: Pittsburgh, PA · Job Type: Full time · About the role: The Data Architect will specifically help to generate and assess the calculated data quality, Risk Tech Analyst also supports Quants and Risk models validation ...

  • Free Market Health

    Manager, Data Engineering

    2 weeks ago

    Free Market Health Pittsburgh, United States

    Job Description · Job DescriptionSalary: $110,000-$120,000 · Overview · As part of the Technology Services team at Free Market Health, the Manager of Data Engineering will be responsible for helping to guide a team that will implement, grow, and optimize our data intake process, ...

  • University of Pittsburgh

    Data Engineer II

    2 days ago

    University of Pittsburgh Pittsburgh, United States

    · Data Engineer II · Job Summary · Job Title: Data Engineer II · Job Location: Pittsburgh, PA · Job Description: Support data acquisition by locating datasets and software produced by MIDAS researchers through online searching, reviewing the scientific literature, and communi ...

  • EOS USA

    Senior Data Engineer

    3 days ago

    EOS USA Pittsburgh, United States

    WHO WE ARE: · EOS IT Solutions is a Global Technology and Logistics company, providing Collaboration and Business IT Support services to some of the world's largest industry leaders, delivering forward-thinking solutions based on multi-domain architecture. Customer satisfaction ...

  • Motional

    Data Pipeline Engineer

    3 weeks ago

    Motional Pittsburgh, United States

    Mission Summary · We are seeking a talented Data Pipeline Engineer to implement robust large scale data processing and transformation pipelines for our Data Pipelines team under Data Platform. · The Data Platform builds a scalable compute platform that is used throughout the com ...

  • Eaton

    Specialist - Data Engineer

    2 weeks ago

    Eaton Pittsburgh, United States

    Eaton's IT team is currently seeking a Specialist – Data Engineer to join team. This position will be based at our Beachwood, Ohio, Moon TWP, PA, Houston, TX, Galesburg, MI, Raleigh, NC or Menomonee Falls, WI. The position is eligible for hybrid work setup. · The expected annual ...

  • PDF Solutions

    Senior Data Engineer

    1 week ago

    PDF Solutions Pittsburgh, United States

    Overview: · PDF was recognized by Forbes as one of America's Best Small Employers for 2023. This ranking highlights only 300 companies out of 10,000+ that were considered in the Forbes nationwide analysis and PDF solutions ranked 55 The results are based on an employee satisfact ...

  • Synergy Staffing

    AI Data Engineer

    2 days ago

    Synergy Staffing Pittsburgh, United States

    Job Description · Job Description · In this role you will democratize AI through the enterprise ML platform, accelerate the adoption of Generative Artificial Intelligence (Gen AI) and drive Responsible AI. · Youll enjoy the flexibility to work remotely * from anywhere within t ...

  • DeRisk Technologies

    Data Center Engineer

    2 weeks ago

    DeRisk Technologies Pittsburgh, United States

    Job Description · Job DescriptionJob Responsibilities: · Deployment / In-Scope Configuration Items · • Servers (Virtual & Physical) · • Storage & Backup Devices · • Server Appliances · • Hyper Converged Infrastructure (E.g. Nutanix, Cisco UCS) · • Tape Storage Units · • Pow ...

  • Diverse Lynx

    Snowflake Data Engineer

    2 weeks ago

    Diverse Lynx Pittsburgh, United States

    JOB DESCRIPTION: · Experience in Snowflake data pipelines. · Performance. · Optimization. · Snowflake development tasks. · Authentication & Authorization. · Good communication skills, work in a team. · Diverse Lynx LLC is an Equal Employment Opportunity employer. All quali ...

  • Sheetz

    Data Engineer

    3 weeks ago

    Sheetz Pittsburgh, United States

    OVERVIEW · Responsible for developing complex, large scale data models and pipelines that · organize and standardize the data to make it readily accessible and consumable by the business for · reporting and data science needs. Collaborate with various areas of the Business in ...

  • Armada

    Data Engineer

    2 weeks ago

    Armada Pittsburgh, United States

    SUMMARY · As a Data Engineer for our Digital Services division, you will drive the evolution of our next generation data infrastructure, which powers a wide range of analytics and software products. You will be responsible for designing, developing, and maintaining advanced data ...

  • Accion Labs

    Data Engineer

    2 weeks ago

    Accion Labs Pittsburgh, United States

    Those authorized to work in the United States without sponsorship are encouraged to apply.We are unable to sponsor visas for this position. No H1B's. · All the relevant skills, qualifications and experience that a successful applicant will need are listed in the following descri ...

  • InfoGravity LLC

    Data Engineer

    2 weeks ago

    InfoGravity LLC Pittsburgh, United States

    Data Engineer - ETL with Python, Spark & Hadoop · Location: Pittsburgh only (Hybrid 2-3 days a week). · Citizens, GC, or L2S · Ensure you read the information regarding this opportunity thoroughly before making an application. · Note: This is a Hyrbid role. The Hiring Manager ...

  • Accion Labs

    Data Engineer

    2 weeks ago

    Accion Labs Pittsburgh, United States

    Those authorized to work in the United States without sponsorship are encouraged to apply.We are unable to sponsor visas for this position. No H1B's. · Our direct client is an American chain of convenience stores started interviewing Data Engineering Pittsburgh, PA (Hybrid – 2 d ...