- Organize business needs into ETL/ELT logical models and ensure data structures are designed for flexibility to support the scalability of business solutions.
- Craft and implement data pipelines utilizing Spark and Python
- Define and deliver reusable components for the ETL/ELT framework.
- Define optimal data flow for system integration and data migration.
- Integrate new data management technologies and software engineering tools into existing structures.
- Design, build, and maintain CI/CD pipelines in multiple integration and test environments.
- Install, configure, and manage automated testing tools in the environment.
- Experienced in the Design, Development, and Implementation of large-scale projects in financial industries using Data Warehousing ETL tools(Spark)
- Experience in creating ETL transformations and jobs using PySpark and automating workflows using Orchestration tools like Airflow, Control-M
- Strong knowledge and experience in SQL, Python, and Spark
- Experience with Big Data/distributed frameworks such as Spark, Kubernetes, Hadoop, and Hive
- Ability to design ETL/ELT solutions based on user reporting and archival requirements
- Strong sense of customer service to consistently and effectively address client needs
- Self-motivated; comfortable working independently under general direction
- Hands-on experience in building and managing CI/CD pipelines
- Basic knowledge of Azure Cloud Components
-
Data Engineer
2 weeks ago
Govini Pittsburgh, United States Full timeCompany Description · Govini is the leading Commercial Data company in the Defense Technology space. Built by Govini, is used at scale across the National Security sector of the federal government. This platform enables government analysts, program managers, and decision-makers ...
-
Data Engineer
2 weeks ago
Software Engineering Institute Pittsburgh, United StatesWhat We Do · The CERT Cyber Mission Readiness (CMR) directorate fortifies the U.S. and its partners against cyber threats. We design large and small-scale exercises, facilitate tabletop scenarios, run cybersecurity contests, and create open-source tools to bolster cyber defense ...
-
Data Engineer
1 day ago
Sheetz Pittsburgh, United StatesOVERVIEW · Responsible for developing complex, large scale data models and pipelines that · organize and standardize the data to make it readily accessible and consumable by the business for · reporting and data science needs. Collaborate with various areas of the Business in ...
-
Data Engineer
4 days ago
Govini Pittsburgh, United StatesCompany Description · Govini builds software to accelerate the Defense Acquisition Process. Ark, Govini's flagship product, is a suite of AI-enabled applications, powered by integrated government and commercial data, that solves problems across the entire spectrum of Defense Acqu ...
-
Data Engineer
3 weeks ago
Accion Labs Pittsburgh, United StatesThose authorized to work in the United States without sponsorship are encouraged to apply.We are unable to sponsor visas for this position. No H1B's. · Our direct client is an American chain of convenience stores started interviewing Data Engineering Pittsburgh, PA (Hybrid – 2 da ...
-
Data Engineer
3 days ago
Innovation Works, Inc. Pittsburgh, United StatesJob Description · Job DescriptionABOUT IDELIC · Idelic uses cutting-edge technology and beautifully designed interfaces to help predict and prevent trucking accidents and reduce driver turnover to ensure drivers get home safely each and every night. Our SaaS solutions are radical ...
-
Data Engineer
1 week ago
Tekfortune Inc Pittsburgh, United StatesPython(pyspark), jupyter hub, airflow, Java · Database knowledge - Oracle / SQL / Data Analytics · Regulatory reporting knowledge preferred · Good communication skills · Open / Ability to learn new technologies · Very good Analytical Skills · Experience in ETL is preferable · ...
-
Data Engineering Architect
1 week ago
Synechron Pittsburgh, United StatesTitle: Data Engineering Architect (Neo4j and PySpark) · Location: Pittsburgh, PA · Job Type: Full time · About the role: The Data Architect will specifically help to generate and assess the calculated data quality, Risk Tech Analyst also supports Quants and Risk models validation ...
-
Data Engineer II
1 week ago
University of Pittsburgh Pittsburgh, United States· Data Engineer II · Job Summary · Job Title: Data Engineer II · Job Location: Pittsburgh, PA · Job Description: Support data acquisition by locating datasets and software produced by MIDAS researchers through online searching, reviewing the scientific literature, and communi ...
-
Data Engineer Associate
17 hours ago
PNC Pittsburgh, United StatesPosition Overview · At PNC, our people are our greatest differentiator and competitive advantage in the markets we serve. We are all united in delivering the best experience for our customers. We work together each day to foster an inclusive workplace culture where all of our emp ...
-
Senior Data Engineer
1 week ago
EOS USA Pittsburgh, United StatesWHO WE ARE: · EOS IT Solutions is a Global Technology and Logistics company, providing Collaboration and Business IT Support services to some of the world's largest industry leaders, delivering forward-thinking solutions based on multi-domain architecture. Customer satisfaction ...
-
Senior Data Engineer
4 days ago
Carnegie Mellon University Pittsburgh, United StatesReference #: What We Do · The CERT Cyber Mission Readiness (CMR) directorate fortifies the U.S. and its partners against cyber threats. We design large and small-scale exercises, facilitate tabletop scenarios, run cybersecurity contests, and create open-source tools to bolster cy ...
-
Senior Data Engineer
2 days ago
Mindera Pittsburgh, United States· Here at Mindera, we are continuously building a fantastic team and would love for you to join us. · We're looking for data engineers with at least 10 years of experience who have an appetite for solving complex and exciting problems. We develop to a very high standard, mainta ...
-
AI Data Engineer
1 week ago
Synergy Staffing Pittsburgh, United StatesJob Description · Job Description · In this role you will democratize AI through the enterprise ML platform, accelerate the adoption of Generative Artificial Intelligence (Gen AI) and drive Responsible AI. · Youll enjoy the flexibility to work remotely * from anywhere within t ...
-
Manager, Data Engineering
3 weeks ago
Free Market Health Pittsburgh, United StatesJob Description · Job DescriptionSalary: $110,000-$120,000 · Overview · As part of the Technology Services team at Free Market Health, the Manager of Data Engineering will be responsible for helping to guide a team that will implement, grow, and optimize our data intake process, ...
-
Data Engineer II
6 days ago
Net Health Pittsburgh, United StatesAbout Net Health · Belong. Thrive. Make a Difference. · Are you looking for a meaningful and satisfying career where you have endless opportunities to grow and be financially rewarded? Net Health may be the perfect place for you. · A high-growth and profitable company, we help ...
-
Data Analyst/Engineer
2 days ago
Cypress HCM Pittsburgh, United StatesJob Description · Job Description · Senior Data Engineer / AnalystWe are looking for a Senior Data Engineer to join our passionate team, adding your background and experience making us even stronger. In this role, you will build datasets and make them accessible to our partner te ...
-
Data Engineer Senior
4 days ago
PNC Pittsburgh, United StatesPosition Overview · At PNC, our people are our greatest differentiator and competitive advantage in the markets we serve. We are all united in delivering the best experience for our customers. We work together each day to foster an inclusive workplace culture where all of our em ...
-
Data Engineering Manager
1 day ago
Pantherx Specialty LLC Pittsburgh, United StatesJob Description · Job Description7,000 Diseases - 500 Treatments - 1 Rare Pharmacy · PANTHERx is the nation's largest rare disease pharmacy, and we put the patient experience at the top of everything that we do. · If you are looking for a career in the healthcare field that embr ...
-
Specialist - Data Engineer
3 weeks ago
Eaton Pittsburgh, United StatesEaton's IT team is currently seeking a Specialist – Data Engineer to join team. This position will be based at our Beachwood, Ohio, Moon TWP, PA, Houston, TX, Galesburg, MI, Raleigh, NC or Menomonee Falls, WI. The position is eligible for hybrid work setup. · The expected annual ...
Data Engineer - Pittsburgh, United States - InfoGravity LLC
Description
Data Engineer - ETL with Python, Spark & Hadoop
Location: Pittsburgh only (Hybrid 2-3 days a week).
Citizens, GC, or L2S
Ensure you read the information regarding this opportunity thoroughly before making an application.
Note: This is a Hyrbid role. The Hiring Manager requires the individual to be onsite for 2-3 days a week. And they must be willing to go fulltime after a year, salary can be negotiated with hiring manager while converting to their fulltime from our payroll. There are no sponsorships available for this role.
Responsibilities:
Qualifications: