Sr. (Lead) Data Engineer I (II) - Norristown, United States - PJM Inc

Description

Flexible Work Arrangement

• HybridThe Sr.

(Lead)

Data Engineer is responsible for designing and implementing methods to improve data:
reliability, security and quality across the enterprise. They combine raw information from different sources to create consistent and machine-readable formats.

They also develop, test and build algorithms, architectures and infrastructure that enable data extraction and transformation for various modeling needs for example predictive or prescriptive modeling.

Other functions can include organizing large datasets.


Essential Functions:

Analyze and organize raw data, cataloging fields and entitiesBuilding data architectures, and data pipelines to collectively bring together dataAnalyzing, planning, and defining data architecture framework(s), including security, reference data, metadata, and master dataParticipation in analyzing business needs and objectives with analysts, data Architects and engineers, developers, report writers and report consumersInform and support the organizations data strategyConduct complex data analysis and collaboration with analysts, developers andPrepare data for prescriptive and predictive modelingBuild algorithms and prototypes to support data integration and other data related functionsIdentify opportunities for data acquisition and determine data update velocityCollaborating with other teams within the organization to devise and implement data strategies, build models, and assess customers needs and goalsWork with Data Architects in developing and supporting patterns that data engineers, developers and others can followSupport and inform data governance practicesCreating and implementing data management processes and procedureCharacteristics & Qualifications:Required:BS degree in Computer Engineering, Information Systems, Statistics, Informatics or at least 5 years of experience Data Engineer or Architect roleAt least 5 years of experience Strong knowledge of SQLExperience with data pipeline and workflow management toolsExperience in Azure data engineering and cloud data processing servicesKnowledge of Programming e.g.

SAS, Python, PySpark, R, Oracle PL/ SQLExperience in one or more of the following:
C, C++, Java, Python, Go, Perl or Ruby.
Experience with algorithms, data structures, complexity analysis and software design.

Experience with managing database infrastructure (particular emphasis on Oracle and Sql Server)Experience with configuration management tools like Puppet, ChefExperience with Unix, AIX, Debian, Windows operating systems, containers, container orchestratorsExperience with git based code repositories.

Experience using and administering JEE application servers including JBoss, WebSphere, or WebLogicPreferred:
MS degree in Computer Science, Information Systems, Statistics or InformaticsAt least 5 years of work experienceExperience with SAS development tools and datasets (e.g.

SAS Grid, SAS Display manager, SAS Studio )Hands on experience in Data Warehousing and ETL toolsExperience in Big Data technologies (e.g.

Spark, Databricks,) Experience building and optimizing 'big data' data pipelines, architectures, and data setsPerformance/security-oriented data design, including partitions, different types of indexes, views, and most effective data management practices.
Experience with Azure Synapse Analytics, Azure DevOpsExperience with Purview, PowerBI, Alteryx, Qlik, or Tableau Experience with NoSQLExperience with PJM operations, markets, and planning functions#J-18808-Ljbffr