Jobs
>
Crockett

    Sr. Data Engineer - Crockett, United States - Recru

    Recru
    Recru Crockett, United States

    4 weeks ago

    Default job background
    Description


    We're looking for a Senior AWS Data Engineer skilled in Python open to working hybrid in Houston This role is contract-to-hire.

    The information below covers the role requirements, expected candidate experience, and accompanying qualifications.

    PurposeEstablish data feeds to the digital platform and manage ETL processes. Design raw file data structures to be used in data science applications. Associated commercial data includes data required for forecasting, analysis, optimization modeling and data visualization.
    Work with Data Analysts to ensure enterprise data is cataloged in accordance with data governance standards.
    Develop system test and integration plans, execute test procedures, support user community, and obtain approvals for change management.
    Provide hands-on technical configuration, application development, integration and testing. Implement cloud data integration strategies between cloud providers.

    Provide technical expertise to the business and internal IT with respect to digital solutions, including data visualization and facilitate business expansion with solutions to scale, managing the critical input and output data.

    Design, develop, and maintain data solutions that leverage the power and flexibility of AWSEssential Job FunctionsDeveloping, testing, and deploying data pipelines, data lakes, data warehouses, and data marts using AWS services such as S3, Glue, Athena, Redshift, EMR, Kinesis, Lambda, and more.

    Implementing data ingestion, processing, transformation, and delivery using various methods such as batch, streaming, and real-time.

    Applying best practices for data quality, security, reliability, and performance using AWS services such as Lake Formation, IAM, KMS, CloudTrail, CloudFormation, and more.

    Collaborating with data architects, data analysts, data scientists, citizen data scientists, and developers to understand data requirements and provide data solutions that meet business needs and follow the AWS Well-Architected Framework.

    Evaluating and adopting new AWS technologies and features to improve our data capabilities and optimize our data costs.
    Troubleshooting and resolving data issues and ensuring data availability and integrity.
    Documenting and maintaining data solutions and data standards.
    Using Dataiku DSS for doing pipeline workflows and supporting citizen data scientist.

    Designing and building web applications, REST APIs, and GraphQL APIs using AWS services such as API Gateway, Lambda, DynamoDB, Flask, and more.

    Applying software engineering rigor and best practices to machine learning, including AI/MLOps, CI/CD, automation, etc.
    Facilitating the development and deployment of proof-of-concept machine learning systems using AWS services such as SageMaker, Dataiku, and more.

    Developing and deploying scalable tools and services for our clients to handle machine learning training and inference using AWS services such as SageMaker, ECR, EKS, ECS, and more.

    Architect digital data solutions for GEM US for all Commercial Operations (Trading & Assets).Implement ETL for required enterprise data.
    Work with other IT operations support resources in a collaborative manner to achieve cross-functional goals.
    Identify deficiencies and recommend solutions to management and the user community.
    Provide automation where applicable to minimize business impact and ongoing IT support.
    Continual performance assessment of processes and applications.
    Oversee and mentor key support resources, training them to provide primary and backup system operations and maintenance functions.
    Provide automation of asset related data imports where needed and provide complex reporting solutions.
    Effectively communicate with users, peers and management.
    Leverage people, processes and technologies from GEM Europe and ENGIE DIGITAL teams.

    Work closely with the new Data team as it is setup in GEM US and monitor critical work in Collibra.

    Assist in release management planning and execution.

    Work with vendors such as Dataiku and Databricks to ensure reliable and secure operations on AWS (Common Data Hub) and other cloud solution providers.

    Provide input to program managers of large projects and present issues for team resolution when appropriate.
    Coordinate with other application teams for data integration, such as ETRM systems, ISO Market Systems, SCADA and data warehouses.
    Develop and implement data integration and ETL processes, including data quality checks, error handling, and performance optimization.

    Work with data scientists, business analysts, and other stakeholders to understand data requirements and design data solutions that meet their needs.

    Develop and maintain data models, metadata, data lineage documentation, and associated project documentation.
    Troubleshoot and resolve data pipeline and processing issues in a timely manner.
    Participate in code reviews, design discussions, and agile ceremonies.
    Keep up-to-date with the latest AWS technologies, data engineering best practices, and industry trends.
    Assist in training other IT team members and business users where needed, including development processes on the digital platform.
    RequirementsKnowledge of energy market data.
    In-depth knowledge of digital platforms such as AWS, Dataiku and Databricks.
    In-depth proven software engineering experience using Python and machine learning knowledge.

    Designing and building scalable data architectures on AWS, with strong experience using AWS data storage and processing services such as S3, EMR, Data Lake Formation, AWS Glue, and Amazon Athena.

    Experience in working with DevOps tools and methodologies such as Git, Jenkins, Docker, or Kubernetes.

    Proficiency in programming languages such as Python, Java, Scala, SQL, Spark, and Unix shell scripting, including AWS CLI.Automating data processing workflows using AWS services, AWS CLI, APIs, and scripting languages like Python, Java, and/or Scala.

    Experience with SOAP/RESTful and GraphQL APIs using AWS services and frameworks and familiarity with other cloud services such as Snowflake.

    Knowledge of ETL tools and frameworks such as Apache Spark, Airflow, AWS IoT Analytics, IoT Device Management, and AWS IoT Events.

    Familiarity with database technologies such as SQL, NoSQL, and data warehousing concepts. Working knowledge of Amazon Aurora, DynamoDB, and Amazon RDS.Understanding of data security, privacy, and compliance requirements.
    Experience with data modeling, metadata management, and data lineage tracking.
    Strong analytical, problem-solving, and communication skills.
    Ability to work in a team environment and collaborate with cross-functional teams.
    Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
    Current data visualization tool experience such as Power BI Service and Desktop.
    Prior design and implementation experience with data warehouses.

    Expert knowledge of SQL databases, SQL queries, Amazon Athena, SQL Integration Services, SQL Reporting Services and data formats such as XML and JSON.Knowledge of ISO's and major market activities preferred.

    Education/ExperienceBachelor's degree in a technical field or 15+ software development experience in lieu of degree.10-15years of information technology, systems, and application development experience.

    Minimum 5 years' experience supporting the needs of an Energy Trading or Asset Management is business preferredProficient with Microsoft Office products (Excel, Word, Visio, PowerPoint)Proficient with Windows and Linux operating systems.

    ISO Market systems knowledge preferred.
    Modern cloud technologies used for managing data (AWS is highly preferred).

    Remote working/work at home options are available for this role.
    #J-18808-Ljbffr
  • X4 Technology

    Data Engineer

    3 weeks ago

    X4 Technology Crockett, United States

    About the Company: · For a complete understanding of this opportunity, and what will be required to be a successful applicant, read on. · This is a fantastic opportunity to join an internationally renowned powerhouse within the supply chain industry. You will have the opportuni ...

  • Vaco

    Senior Data Engineer

    3 weeks ago

    Vaco Crockett, United States

    ***Senior Data Engineer*** · All the relevant skills, qualifications and experience that a successful applicant will need are listed in the following description. · ***Onsite in Houston TX-77077*** · Job Summary · The Senior Data Engineer plays a crucial role in architecting, ...

  • Harnham

    Senior Data Engineer

    2 weeks ago

    Harnham Crockett, United States

    SENIOR DATA ENGINEER · HOUSTON, TEXAS - 4 DAYS IN OFFICE · $120,000 - $160,000 BASE + BONUS + BENEFITS · Ready to apply Before you do, make sure to read all the details pertaining to this job in the description below. · Company Overview: · This automotive/retail company is s ...

  • Wipro

    Network Data Engineer

    3 weeks ago

    Wipro Crockett, United States

    Job Title: Network Data Engineer - L3 · Location: Houston TX · Primary Skill (Technical Skill) : Cisco Routing, Switching and Wireless, Infoblox. · Educational Qualifications: B.E. / B. Tech · Experience Level: Years · If you are interested in applying for this job, ple ...

  • Kaizen Technologies

    Data Engineer

    3 weeks ago

    Kaizen Technologies Crockett, United States

    Job Title: Data Engineer (Full time) · Location: Houston, TX (Onsite) USA · Band: U3 · Year of Experience: 5 to 7 Years · Duration: 6 Months (Will be extended for 12 months) · No Of Positions: positions is needed in a month time.) in one Quarter time. · 110K to 130K PA · ...

  • IntagHire

    Data Engineer

    3 weeks ago

    IntagHire Crockett, United States

    Data Engineer with Python, SQL, Slurm, and Kubernetes - Onsite in Houston, TX. Fulltime role with a benefits package · The experience expected from applicants, as well as additional skills and qualifications needed for this job are listed below. · About the Role: · We are seek ...

  • IDR, Inc.

    Data Engineer

    2 weeks ago

    IDR, Inc. Crockett, United States

    IDR is seeking a Data Engineer to join one of our top clients for a Hybrid Position in the Houston, TX area If you are looking for an opportunity to join a large organization and work within an ever-growing team-oriented culture, please apply today · Remember to check your CV be ...

  • IntagHire

    Senior Data Engineer

    3 weeks ago

    IntagHire Crockett, United States

    Senior Data Engineer (Python/SQL) - Onsite in Houston, TX. · The following information aims to provide potential candidates with a better understanding of the requirements for this role. · No C2C please. Permanent position with full benefits. · The Senior Data Engineer (Python ...

  • Recru

    Sr. Data Engineer

    3 weeks ago

    Recru Crockett, United States

    We're looking for a Senior AWS Data Engineer skilled in Python open to working hybrid in Houston This role is contract-to-hire. · The information below covers the role requirements, expected candidate experience, and accompanying qualifications. · Purpose · Establish data feed ...

  • Wipro

    Network Data Engineer

    3 weeks ago

    Wipro Crockett, United States

    Job Title: · Network Data Engineer - L3Location: · Houston TXPrimary Skill (Technical Skill) : · Cisco Routing, Switching and Wireless, Infoblox.Educational Qualifications: · B.E. / B. TechExperience Level: · 7 - 10 Years · If you are interested in applying for this job, pl ...

  • Insight Global

    Remote Lead Data Engineer

    3 weeks ago

    Insight Global Crockett, United States

    Position: Lead Data Engineer (Databricks/AWS) · Duration: 6 Month Contract – w/extensions or conversion · Location: Remote ; East Coast · If the following job requirements and experience match your skills, please ensure you apply promptly. · Must Haves: · 2-3 years exp. as a ...

  • Insight Global

    Remote Lead Data Engineer

    1 week ago

    Insight Global Crockett, United States

    Position: · Lead Data Engineer (Databricks/AWS)Duration: · 6 Month Contract – w/extensions or conversionLocation: Remote ; East Coast · If the following job requirements and experience match your skills, please ensure you apply promptly. · Must Haves:2-3 years exp. as a data ...

  • TekValue IT Solutions

    Oracle Data Engineer with Scripting

    3 weeks ago

    TekValue IT Solutions Crockett, United States

    · • Detailed work experience on Oracle, SingleStore or IBM I DB2 database, SQL & PL/SQL, including ANSI SQL. · The experience expected from applicants, as well as additional skills and qualifications needed for this job are listed below. · • Recent PL/SQL development work expe ...

  • Savills North America

    Client Data Integration and Application Deployment Engineer, Global Occupier Services

    1 week ago

    Savills North America Crockett, United States

    Role Summary:The Client Data Integration and Application Deployment Engineer will play a pivotal role in advancing Savills' strategy for integrating and operationalizing technology to enhance our backend data services and application deployments. Central to this role is the effec ...

  • Mindex Technologies

    Sr. Data Integrations and Reporting Engineer

    3 weeks ago

    Mindex Technologies Crockett, United States

    Founded in 1994 and celebrating 30 years in business, Mindex is a software development company with a rich history of demonstrated software and product development success. We specialize in agile software development, cloud professional services, and creating our own innovative p ...

  • Hirextra -World's First Staffing Aggregator

    Lead Python Data engineer 12+years of experience

    2 weeks ago

    Hirextra -World's First Staffing Aggregator Crockett, United States

    Role : Lead Python Data Engineer— years of experience · Take the next step in your career now, scroll down to read the full role description and make your application. · Location:Houston, TX (Onsite) · Duration : 12 months · Job Description: · We are currently seeking an ex ...

  • Vaco

    Senior Machine Learning Engineer

    3 weeks ago

    Vaco Crockett, United States

    ***Senior Machine Learning Engineer*** · To be considered for an interview, please make sure your application is full in line with the job specs as found below. · ***Onsite in Houston, TX*** · About the role: · As a Senior Machine Learning Engineer, you will work on building ...

  • Capgemini Engineering

    Data Management Lead

    3 weeks ago

    Capgemini Engineering Crockett, United States

    Title: Data Management Lead – SDx/Data and Solutions Delivery · Location: Houston, TX / Hybrid · Like the look of this opportunity Make sure to apply fast, as a high volume of applications is expected Scroll down to read the complete job description. · Role and Responsibilitie ...

  • Insight Global

    Data Analyst

    2 weeks ago

    Insight Global Crockett, United States

    Client: Murphy Oil CorpPosition: Data AnalystDuration: Contract-to-Hire (6-9 month contract) · Must be able to work directly on W2 (No C2C)Rate: $30-$45/hr based on experienceSchedule: 2 days onsite, 3 days WFHJob DescriptionInsight Global is looking for a Data Analyst to join t ...

  • The Select Group

    Data Scientist/GIS

    3 weeks ago

    The Select Group Crockett, United States

    Data Scientist - Hybrid: Houston, TX · The Select Group is looking for a Data Scientist to join the team at one of their utility clients This person will build data products and solutions to complex and advanced data problems faced throughout the organization. They will also par ...