Jobs
>
Dallas

    Senior Data Engineer - Dallas, United States - Anblicks

    Anblicks
    Anblicks Dallas, United States

    1 day ago

    Anblicks background
    Description

    Job Title:
    Senior Data Engineer

    JOB DUTIES:


    Participate in daily agile and scrum processes to understand changing business requirements, including examining system configurations and operating procedures, as well as functional requirements gathering.

    Ingest and prepare business-ready data in the cloud using Azure Data Factory (ADF) to build ELT(Extract Load and Transform)/ETL (Extract Transform and Load) data pipelines, then move the data into a data warehouse (Dedicated SQL Pool) and create data lake zones for data analytics and visualization.

    Work with a combination of Azure Data Factory and Azure Databricks, extract, load, and transform data from cloud sources and on-premises databases such as Oracle, SAP, and SQL Server to Data Lake Storage and Azure Synapse Analytics.

    Create an Azure Data Factory Pipeline Template to migrate an on-premises data platform to the Azure Cloud using batch processing methods by incremental or full load, and an ADF config driven framework to pull data from multiple sources with different table structures with less manual work and less resource usage.

    Analyze, design, and build modern data solutions using Azure PaaS services to support data visualization. Understand the current production state of the application and the impact of new implementation on existing business processes. Enable private end point, firewall setting and Azure Key Vault for robust data security.

    Analyze the existing SSIS packages and integrate it with Azure Data Factory, using SSIS transformations like Lookup, Derived column, Data conversion, Aggregate, Conditional split, SQL task, Script task, and Send Mail task.

    Create a JSON structure for data storage in Azure Cosmos DB (SQL API), write stored procedures and functions and work with the API team to create Cosmos DB queries that use fewer request units.

    Data Model in Snowflake and ELT using Snowflake SQL, implementing complex stored procedures and best practices with data warehouse and ETL concepts.

    Design and customize dimension data models for data warehouse supporting data using Azure Synapse Analytics, and select the appropriate distribution method in dimension and fact tables to load the data in an optimized manner, as well as implement complex stored procedures and best practices with data warehouse.

    Build a distributed in-memory application using spark applications and perform analytics efficiently on large datasets using python and Spark SQL, also use Spark SQL to implement transformation logic in Databricks and mount/unmount Azure Blob Storage.

    Read the data from different file format parquet, avro, csv and json using pySpark (Python API) in Azure Databricks and perform data extraction, transformation to uncover insights into customer usage patterns and insert curated data into a data warehouse.

    Create data visualization reports and dashboards in Power BI using data from the data warehouse, flat files, and Azure SQL.

    Responsible for fixing problems and conducting investigations into SQL queries, Stored Procedures related to long running jobs and Azure service performance.

    Utilize Azure Monitor and Alert services, create monitors, alerts, and notifications for Data Factory, Synapse Analytics, and Data Lake Storage.

    Perform the required daily GIT support for various projects. Be in charge of maintaining GIT repositories and access control procedures.

    Create CI&CD using azure dev ops pipeline to deploy Azure Services (Storage, Data factory, Key vault & Logic App) using ARM Templates.


    JOB REQUIREMENT:


    Master's degree in Computer Science, Computer Information Systems, or Engineering related or Technical related fields plus 2 years of experience.

    In lieu of the above, we will also accept a Bachelor's degree in Computer Science, Computer Information Systems, or Engineering related or Technical related fields plus 5 years of progressively responsible post-baccalaureate experience.

    Foreign degree equivalent is acceptable. We will also accept any suitable combination of education, training and/or experience.

    Experience to include working on Azure Data Factory (ADF), Oracle, SQL server, Azure Databricks, Azure Synapse Analytics, Data Lake Storage, Azure PaaS services, SSIS packages, Azure Cosmos DB (SQL API), Python, Spark applications, Azure SQL, SQL queries, Azure Monitor and Alert services, GIT Support, ARM Templates.

    HOURS:
    M-F, 8:00 a.m. - 5:00 p.m.

    JOB LOCATION:
    Dallas, Texas.

    Travel is not required, but candidates must be willing to relocate to unanticipated work locations across the country per contract demand.

    CONTACT:
    Email resume referencing job code# SDE ANB to Maruthi Technologies INC. DBA Anblicks at
  • Anblicks

    Data Engineer

    1 week ago

    Anblicks Dallas, United States

    Skills: · • Analyze and understand data sources & APIs · • Design and Develop methods to connect & collect data from different data sources · • Design and Develop methods to filter/cleanse the data · • Design and Develop SQL , Hive queries, APIs to extract data from the store · • ...

  • Anblicks

    Data Engineer

    1 week ago

    Anblicks Dallas, United States

    Role: Data Engineer · Location : Dallas, Texas · Job Type: Hybrid · • Analyze and understand data sources & APIs · • Design and Develop methods to connect & collect data from different data sources · • Design and Develop methods to filter/cleanse the data · • Design and Develop S ...

  • Tiger Analytics

    Data Engineer

    2 weeks ago

    Tiger Analytics Dallas, Texas, United States Permanent

    Tiger Analytics is a fast-growing advanced analytics consulting firm. Our consultants bring deep expertise in Data Science, Machine Learning and AI. We are the trusted analytics partner for several Fortune 100 companies, enabling them to generate business value from data. Our bus ...

  • Ascendion

    Data Engineer

    2 weeks ago

    Ascendion Dallas, United States

    Position Details: · Company: Banking Giant · Job Title: Data Engineer · Duration: 6 to 12 Months (with possible extension) · Location: Hybrid (Dallas, TX) (3 days Onsite, 2 days remote) · Interview Times: ASAP · Roles and Responsibilities : · 8+ years' experience in data integra ...

  • Rightpoint

    Data Engineer

    1 day ago

    Rightpoint Dallas, United States

    Description · Rightpoint, a Genpact company, is the digital consultancy with technology at its core. We design and engineer end-to-end experiences that help our clients succeed at the speed of innovation. Rightpoint serves more than 250 Fortune 1000 companies. · At Rightpoint, ...

  • Zendr

    Data Engineer

    1 day ago

    Zendr Dallas, United States

    One of our top clients, a multinational consulting firm, is looking to add experienced Data Engineers to work within their Financial Services Division. · This opportunity is perfect for experienced data engineers looking for a more client-facing role and to widen their skillset w ...

  • Capgemini

    Data Engineer

    5 days ago

    Capgemini Dallas, United States

    Sogeti is a leading provider of professional technology services, specializing in Application Management, Infrastructure Management and High-Tech Engineering. Sogeti offers cutting-edge solutions around Testing, Business Intelligence, Mobility, Cloud and Security, combining world ...

  • ShiftCode Analytics

    Data Engineer

    1 day ago

    ShiftCode Analytics Dallas, United States

    DATA ENGINEER · 9+ MONTHS CONTRACT · DALLAS, TEXAS · NEED LOCAL TO DALLAS MILES) · HYBRID (2-3 DAYS/WEEK) · CLIENT DOMAIN- MORTGAGE INDUSTRY · This nationwide mortgage industry leader has an immediate 9+ month contract opportunity for a Data Integration ETL Engineer/Develope ...

  • Protagona

    Data Engineer

    1 day ago

    Protagona Dallas, United States

    As a Data Engineer, you will be part of a talented team of engineers responsible for the deployment and configuration of cloud resources to meet individual client business needs in AWS. Client engagements cover a wide variety of business requirements and require our engineers to ...

  • EY

    Data Engineer

    1 day ago

    EY Dallas, United States

    EY focuses on high-ethical standards and integrity among its employees and expects all candidates to demonstrate these qualities. At EY, you'll have the chance to build a career as unique as you are, with the global scale, support, inclusive culture and technology to become the b ...

  • PDS Tech

    Data Engineer

    1 week ago

    PDS Tech Dallas, United States

    Lets go to work . . . needing a Data Engineer in Dallas Texs. You will have: · Design, build, maintain, debug, and improve data pipelines · Assemble large, complex data sets · Create business intelligence dashboards · 2+ years of experience as a Data Engineer in a similar rol ...

  • Bridge Technologies and Solutions

    Data Engineer

    4 days ago

    Bridge Technologies and Solutions Dallas, United States

    Data engineers are responsible for developing, testing, and maintaining data pipelines and data architectures. Data engineers are responsible for developing, testing, and maintaining data pipelines and data architectures. They combine raw information from different sources to cre ...

  • Austin Professional Svcs, Inc.

    IT Data Engineer

    1 week ago

    Austin Professional Svcs, Inc. Dallas, United States

    Austin Industries is one of the nations largest, most diversified U.S.-based construction companies. With more than 7,000 employee-owners, Austin Industries provides nearly every type of civil, commercial and industrial construction services through the expertise of our operating ...

  • Grid Dynamics

    Data Engineer

    5 days ago

    Grid Dynamics Dallas, United States

    Position at Grid Dynamics We are seeking an experienced Data Engineer with an exceptional command of SQL queries and proficiency in handling data from our warehouse system. This role will be instrumental in filtering and processing data using specified parameters and ensuring com ...

  • PDS Tech Commercial, Inc.

    Data Engineer

    1 week ago

    PDS Tech Commercial, Inc. Dallas, United States

    Let's go to work . . . needing a Data Engineer in Dallas Texs. You will have: · Design, build, maintain, debug, and improve data pipelines · - Assemble large, complex data sets · - Create business intelligence dashboards · - 2+ years of experience as a Data Engineer in a simi ...

  • G-Research

    Engineer - Data Engineer

    3 days ago

    G-Research Dallas, TX, United States

    Do you want to tackle the biggest questions in finance with near infinite compute power at your fingertips? · G-Research is a leading quantitative research and technology firm, with offices in London and Dallas. We are proud to employ some of the best people in their field and to ...

  • eTek IT Services, Inc.

    Data Engineer

    1 day ago

    eTek IT Services, Inc. Dallas, United States

    Job Description · Job DescriptionJob Description · • Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases. · • Extensive Experience on BigQuery, DataProc and DataFlow plat ...

  • Diverse Lynx

    Data Engineer

    1 day ago

    Diverse Lynx Dallas, United States

    Job title: Data Engineer · Location: TCS - Dallas, TX | AUSTIN | Austin, TX · Competencies: Digital : BigData and Hadoop Ecosystems · Essential Skills: · At least 4 to 6 years in a Site Reliability Engineering, DevOps, or Infrastructure focused roleExperience coding in Python, ...

  • Tiger Analytics

    Data Engineer

    4 days ago

    Tiger Analytics Dallas, United States

    Job Description · Job DescriptionTiger Analytics is a fast-growing advanced analytics consulting firm. Our consultants bring deep expertise in Data Science, Machine Learning and AI. We are the trusted analytics partner for several Fortune 100 companies, enabling them to generate ...

  • Sogeti

    Data Engineer

    6 days ago

    Sogeti Dallas, United States

    Sogeti is a leading provider of professional technology services, specializing in Application Management, Infrastructure Management and High-Tech Engineering. Sogeti offers cutting-edge solutions around Testing, Business Intelligence, Mobility, Cloud and Security, combining world ...