- Analyzes complex data structures from disparate data sources and design large scale data engineering pipeline
- Develops large scale data structures and pipelines to organize, collect and standardize data that helps generate insights and addresses reporting needs .
- Collaborates with product business and data science team to collect user stories and translate into technical specifications
- Uses knowledge in Cloud & Hadoop architecture, HDFS commands and experience designing & optimizing queries to build data pipelines
- Uses strong programming skills in PySpark, Python, Java or any of the major languages to build robust data pipelines and dynamic systems
- Builds highly scalable and extensible data marts and data models to support Data Science and other internal customers on Cloud.
- Integrates data from a variety of sources, assuring that they adhere to data quality and accessibility standards.
- Analyzes current information technology environments to identify and assess critical capabilities and recommend solutions
- Experiments with available tools and advice on new tools to determine optimal solution given the requirements dictated by the model/use case Required Qualifications
- 3+ years of progressively complex related experience in cloud data engineering and data analysis
- Knowledge in programing languages such as PySpark, Java, Python, Hive, SQL
- Knowledge in Cloud Technology, Hadoop architecture, HDFS commands and experience designing & optimizing queries against data in the HDFS environment
- Strong knowledge of large-scale search applications and building high volume data pipelines, preferably using Dataproc, composer services on GCP or other Cloud Platforms Preferred Qualifications
- Ability to leverage multiple tools and programming languages to analyze and manipulate datasets from disparate data sources
- Ability to understand complex systems and solve challenging analytical problems
- Experience with bash shell scripts, UNIX utilities & UNIX Commands
- Pyspark, Java, Python, Hive, SQL
- Hadoop architecture and commands
- Dataproc, Composer services on GCP
-
Lead Software Engineer
10 hours ago
Epsilon Irving, United States Full timeCompany Description · Epsilon is the leader in outcome-based marketing. We enable marketing that's built on proof, not promises. Through , the marketing platform for personalizing consumer journeys with performance transparency, Epsilon helps marketers anticipate, activate and p ...
-
Engineering Team Lead
4 weeks ago
00002 Citibank, N.A. Irving, United States Full timeThe Applications Development Technology Lead Analyst is a senior level position responsible for establishing and implementing new or revised application systems and programs in coordination with the Technology team. The overall objective of this role is to lead applications syste ...
-
Lead Data Engineer
3 weeks ago
Resource Informatics Group Inc Irving, United StatesJob Description · Job DescriptionPlease help us with Lead Data engineers with 12+ yrs of exp into ETL. Ask here is strong Lead who can also do coding, lead the team and take care of Deliverables. · Please help us with Lead Data engineers with 12+ yrs of exp in to ETL. Ask here is ...
-
Lead Software Engineer
3 weeks ago
Resource Informatics Group Irving, United StatesRole: Lead role Software Engineer (Python/Java) · Location: Remote · Duration: 12+ Months · LOOKING FOR INDEPENDENT CONSULTANTS · completely hands on role and individual contributor, candidate should have both Java and Python. · 7+ years of experience required. · DevOps/Sof ...
-
Lead Data Engineer
3 weeks ago
Alcority Irving, United StatesAbout the Role: · The Lead Data Engineer is the senior software engineer in the Business Intelligence and Data Warehousing team. This role develops data ingestion to enable high-value business intelligence products. This is a senior individual contributor role with a focus on han ...
-
Lead Software Engineer
2 days ago
Resource Informatics Group Irving, United StatesJob Title: - Lead Software Engineer · Work Location: St. Louis, MO - 100% remotely · Duration: 6+ Months · Experience: years · PURPOSE OF POSITION: · Lead Software Engineer in Test will play a key role in leading the team through planning and delivering Infrastructure and Se ...
-
Automation Lead Engineer
1 week ago
InterSources Irving, United StatesResponsibilities · Position: Automation Lead Engineer · Location: Irving TX · Duration: Long Term · Required Skills: · qTest · React, , Python · Visualization tool is Grafana · GCP, AWS and other environments · Senior QA Engineer and qTest (nodejs api testing, postman) · QA with ...
-
Lead Software Engineer
3 weeks ago
Caterpillar Financial Service Corp Irving, United StatesCareer Area: · Business Technologies, Digital and Data · Job Description: · Your Work Shapes the World at Caterpillar Inc. · When you join Caterpillar, you'rejoining a global team who cares not just about the work we do – but also about each other. We are the makers, problem sol ...
-
Lead Software Engineer
1 week ago
Caterpillar Irving, United States**Career Area:** · Business Technologies, Digital and Data · **Job Description:** · **Your Work Shapes the World at Caterpillar Inc.** · When you join Caterpillar, you're joining a global team who cares not just about the work we do · but also about each other. · We are the ...
-
Lead Analytics Engineer
3 days ago
Gartner Irving, United StatesGartner's core asset is its people. Our people develop insights, share advice and create experiences that help our clients be more successful. The People Analytics & Operations team derives actionable insight from data to recommend and implement better ways of attracting, develop ...
-
Lead Software Engineer
2 days ago
Gartner Irving, United StatesHiring near our Irving, TX Center of Excellence. · Hybrid, flexible environment. · About Gartner IT: · Join a world-class team of skilled engineers who build creative digital solutions to support our colleagues and clients. We make a broad organizational impact by delivering c ...
-
Lead Software Engineer
2 days ago
Gartner Irving, United StatesHiring near our Irving, TX Center of Excellence.Hybrid, flexible environment.About Gartner IT:Join a world-class team of skilled engineers who build creative digital solutions to support our colleagues and clients. · We make a broad organizational impact by delivering cutting-ed ...
-
Automation Lead Engineer
3 weeks ago
Sidram Technologies LLC Irving, United StatesSenior QA Engineer with WebdriverIO and qTest · QA with and qTest for FinOps project - monitoring and reporting tool developed in React, , Python, SQL. · DB is MS Aurora. · GCP, AWS and other environments. · Visualization tool is Grafana, Looker · Sr QA Engineer will be required ...
-
Lead Network ACI Engineer
2 days ago
Resource Informatics Group Irving, United StatesJob Title: Lead Network ACI Engineer · Location: Cleveland, OH (onsite) · Duration: 6+ months · Experience: 10+ years · It's more of a lead ACI role for the Cisco engineer to help build skillset on the team. · Senior Engineer will be preforming Cisco ACI environment support a ...
-
Sr Lead Fullstack Engineer
3 weeks ago
InfoVision Inc. Irving, United StatesHi · We have an immediate Openings with Our Direct Client for a Long-term contract position · Job Title: Sr Lead Fullstack Engineer (NodeJS) · Location: Irving TX – 3 days office every week · Duration: 12 Months · Required Qualifications: · Bachelor's degree or foreign equivalent ...
-
Software Engineer-Tech Lead
3 weeks ago
7-Eleven Irving, United StatesAbout This Opportunity: · We are looking for a dedicated and experienced Software Engineer Lead to help design, develop, and implement solutions for Order Fulfilment within our Retail Supply Chain capabilities. The individual in this role will work closely with the Value Chain bu ...
-
Lead Information Security Engineer
3 weeks ago
Wells Fargo Irving, United StatesAbout this role: · Wells Fargo is seeking a motivated Lead Information Security Engineer to join an exciting, fast paced team working on cutting edge encryption, tokenization and key management technologies that are leveraged to protect information companywide. This role will pr ...
-
Lead Contact Center Engineer
1 week ago
McKesson Irving, United StatesMcKesson is an impact-driven, Fortune 10 company that touches virtually every aspect of healthcare. We are known for delivering insights, products, and services that make quality care more accessible and affordable. Here, we focus on the health, happiness, and well-being of you a ...
-
Lead Software Engineer, Emerging Tech
2 days ago
Caterpillar Irving, United States**Career Area:** · Business Technologies, Digital and Data · **Job Description:** · **Your Work Shapes the World at Caterpillar Inc.** · When you join Caterpillar, you're joining a global team who cares not just about the work we do but also about each other. We are the maker ...
-
Vice President, Engineering Lead Analyst
2 days ago
Hispanic Technology Executive Council Irving, United StatesBrief Description of the Organization: · Citi, the leading global bank, has approximately 200 million customer accounts and does business in more than 160 countries and jurisdictions. Citi provides consumers, corporations, governments, and institutions with a broad range of finan ...
Sr Data Engineer Lead - Irving, United States - Diverse Lynx
Description
Sr Data Engineer LeadOnsite - Irving, TX
Perform the Data Engineer activities
Experience 10to14Yrs 4.Required Skills , Hive, PySpark, Cloud Dataproc,
Unix Shell Scripting, GCP
Roles & Responsibilities