ETL PySpark Lead Developer - Philadelphia, United States - Tata Consultancy Services

Description

Job Title Spark Developer Lead (on-site in Philadelphia, PA)

Relevant Experience

(in Yrs years

Technical/Functional SkillsScala Spark, Python, Big Data, Kafka, AWS, Oracle SQL, Hive SQL, Databricks, and Jenkins,

Experience Required Years

Roles & Responsibilities 1. Speak to business and intake team to define the requirements.

• Coordinate with Solution and Business Analysts and create the tech designs
• Hands-on architecture/development of ETL pipelines using our internal framework written in Java
• Interpret data, analyze results using statistical techniques and provide ongoing reports
• Develop and implement databases, data collection systems, data analytics and other strategies that optimize statistical efficiency and quality
• Acquire data from primary or secondary data sources and maintain databases/data systems
• Identify, analyze, and interpret trends or patterns in complex data sets
• Filter and clean data by reviewing reports and performance indicators to locate and correct problems
• Work with management to prioritize business and information needs

Generic Managerial Skills Motivated team player with excellent communication, interpersonal, analytical and problem solving skills.