Sr. Data Engineer - Hyderabad, United States - Randstad Enterprise

Description

The Randstad Offshore team is seeking an experienced Data Engineer with a passion for developing and implementing data solutions in Google Cloud Platform. The ideal candidate will have hands-on experience working with tools such as Cloud Composer (Apache Airflow), Cloud Storage, BigQuery, Cloud Functions and have experience with deploying batch and streaming pipelines.

The candidate must be inquisitive, detail oriented, and highly motivated for delivering high quality solutions. They must possess excellent communications skills, strong work ethic, and the ability to thrive in a fast paced, changing, high pressure environment.

Responsibilities:

• Design, develop, and maintain data pipelines, to move and transform data from various sources (REST APIs, Emails, Flat files etc) into a unified data platform on BigQuery.
• Build and maintain scalable data architecture and infrastructure that can handle large volumes of data with high velocity.
• Implement reusable components around Data Quality and Data Standardization using BigQuery
• Adheres to and advocates for software & data engineering best practices (eg technical design and review, unit testing, monitoring, alerting, source control, code review & documentation)
• Deploys secure and well-tested software that meets privacy and compliance requirements; develops, maintains and improves CI/CD pipeline
• Collaborate with business analysts, data scientists, data analysts, BI teams and business stakeholders to understand the data needs and provide solutions to enable them to access the data they require.
• Monitor and troubleshoot data pipelines to ensure their reliability and performance.
• Mentors and shares knowledge with the team to provide design reviews, discussions and prototypes
• Work seamlessly in an agile development environment to prioritize tasks and deliver results. Qualifications:
  • Bachelor's degree in Computer Science, Computer Engineering, or a related field (BE/BTech)
  • 7+ years of experience within the field of Data Engineering
  • Experience in leading a team of 4 to 5 members
  • At least 3 years of experience working with Google Cloud Platform, BigQuery, Cloud Storage and Cloud Functions
  • At least 2 years of experience working with Cloud Composer (Apache Airflow)
  • At least 3 years of experience in Python
  • Strong experience in SQL and writing complex queries
  • Experience with Infrastructure management tools (Terraform etc)
  • Experience working with tools like Github, Gitlab, JIRA, Confluence and related tools
  • Excellent problem-solving skills, attention to detail and the ability to troubleshoot complex technical issues Tools: Google Cloud Platform, Cloud Composer (Apache Airflow), BigQuery, Python, SQL, Terraform, Cloud Storage