- Design and implement distributed data processing pipelines using Spark, Hive, Python, Airflow, and other tools and languages prevalent in the Hadoop ecosystem. You will be given the opportunity to own the design and implementation. You will collaborate with Product managers, Data Scientists, Engineering folks to accomplish your tasks.
- Design, Develop, Modernize/Migrate pipelines to Databricks
- Develop applications on Cloud preferably AWS
- Explore and build proof of concepts using open source NOSQL technologies such as HBase, DynamoDB, Cassandra and Distributed Stream Processing frameworks like Apache Spark, Kafka stream.
- Take part in DevOps by building utilities, user defined functions and frameworks to better enable data flow patterns.
- Work with architecture/engineering leads and other teammates to ensure high quality solutions through code reviews, engineering best practices documentation.
- Experience in Business Rule management systems like Drools will also come in handy.
- MS/BS degree in a computer science or related discipline
- 5+ years experience in large-scale software development/Big Data technologies
- Programming skills in Java/Scala, Python, Shell scripting, and SQL
- Development skills around Hadoop, Spark, PySpark, Hive
- Expert knowledge and experience in Databricks, Lake House, Structured Streaming, Kafka, Delta Lake, Delta Live table, Delta shareetc.
Big Data Engineer - Washington, United States - US Tech Solutions
Description
Big Data Engineer
Beaverton OR Remote for some time - candidate open to relocating to be onsite in the future
2+ Months Contract (Possible Extension)
Job Description:
Client is embracing Big Data technologies to enable data-driven decisions. Were looking to expand our Big Data Engineering team to keep pace. As a Sr. Big Data Engineer, you will work with a variety of talented teammates and be a driving force in technical initiatives that will accelerate analytics at Client. You will be working on projects that build data artifacts to answer questions about consumer behavior, commerce trends, consumer touchpoint preferences and more
Here are some tasks that you could do day to day.
Some combination of these qualifications and technical skills will position you well for this role:
About US Tech Solutions:
US Tech Solutions is a global staff augmentation firm providing a wide range of talent on-demand and total workforce solutions. To know more about US Tech Solutions, please visit
Recruiter Details:
Adil Saifi - Team Lead (IT Recruitment)
Telephone: Ext: 7222 | Direct:
Fax:
Email: