
sreeja p
Technology / Internet
About sreeja p:
I've been working in big data ecosystems like Spark, Hive, Athena, Python, Pyspark, Redshift, AWS kinases, and DynamoDB. I’m currently working in architecture where we ingest data both in batch and real-time. for batch, we ingest data from Teradata and SQL Server land data in S3, write Pyspark ETL scripts to read data from S3, and do all the computations in Spark and EMR, write data back into Hive and Athena, and I also export the latest snapshot data into Redshift for analytical purposes, where we have visualization developers that build Power bi / Tableau dashboards on top of that, for real-time I write Spark streaming applications where we ingest data from AWS kinases and write data into DynamoDB for API consumption. our downstream consumers are stakeholders, data scientists, and other engineering teams as well. We use airflow DAGs to schedule all the workflows we develop in Python, and that's pretty much about my experience in my current project as well.
Experience
I've been working in big data ecosystems like Spark, Hive, Athena, Python, Pyspark, Redshift, AWS kinases, and DynamoDB. I’m currently working in architecture where we ingest data both in batch and real-time. for batch, we ingest data from Teradata and SQL Server land data in S3, write Pyspark ETL scripts to read data from S3, and do all the computations in Spark and EMR, write data back into Hive and Athena, and I also export the latest snapshot data into Redshift for analytical purposes, where we have visualization developers that build Power bi / Tableau dashboards on top of that, for real-time I write Spark streaming applications where we ingest data from AWS kinases and write data into DynamoDB for API consumption. our downstream consumers are stakeholders, data scientists, and other engineering teams as well. We use airflow DAGs to schedule all the workflows we develop in Python, and that's pretty much about my experience in my current project as well.
Education
masters in Data Science
bachelor's in Computer Science and engineering
Professionals in the same Technology / Internet sector as sreeja p
Professionals from different sectors near Baltimore, Baltimore
Other users who are called sreeja
Jobs near Baltimore, Baltimore
-
A mission-driven organization supporting a vital medical professional community is seeking a Data Engineer to join its Data Governance and Analytics team. · ...
Silver Spring1 month ago
-
The Data Engineer is responsible for developing innovative and complex data solutions and applications to support actuarial analytics specializing in medical economics. · ...
Baltimore, MD1 month ago
-
Job summary: · Data Engineer · <ul<li>Security Clearance-Current active TS/SCI eligible</li></ul<p><br><br> ...
Columbia3 weeks ago