Rahul Shakya

Data engineer with a strong passion for solving complex business problems using cutting-edge technologies. I have extensive experience in building scalable and reliable data solutions that deliver high-quality data insights and drive business growth. Always eager to work with innovative and collaborative data professionals who share my vision and enthusiasm for data engineering

Research and Development Data Engineer | Tejas Networks India Decreased data processing error rates by 25% by orchestrating data workflows using Airflow, automating end-to-end processing reducing manual intervention Performed deep packet inspection on Wireshark to analyze network anomalies using quantitative analysis and diagnosed network bottlenecks to optimize data flow, decreasing packet drop rate by 15% and enhancing overall system performance by 10% Collaborated with Devops and engineers to design and implement solutions, such as data lakes, and data warehouses, meeting business requirements, and specifications; Conducted knowledge transfer sessions for networking concepts Developed efficient data warehouse design defining snowflake schema design based on an understanding of data sources and end-user reporting requirements

January 2020 - March 2021

Data Engineer II | Elevance Health (Anthem Inc.) Implemented an automated event processing data pipeline (Lambda, Glue, EventBridge, EMR, PySpark, Redshift), combining healthcare records (claims & EHR) from 12 sources, that reduced duplicate claims (15%), sped up processing (65%), and improved business decisions via comprehensive member journeys. Developed a federated data model on AWS Redshift and RDS MySQL, leveraging PySpark to transform 250M Medicare/Medicaid records from 28 US states improving customer service response times by 33% and enabling better fraud detection through anomaly models. Designed and deployed data solutions on the cloud by leveraging AWS architectural best practices, by employing Terraform modules to enhance the efficiency and reliability of data ingestion, processing, and storage systems Analyzed SQL query execution plans to identify bottlenecks and utilized indexing strategies (clustered, non-clustered) to enhance query speed and rewrite suboptimal queries achieving a 20% reduction in query execution time for faster data retrieval. Built a real-time ETL data streaming service using Kafka to enable faster and more reliable data ingestion for a healthcare analytical project, reducing latency by 35% and expediting the processing of sensitive PII and PHI data; encrypted data using AWS KMS and tokenization using Protegrity Leveraged AWS CLI to build automated bash scripts to build statistics for the project’s cloud resource utilization enabling quick cost estimation

June 2022 - October 2023

Data Engineer | Kasjan Techserve LLP Lucknow India Prepared an efficient data model for freight and transport using dimensional modeling and implemented it with ETL using SSIS Conducted extensive data cleaning removing inconsistencies and transforming unstructured data to actionable data for over 300,000 records

April 2019 - December 2019

Data Engineer | Radiance Technologies LLC Built a data lineage framework leveraging CDC and Airflow to document the real-time state change information in AWS RDS, ensuring transparency and compliance across the organization and automated the process of report generation to build insights Built failure recovery mechanisms by writing Python scripts making the framework robust and less prone to errors decreasing RTO by 30%

February 2024 - Present

Masters in Information Systems

Bachelors in Computer Science and Engineering