Ram Tiwari

• Creative analytical person and self-motivated Data Operations Engineer with a strong background in Big Data Environments including, Infrastructure Optimization, Linux Administration, Automation techniques, Network Management, Security Management, and Incident management.
• Subject Matter Expert (SME) in Capacity Planning, Platform Operating, Performance Tuning, Monitoring, Troubleshooting, Deployment, and management of Cloudera (CDP, CDH), Hortonworks (HDP), EMR, and Databricks distributions.
• Expert on the deployment of Big Data tools and technologies in on-premise and Cloud environments like: Amazon Web Services (AWS), Microsoft Azure, and Comcast Cloud Platform (CCP).
• Hands-on experience in managing, configuring, upgrading, and maintaining Hadoop services like: Cloudera Manager, YARN, HDFS, Tez, Spark, Hive, Sqoop, Oozie, HBase, Zookeeper, Python, Nifi, PL/SQL.
• Experience in Data migration, Disaster recovery planning, and risk assessment.
• Experience in enabling high availability of NameNode, Resource Manager, KeyTrustee, HBase, and Hive.
• Experience in integrating Tableau, Control-M, Teradata, and Qlik Replication with CDP, as well as structured Spark Streaming and Kafka Streaming with PySpark and Scala.
• Experience in automating tasks using bash and Ansible for platform reliability and performance optimization.
• Experience in creating and managing AWS tools such as VPC, Subnets, security groups, S3 bucket, EC2, EMR, and IAM Policy.
• Experience in EMR administration and in designing highly available, scalable, cost-effective, and fault-tolerant systems using multiple AWS services.
• Created VPCs and subnets, including setting up routing tables and adding an internet gateway to enable internet access for resources in the private subnet.
• Configured a public subnet within a VPC to allow direct access to the internet for specific resources.
• Worked with peering connections (PCX) to establish secure communication and connectivity between different VPCs.
• Integrated Salesforce with Amazon S3 buckets using Appflow, enabling seamless data transfer and synchronization between the two platforms.
• Competent in setting up Continuous Integration and Continuous Deployment (CICD) pipelines using AWS Codestar, streamlining the development, testing, and deployment processes to achieve efficient and reliable software delivery.
• Experience in securing clusters with Data Encryption, Kerberos, AD/LDAP, TLS, Ranger, Hardware Security Module (HSM) and Knox.
• Experience in managing source control repositories such as GIT and hands-on experience in working with Sqoop import/export along with performance optimization.
• Experience in the backup of files/folders along with commissioning and decommissioning of nodes.
• I am highly skilled in Data Engineering, Data analysis, Data Modeling, Machine Learning Engineering, and Data visualization.
• Excellent leader and team player, detail-oriented, and proven ability to meet deadlines of diverse projects by working in a fast-paced working environment.
• Dedicated to working in various environments enjoying new challenges and technologies evidenced by 8+ awards
• Experience in all stages of Agile/SCRUM/JIRA/Confluence methodology.

Senior Big Data Administrator at Automobile Club of Southern California, Coppell, Texas (March 2022 - Present)

• Patch Servers monthly to protect data from security vulnerabilities.
• Upgrade the Big Data Platform along with its components periodically to fix bugs in the cluster.
• Remediate Cluster vulnerability of Log4j and Spring Framework.
• Secure Cluster by implementing LDAP, authentication, Multi-Factor Authentication (MFA), and Kerberos authentication techniques for user access control.
• Implement Ranger for access authorization to keep Data secured along with Personal Identity Information (PI).
• Apply Quest tool and Realm for authorization of the users access.
• Set up alerts for Cloudera servers and services using both Cloudera configuration and shell/bash scripts.
• Implement data encryption at rest to keep data secured in the storage and data encryption at transit (SSL/TLS) to keep data highly secured during the transfer from one endpoint to another endpoint.
• Secure Cluster Network by implementing Firewall and iptables rules.
• Upgrade the Big Data parcels periodically to improve the overall cluster performance by lowering the query failure rates.
• Apply Hot fixes to resolve bugs issues of the Cloudera Cluster.
• Research continuously to utilize the infrastructure optimally and configure the hardware and software parameters based on the findings.
• Manage resources needed for the Big Data Jobs which are run by the Data Engineering and Data Science team.
• Troubleshoot the failed jobs and services of Production and Development environments on a daily basis as an Operation Engineer.
• Performs root cause analysis on failed jobs and components to implement corrective measures, and develop automation using scripting languages such as Python, Ansible, Shell, PowerShell, and Bash scripts.
• Recommend Hive, Spark, and Impala query optimization techniques to Big Data Developers to utilize the resources in an efficient manner by avoiding the resource contention of the jobs during peak hours.
• Improve the Cluster performance by adjusting the parameters of the OS kernel, Disk 1/0, Network, Memory, Mappers, & JVM heap size.
• Develop tools and utilities to administer Hadoop jobs, manage queues, and monitor Cluster performance.
• Monitor Control-M and Qlik replication jobs and troubleshoot as per the needs.
• Manage Disaster Recovery Plan by implementing High Availability (HA) and by creating databases and Hadoop Distributed Files System (HDFS) replication policies.
• Prepare a Cluster report and present it to management along with recommendations for improvement strategies.
• Set up Alerts for availability of services and hosts to facilitate On-call people for their support. 
• Planned and monitored AWS cloud infrastructure to ensure optimal performance and resource utilization.
• Built and configured production systems in the AWS cloud environment to support the deployment and scaling of applications.
• Managed updates and patches for server-based technologies to keep systems up-to-date and secure.
• Collaborated with architecture teams to design and implement new software technologies to meet client requirements.
• Tested and implemented continuous updates, identifying opportunities for improvement and efficiency.

Lead Big Data Engineer at Comcast, Philadelphia, PA (Employer: Vastika Inc.) July 2019 – Feb 2022

• Deployed Atlas and Ranger for metadata and data governance and access control.
• Implemented secure data platforms with SSL, SSO, and LDAP/S authentication integrated with AD.
• Managed network connections between on-premise and cloud hosts using TSF.
• Benchmarked systems, analyzed bottlenecks in a hybrid environment, and proposed solutions.
• Configured HDFS, Hive cluster replication policy, and snapshot replication policy as disaster recovery plans.
• Deployed Cloudera Data Platforms (CDP), Hortonworks Data Platform (HDP), Elastic MapReduce (EMR), and Databricks in AWS, Microsoft Azure, and Comcast Cloud Platform (CCP).
• Migrated data from HDP to CDP along with use cases and network configurations.
• Upgraded Cloudera and Hortonworks Data Platforms for bug fixes and performance improvement.
• Monitored resource utilization using Ambari, Cloudera Manager, Grafana, and Databricks Dashboards and adjusted capacity.
• Troubleshoot network latency and host performance as needed.
• Configured YARN Queue Manager for capacity scheduling and fair scheduling.
• Developed automation using Ansible, Terraform, and Crontab for log rotation and task scheduling.
• Administered Databricks platform with S3 integration and IAM policy management.
• Read streaming Kafka topics and aggregated data in Spark Cluster using PySpark/Scala.
• Imported and exported data using Sqoop with performance optimization techniques.
• Analyzed data from Hive, Spark, and HBase using SparkSQL, SparkR, and PySpark.
• Experience with deploying and maintaining Hadoop clusters, adding/removing nodes, configuring NameNode, Resource Manager, HBase, Hive high availability (HA), and monitoring Hadoop jobs and log files.
• Involved in the day-to-day running of Hadoop clusters and worked with cross-functional teams.
• Strong UNIX/LINUX and configuration management skills.
• Good understanding of OS, process management, resource scheduling, networking, CPU, memory, and storage.

Data Analyst at The University of Tulsa, OK     August 2014 - July 2019

• Worked on data cleansing and data integration to make it ready for data analysis.
• Managed databases for both structured and unstructured data.
• Visualized data using tools such as Tableau, Python, and R.
• Installed and configured MySQL, PostgreSQL, and Hadoop for big data analysis.
• Assisted undergraduate students in their data manipulation, data analysis, and data visualization.
• Analyzed optical, electronic, and structural properties of Quantum Dots.
• Involved in technical writing - research report, manuscript, and external grant.
• Presented research findings at colloquia and conferences.

Software Engineer at Bikalpa Software, Kathmandu, Nepal August 2011 – June 2014

• Installed MySQL database and used SQL queries for statistical analysis of data.
• Installed Microsoft Visual Studio Code, Python, and MATLAB for data manipulation, data integration, and data visualization.
• Installed and configured Linux operating systems along with network management.
• Developed and tested bash and python scripts for scheduling queries on numerous tables.
• Implemented automated jobs using bash scripts with crontabs.
• Worked on the backup strategy of configurations and important data.

Masters in Electrical and Computer Engineering                                                  Aug 2017 – Dec 2018

The University of Tulsa, Tulsa, Oklahoma, USA