Experience of Big Data developer with around 3 years on project in Judicial services industries performing ETL operations and primary focus on Data Cleansing, Data Profiling, Data ingestion and Data Loading.
Experience in all phases of Software Development Life Cycle. Direct interaction with business community, collecting functional requirements , Analysis , Design , Development , Implementation , Enhancement , Maintenance and Production support .
Experience working in ETL project and ETL tools technologies like Spark,HIVE and Hadoop . Developing Python coded Spark projects
Developing Spark applications using Python.
Experience in processing text, Delimited, CSV and Complex XML file in Hadoop using Spark
Using Spark for streaming data processing
Good knowledge in HIVE Warehouse and Internal/External tables, Partitioning, Bucketing, Joins Optimization of HIVE Warehouse
Experience in writing SQL Queries, Dynamic Queries, Join for generating Stored Procedure, Triggers and Functions
Sound knowledge of Machine Learning & Python.
Good understanding of Data Warehousing concepts (SCD's, Dimension & Fact Tables).
Good understanding of SQl.
Hands-on experience on DevOps tools like Jupyter, Hortonworks sandbox HDP, Wharf for QA and production deployments.
Direct Interaction with business community, collecting/creating functional requirements and converting them to technical specifications.
Good multi-tasking skills with flexible timings and good communication skills.