Reading and Writing Parquet nested datatype file using Pyspark

$2-8 USD / hour

Lukket

Slået op

næsten 3 år siden

$2-8 USD / hour

write a pyspark job which should read a parquet file which has nested datatypes & values( records) and change the one of the column value with xxx and write into a new parquet file. so the actual source file and newly created file should be same with the small change ( changed to xxx for of the row value for one column), the new parquet file should same as the original file (schema, no of records, order of records) with the changed one of the column value 'xxx' Note:- logic should be dynamic ,parquet file schema will not be the same all the time.....our code should read the parquet file schema dynamically and and create the parquet file with changed data ( xxx) ....the rows, schema and columns should be same

PySpark

Projekt-ID: 31013380

Om projektet

3 forslag

Projekt på afstand

Aktiv 3 år siden

Leder du efter muligheder for at tjene penge?

Emailadresse

Fordele ved budafgivning på Freelancer

Fastsæt dit budget og din tidsramme

Bliv betalt for dit arbejde

Oprids dit forslag

Det er gratis at skrive sig op og byde på jobs

3 freelancere byder i gennemsnit $7 USD/time på dette job

@MohammedSulhi

I read your project description carefully. I am bidding on your project because I am very much familiar with Python, Pyspark and Parquet. I am an experienced Data Scientist and Machine Learning Engineer. Data Visualization, NLP, Deep learning, Artificial intelligence, machine learning, Data structures, and algorithms are my major fields. I finished specializations on Data Science, Machine learning, Deep neural Network, Convolution NN, Recurrent NN, Tuning Hyper Parameter .this project will well fit for me. I have won the 2nd runners up award in Sri Lanka Biggest Data Science Competition. I am very fluent with python and did a lot of data science and ml project. So I am familiar with these related libraries such as matplotlib, seaborn, pandas, numpy, sikit-learn, Keras, TensorFlow, spark etc . I am an expert in R language and did lot of projects to data visualization , data manipulation and supervised and unsupervised learning.

$5 USD på 40 dage

0,0

(0 anmeldelser)

2,0

@jainabhay1796

I will work on this , if I am given opportunity. So basically I would like to discuss few things before I am alloted to this

$8 USD på 40 dage

0,0

(0 anmeldelser)

0,0

@jsljaisingh

I do have 6 years of IT experience with Python, pyspark, Spark, SQL, ETL, data analysis and data engineering. I do have exposure to Azure and Aws cloud platforms. Though, i am quite new to this platform but i can assure you i do have a rich experience in robust and scalable data pipelines using pyspark. i have handled static and dynamic schema in feeds. We can have a 20 mins session to understand your needs. Let's connect to discuss more about same.

$7 USD på 20 dage