Spark code to generate test data

I need to generate test data using spark code in HDFS path. If storing in AWS will be also more useful

Requirments :

We need to give the column names that needs to be created

Number of rows to be generated

Output format can be csv,parquet,txt,json

For the columns created we need to provide the data from another file

Evner: Hadoop, Spark

Om klienten:
( 1 bedømmelse ) Chennai, India

Projekt ID: #33736989

3 freelancere byder i gennemsnit ₹1117 timen for dette job


Hai, I am Bigdata engineer and I am having rich experience in data pipelines and data processing on Hadoop,Azure and AWS using pyspark and java I can build a simple script for your requirements and we can make a great Flere

₹1050 INR in 7 dage
(0 bedømmelser)

Hi, I am a certified Azure Solution Architect and Data Engineer with Vast experience on on-prem spark and Databricks on Azure. I have 10+ years of experinece working in Data and Analytics using ETL, SQL and Spark.

₹1250 INR in 7 dage
(0 bedømmelser)

I have 6 years of experience working with Spark, Hadoop, Cloudera, Impala, Hive. I also have experience in Java and Python.

₹1050 INR in 7 dage
(0 bedømmelser)