I need to generate test data using spark code in HDFS path. If storing in AWS will be also more useful
We need to give the column names that needs to be created
Number of rows to be generated
Output format can be csv,parquet,txt,json
For the columns created we need to provide the data from another file
3 freelancere byder i gennemsnit ₹1117 timen for dette job
Hi, I am a certified Azure Solution Architect and Data Engineer with Vast experience on on-prem spark and Databricks on Azure. I have 10+ years of experinece working in Data and Analytics using ETL, SQL and Spark.