Reading and Writing Parquet nested datatype file using Pyspark

Job ID: 31013380

Budget: $2 – $8 USD

write a pyspark job which should read a parquet file which has nested datatypes & values( records) and change the one of the column value with xxx and write into a new parquet file.

so the actual source file and newly created file should be same with the small change ( changed to xxx for of the row value for one column), the new parquet file should same as the original file (schema, no of records, order of records) with the changed one of the column value 'xxx'

Note:- logic should be dynamic ,parquet file schema will not be the same all the time.....our code should read the parquet file schema dynamically and and create the parquet file with changed data ( xxx) ....the rows, schema and columns should be same
Related categories: PySpark