Pyspark job - 1000 gz files with 1m json objs of 6KB each to 1B files in s3

Hi,

I need help to execute a pyspark job. I need to process them. I have 1000 gz files with 1m json objs of 6KB each which we need to convert to 1B json files and save in s3