Convert pandas code to pyspark
Budget: $30 – $250 USD
I am looking for a freelancer who can convert my pandas code to pyspark. The dataset is small, less than 1 GB in size. I don't have specific transformations or operations in mind, but I am open to suggestions. It is important that the pyspark code is optimized for performance.
Ideal skills and experience:
- Strong knowledge and experience in both pandas and pyspark
- Ability to understand and convert pandas code to pyspark
- Familiarity with optimizing pyspark code for performance
The output should be same here in python with pandas and the code with pyspark. Please Add the print statements to verify.
Versions
-----------------
spark - 2.4.7.7
Anaconda3-2018
Ideal skills and experience:
- Strong knowledge and experience in both pandas and pyspark
- Ability to understand and convert pandas code to pyspark
- Familiarity with optimizing pyspark code for performance
The output should be same here in python with pandas and the code with pyspark. Please Add the print statements to verify.
Versions
-----------------
spark - 2.4.7.7
Anaconda3-2018