Rapid Pyspark Data Analysis in Python

I urgently need a Python program using Pyspark to assist in the comprehensive analysis of large datasets.
The input data I will have is dataframe.
I either need pyspark program or convert the dataframe into rdd and achieve the ouput in python.

df = spark.createDataFrame([
("C0032145", "Case00044556", "2023-07-20 06:30:32.428 -500", "Forward", "2023-07-20 06:30:32.428 -500"),
("C0032145", "Case00044556", None, "Reverse", "2023-09-12 11:05:41.354 -500"),
("C0032145", "Case00044556", "2023-10-18 10:06:02.982 -500", "Forward", "2023-10-18 10:06:02.982 -500"),
("C0032146", "Case00044556", "2023-08-29 06:50:30.068 -500", "Forward", "2023-08-29 06:50:30.068 -500"),
("C0032146", "Case00044556", None, "Reverse", "2023-09-19 11:39:48.354 -500")
], ["claim_identifier", "case_identifier", "claim_opened_timestamp_utc", "forward_reverse_indicator", "event_occurred_timestamp_utc"])

# Convert the columns to timestamp
df = df.withColumn("event_occurred_timestamp_utc", to_timestamp(col('event_occurred_timestamp_utc')))
df = df.withColumn("claim_opened_timestamp_utc", to_timestamp(col('claim_opened_timestamp_utc')))

I will pass load_date to process the data along with the dataframe,
date = to_timestamp(lit(date), 'yyyy-MM-dd')

I am using spark version, spark-3.3.0-bin-hadoop3\python\lib\pyspark.zip\pyspark & spark-3.3.0-bin-hadoop3\python\lib\py4j-0.10.9.5-src.zip\py4j

The code should execute all the scenarios given in the excel and I gave explanation word document for one scenario.

In excel, beside scenarios, I gave the output that I am expecting to my after processing the data.