AI-Powered Essay Grading System

Project Scope

I need assistance in setting up a Github repository on google colab for building an AI-powered automated essay-grading and feedback system for Interdisciplinary Core Curriculum (ICC) courses. The AI/LLM layer is responsible for analyzing essays, grading them against a predefined rubric, and producing structured feedback plus evaluation metrics.

Responsibilities for This Role
• Model Pipeline Setup: Prepare a Google Colab + GitHub repository that supports both training and inference.
• Data Integration: Work with the provided dataset and rubric document to structure training/test inputs.
• Model Development:
• Use GPT-4o-mini (with fallback to smaller models if required).
• Explore fine-tuning and STaR prompting methods along with Hugging Face
• Output JSON-formatted results (scores + dimension feedback).
• Evaluation: Implement Quadratic Weighted Kappa (QWK) scoring to measure reliability against human grading.
• Visualization: (Optional for this role) Generate data outputs suitable for radar chart visualization of scores.

Requirements
• Proficiency in Python.
• Strong experience with Google Colab + GitHub workflows.
• Familiarity with LLMs, fine-tuning, and prompt engineering.
• Knowledge of scikit-learn metrics (esp. QWK).
• Ability to manage dependencies and ensure reproducibility in Colab.

Deliverables
1. Colab-ready repository with scripts for training & inference.
2. Working fine-tuned model(s) that accept essays and return scores + feedback.
3. Evaluation results (QWK and other relevant metrics).
4. Documentation explaining setup, training, and evaluation steps.