Text Data Labeling in CSV with Python / GenAI

Project Overview:
We need to develop a Python-based solution to automatically classify skills from German vocational training programs. The solution should process multiple Excel files containing training curricula and assign skill labels to each task/skill entry (see screenshot: "Task / Skill" are the variables that need a label, "label_BGT" would be an example for a hand-labelling solution).

Input Data:

- Multiple Excel files containing vocational training curricula
- Key column: "Task / Skill" containing German text descriptions
- Additional column: "Zeithorizont / skill weight" (weight=99 indicates basic skills)
- Hand-labeled training data available for model development

Required Classification Categories:

- Cognitive
- Social
- Digital
- Manual
- Administrative

Technical Requirements:

Python script that:

- Processes multiple Excel files
- Creates new columns for skill labels
- Automatically assigns "basic" label where weight=99
- Allows multiple labels per skill
- Maintains consistency across different occupations


Choice of implementation:

- Train a model using provided hand-labeled data
- OR utilize GenAI for classification
- OR implement mapping to established skill frameworks (e.g., ESCO skills)



Required Skills:

- Python programming
- Experience with text classification
- Data handling (Excel/CSV files)
- German language proficiency
- Understanding of skill taxonomies (preferred)

Deliverables:

- Python code for automated classification
- Processed Excel files with new label columns

Documentation of:

- Implementation approach
- Classification logic
- Usage instructions
- Accuracy metrics (if using ML approach)