Build NLP Training Pipeline (Text-to-Text)
Budget: €250 – €750 EUR
The goal of this project is to setup a NLP Pipeline in Python that takes our training data (examples of Input and Output texts), transforms it and train the generative model GPT-Neo. In the end the model should be easily swapped out with other generative models as GPT-Neox, GPT-J, Bloom.
You must not use high-level libraries like HappyTransformers etc., instead you should use PyTorch and Huggingface (and other important libraries) to build the project. Your code should be well structured and the hyperparameters should be easy to adjust.
You have to document each step to reproduce the exact same result. Furthermore, you should be able to give us advice how we should best proceed with enhancing the model you created.
You must not use high-level libraries like HappyTransformers etc., instead you should use PyTorch and Huggingface (and other important libraries) to build the project. Your code should be well structured and the hyperparameters should be easy to adjust.
You have to document each step to reproduce the exact same result. Furthermore, you should be able to give us advice how we should best proceed with enhancing the model you created.