SMALL SINGLE Feature Implementation for Open Source Text Diffusion Transformer
Budget: $30 – $250 CAD
Objective:
This is not a project. It's one feature for: https://github.com/louaaron/Score-Entropy-Discrete-Diffusion/tree/main
The training code right now uses DDP (distrbuted data parallel) to train the model.
I want to use FSDP (fully sharded data parallel) instead so a decent model can be trained on consumer gpus.
This is not a project. It's one feature for: https://github.com/louaaron/Score-Entropy-Discrete-Diffusion/tree/main
The training code right now uses DDP (distrbuted data parallel) to train the model.
I want to use FSDP (fully sharded data parallel) instead so a decent model can be trained on consumer gpus.