Help Fix FlashAttention Build Issue for LoRAEdit (Avatar-to-Video AI Model) Today
Budget: $30 – $250 USD
Summary
We are trying to deploy the LoRAEdit repository, which is an Avatar-to-Video AI model based on LoRA fine-tuning and FlashAttention. The setup is being done on a RunPod container using an RTX 4090 or A100 GPU with CUDA 12.4. We followed all the environment setup instructions (Python 3.10, PyTorch 2.6.0, Flash-Attn 2.8.0+cu124), and successfully installed dependencies.
However, training fails due to a flash_attn_2_cuda import error:
ImportError: flash_attn_2_cuda.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN3c105ErrorC2...
This appears to be a C++ ABI mismatch issue between PyTorch and Flash-Attention. Despite trying both binary and source installs with the correct CUDA version and CMake ABI flags, the issue persists.
We are looking for a senior ML/LLM engineer or CUDA expert who can help us:
Resolve the compatibility issue between Flash-Attn and PyTorch
Get the LoRAEdit training pipeline running successfully
Please only apply if you have experience compiling CUDA extensions or resolving low-level PyTorch/C++ symbol conflicts.
We are trying to deploy the LoRAEdit repository, which is an Avatar-to-Video AI model based on LoRA fine-tuning and FlashAttention. The setup is being done on a RunPod container using an RTX 4090 or A100 GPU with CUDA 12.4. We followed all the environment setup instructions (Python 3.10, PyTorch 2.6.0, Flash-Attn 2.8.0+cu124), and successfully installed dependencies.
However, training fails due to a flash_attn_2_cuda import error:
ImportError: flash_attn_2_cuda.cpython-310-x86_64-linux-gnu.so: undefined symbol: _ZN3c105ErrorC2...
This appears to be a C++ ABI mismatch issue between PyTorch and Flash-Attention. Despite trying both binary and source installs with the correct CUDA version and CMake ABI flags, the issue persists.
We are looking for a senior ML/LLM engineer or CUDA expert who can help us:
Resolve the compatibility issue between Flash-Attn and PyTorch
Get the LoRAEdit training pipeline running successfully
Please only apply if you have experience compiling CUDA extensions or resolving low-level PyTorch/C++ symbol conflicts.