You may need to utilize the gpu_memory_limit and/or lora_on_cpu config solutions to stay away from jogging away from memory. If you still run out of CUDA memory, you'll be able to try and merge in process RAM https://livialavl828375.blogoscience.com/profile