Skip to main content
Version: Next

Conceptual guides

📄️ Model memory management

One of the most important things to consider when running a model is how much memory it will use. This is especially important when running large models on a GPU, as the memory may be limited. If you run out of memory, the model will crash and may need to be restarted manually. This can be very frustrating, especially if you have deployed the model to a server and are running it remotely. If you have the luxury of access to multiple GPUs, these memory requirements can be (in effect) combined by taking advantage of multi-gpu deployment.