Deploying an Ooba Instance
There are two primary methods of standing up an Oobabooga interface
Direct via Python
Most LLMs and applications built around them are Python based. As such, setting up Python directly can be a direct path to running LLMs or interfaces.
It is recommended to use Python 3.10+ as many interfaces and modules are built around features available from this version onwards.
To setup Python on your system:
Windows
Install Python 3.10+ from Python.org
Linux
Check if Python is pre-installed with your OS:
python3 --version
If Python 3.10+ is not available, follow this guide to install it.
MacOS
Install Python 3 with Brew:
brew install python
If you do not have brew installed, please follow the Install process here.
In a Container w/ Docker
One disadvantage of a direct Python deployment is the inability to quickly replicate that environment. Say you want to share your setup with others or if an error occurs.
Docker can allow you to easily replicate, share or even redeploy your interface or application.
Docker for Windows
Docker for Linux
- Utilize the Docker provided convenience script
Docker for MacOS
Proceeding with Setup
Now that you have Python and/or Docker installed, you can continue on to deploying Oobabooga text-generation-ui.