Docker-Powered AI: A Beginner's Guide to Deploying Ollama and Open WebUI with Docker Compose

Docker-Powered AI: A Beginner’s Guide to Deploying Ollama and Open WebUI with Docker Compose

by

Docker-Powered AI: A Beginner’s Guide to Deploying Ollama and Open WebUI with Docker Compose

Welcome to the world of self-hosted AI! If you’re a developer or a Docker enthusiast looking to run powerful Large Language Models (LLMs) on your own hardware, you’re in the right place. This guide will show you how to deploy a complete, containerized AI stack featuring Ollama and the user-friendly Open WebUI in just a few minutes, all thanks to the magic of Docker Compose.

What is Ollama?

Ollama is a powerful and lightweight tool that allows you to run open-source LLMs, such as Llama 3, Mistral, and many others, directly on your local machine. It handles the complexities of model management, GPU acceleration, and provides a simple API, making it incredibly easy to get started with local AI development.

What is Open WebUI?

Open WebUI is a user-friendly, ChatGPT-style web interface for your Ollama LLMs. It provides an intuitive chat interface, allows for easy model management, and supports advanced features, making your interaction with local AI models seamless and efficient. Think of it as the perfect front-end for your powerful Ollama back-end.

Prerequisites

Before we begin, ensure you have the following installed on your system:

  • Docker
  • Docker Compose

That’s it! No complex dependencies or manual installations are required.

The Docker Compose File

Docker Compose allows us to define and run multi-container Docker applications. Below is the simple docker-compose.yml file that will orchestrate our entire AI stack. It defines two services: ollama and open-webui.


version: '3.8'

services:
  ollama:
    image: ollama/ollama
    container_name: ollama
    ports:
      - "11434:11434"
    volumes:
      - ./ollama_data:/root/.ollama
    restart: unless-stopped

  open-webui:
    image: ghcr.io/open-webui/open-webui:main
    container_name: open-webui
    ports:
      - "3000:8080"
    volumes:
      - ./open_webui_data:/app/backend/data
    depends_on:
      - ollama
    environment:
      - 'OLLAMA_BASE_URL=http://ollama:11434'
    restart: unless-stopped

volumes:
  ollama_data:
  open_webui_data:

Step-by-Step Deployment

Follow these simple steps to get your AI environment up and running.

Step 1: Create the File

Create a new directory for your project. Inside that directory, create a file named docker-compose.yml and paste the content from the code block above into this file.

Step 2: Run Docker Compose

Open your terminal, navigate to the directory where you created the file, and run the following command to start both containers in detached mode:


docker-compose up -d

Docker will now pull the necessary images for Ollama and Open WebUI and start the services. This might take a few minutes depending on your internet connection.

Step 3: Access Open WebUI

Once the command completes, open your web browser and navigate to http://localhost:3000. You should be greeted by the Open WebUI setup screen. Create your first admin account to log in.

Step 4: Pull Your First AI Model

Now that the stack is running, you need an AI model to chat with. You can do this directly from the Open WebUI interface by clicking “Settings” and then “Models”. Alternatively, you can use the command line.

To pull the recommended Llama 3 8B model, run the following command in your terminal:


docker exec -it ollama ollama pull llama3

Once the model is downloaded, it will appear in the model selection dropdown in Open WebUI. Select it and start chatting!

Conclusion

Congratulations! In just a few minutes, you have successfully deployed a powerful, self-hosted AI chat environment using Docker, Ollama, and Open WebUI. This setup provides a fantastic sandbox for experimenting with various open-source models, developing AI-powered applications, and exploring the future of artificial intelligence, all from the comfort of your own machine.

Leave a Comment