Run DeepSeek Locally: A Guide to Self-Hosting for Privacy and Control

General
1

By the end of this guide, you’ll be able to successfully self-host your DeepSeek model of choice on a home lab or home office server, tapping into the potential of these AI models within a private and managed environment.

DeepSeek offers a powerful AI model that, when self-hosted locally, provides faster performance, enhanced privacy, and configuration flexibility. This guide illustrates how to self-host DeepSeek in a home lab setting, enabling access from multiple devices on your local network. We will cover self-hosting DeepSeek, setting up a web interface, and ensuring it runs as an always-online service.

Run DeepSeek with a Web Interface Using SSH Tunnel

Using an SSH tunnel is the most secure and likely the most effective way to connect to your DeepSeek instance.

Step 1: Ensure the SSH server is installed on the local server.

sudo apt update
sudo apt install openssh-server

Step 2: Start and enable the SSH service on the local server.

sudo systemctl start ssh
sudo systemctl enable ssh

Note: On some distributions, the service might be named sshd instead of ssh. Verify the service name using:

sudo systemctl status ssh
# or
sudo systemctl status sshd

Step 3: Configure the firewall on the local server to allow SSH and Open WebUI ports. Note: Replace ufw with your firewall manager if different. Allow SSH (Port 22) from the local network:

sudo ufw allow from 192.168.1.0/24 to any port 22 proto tcp

Replace 192.168.1.0/24 with your local network’s subnet if different.

Allow Open WebUI (Port 8080) from the local network:

sudo ufw allow from 192.168.1.0/24 to any port 8080 proto tcp

Replace 192.168.1.0/24 with your local network’s subnet if different.

Step 4: Create an SSH tunnel from your local machine.

ssh -L 8080:localhost:8080 user@192.168.1.100

Replace 8080 with your preferred port, user with your SSH username, and 192.168.1.100 with the DeepSeek server’s local IP address.

Explanation:
-L 8080:localhost:8080: Forwards your local port 8080 to the server’s port 8080.
user@192.168.1.100: Replace user with your SSH username and 192.168.1.100 with the server’s local IP address.

Step 5: (Optional) If SSH is on a different port (e.g., 2222), use the -p flag:

ssh -p 2222 -L 8080:localhost:8080 user@192.168.1.100

Step 6: Access Open WebUI locally.

Once the SSH tunnel is established, open your browser and go to:

http://localhost:8080

Now, Open WebUI is only accessible on your local home/office network via http://localhost:8080.

Advantages:

  • Security: Traffic is encrypted via an SSH tunnel.

  • No extra ports open: Including the SSH port, which is only accessible from within your local network.

  • Easy: Uses existing SSH infrastructure.

Considerations:

  • Manual: Requires a tunnel to be established each time you want to access Open WebUI.

  • Local Port Available: Make sure your local port 8080 is free, or choose a different local port.

  • Persistent Tunnels: For automatic reconnection, consider using tools like autossh.

Automation: While the manual method requires setting up the tunnel each time, you can automate this process using tools like autossh to maintain persistent tunnels that also automatically reconnect:

sudo apt install autossh
autossh -M 0 -f -N -L 8080:localhost:8080 user@192.168.1.100

Run DeepSeek with a Web Interface (Open WebUI)

By using Open WebUI, you can effortlessly interact with DeepSeek through a centralized, user-friendly dashboard accessible from any device on your local network. Open WebUI can be installed using pip, the Python package installer.

Step 1: Install Open WebUI using pip.

Open your terminal and run the following command to install Open WebUI:

pip install open-webui

Step 2: Alternatively, you can install with Snap instead:

sudo apt update
sudo apt install snapd
sudo snap install open-webui --beta

Step 3: After installation, start Open WebUI by executing:

open-webui serve

This will start the Open WebUI server, which you can access at http://localhost:8080 or replace localhost with your server’s local IP address to access from other devices on your network.

Using Reverse Proxy with Nginx

Alternatively, you can place DeepSeek behind Nginx.

Step 1: Install Nginx.

sudo apt install nginx

Step 2: Create a reverse proxy configuration file.

sudo vi /etc/nginx/sites-available/deepseek

Step 3: Add the following configuration:

server {
    listen 80;
    server_name your-local-domain.local;

    location / {
        proxy_pass http://localhost:8080;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    }
}

Step 4: Enable the configuration and restart Nginx.

sudo ln -s /etc/nginx/sites-available/deepseek /etc/nginx/sites-enabled/
sudo nginx -t  # Test the configuration
sudo systemctl restart nginx

Now, access via: https://your-local-domain.local/

Why Self Host DeepSeek

Note: DeepSeek-R1 is a 671B model with a Mixture of Experts (MoE) architecture requiring 1.5 TB of VRAM, making it impractical for consumer hardware. DeepSeek-R1 distilled models, like DeepSeek-R1-Distill-Qwen-7B and DeepSeek-R1-Distill-LLaMA-70B, are fine-tuned versions of open-source models like LLaMA and Qwen, trained on data generated by DeepSeek-R1, thus inheriting DeepSeek’s reasoning capabilities while being far more efficient to self-host.

Self-hosting DeepSeek gives you:

  • Privacy: Data resides on your infrastructure, not third-party servers.

  • Speed: Experience lower network latency, especially with smaller models.

  • Custom Hardware: Configure CPU, GPU, and memory allocation according to your needs.

  • Scalability: Expand your home lab as needed.

  • Control: Eliminate external dependencies or vendor lock-in.

  • Learning: Enhance your skills by managing and optimizing your own AI infrastructure.

Why not

Some of the challenges include:

  • Model Bias and Censorship: Restricted/censored responses on sensitive topics (or use open-r1).

  • Cost: High initial and ongoing expenses for hardware and electricity.

  • Longevity: Future AI models will likely require fairly frequent hardware upgrades.

  • Maintenance: Requires regular updates and ongoing technical upkeep.

  • Scalability Limits: Physical space, noise, and heat challenges can hinder expansion.

Preparing to Self-Host DeepSeek

Before setting up DeepSeek, ensure your system meets the necessary hardware requirements, especially if running larger models that require high RAM and GPU resources.

  • CPU: A powerful multi-core processor (12+ cores recommended) for handling multiple requests.

  • GPU: NVIDIA GPU with CUDA support for accelerated performance. AMD will also work (less popular/tested).

  • RAM: Minimum 16 GB, preferably 32 GB or more for larger models.

  • Storage: NVMe storage for faster read/write operations.

  • Operating System: Ubuntu or Ubuntu-based distributions are preferred for compatibility.

  • Censorship/restrictions: If you are looking for DeepSeek-R1 with no censorship or content restrictions, try open-r1.

  • Distilled models: Read more about these official models here: api-docs.deepseek.com/news/news250120.

  • More detailed hardware guidance can be found here.

Here are three recommended GPUs: NVIDIA RTX 4080, NVIDIA RTX 4090, and the NVIDIA A100. Find them also on eBay used.

Install

Step 1: Install and run using:

curl -fsSL https://ollama.com/install.sh | sh
ollama run deepseek-r1:8b

Running DeepSeek in your home lab offers more control, privacy, and performance, eliminating reliance on external cloud dependencies, increasing security, and offering more customization options, making it perfect for home lab enthusiasts, developers, and professionals.