Ollama Quick-Start Guide

Ollama is your local AI engine for running fast, private language models - no cloud required. Shinydocs Pro integrates with Ollama to power document enrichment, data extraction, and intelligent tagging, all behind your firewall.

This guide walks you through installing Ollama, binding it to 0.0.0.0 so Shinydocs Pro can connect, and setting up GPU drivers to get the most out of your hardware.

Graphics Processing Unit (GPU) Drivers

For best performance, make sure your system has the latest NVIDIA or AMD drivers installed. Failing to do so may result in Ollama using only your CPU and RAM (very slow).

To confirm your GPU is supported by Ollama, check the official compatibility list here: https://docs.ollama.com/gpu

If your card is listed and drivers are up to date, Ollama will automatically take advantage of your GPU when available.

Sizing models based on your resources

There’s no exact formula for matching hardware to LLM performance. In general, smaller models are faster but less accurate, and larger models are slower but more capable. The key is finding the sweet spot between speed, cost, and the level of insight you need.

General guide for VRAM

Use this as a starting point when deciding which models your hardware can realistically handle:

Additional Notes

• ~1 GB of VRAM per billion parameters is a rough baseline.

• Actual memory use varies based on model type, quantization, and input length.

• Using quantized models (like :q4_K_M) significantly lowers VRAM demands.

• Always test with a small set of documents first to find what works best on your setup.

The goal isn’t just “bigger is better”, it’s about choosing the model that gets good enough answers fast enough for your workflow.

Windows

Install Ollama

1. Download the installer: Download Ollama on Windows

2. By default, Ollama will install to the active user’s profile directory. You can change this by launching OllamaSetup.exe in CMD with an additional argument:
   Change D:\Ollama to where your desired install location

   1. OllamaSetup.exe /DIR="D:\Ollama" 

3. Run the installer and complete the setup

4. Shinydocs recommends running Ollama as a service via NSSM - the Non-Sucking Service Manager. Shinydocs Pro already comes with the NSSM executable in the Shinydocs Extraction Service ([install_location]\Shinydocs Professional\ExtractionService\nssm.exe). You can copy this executable to your Ollama server or download it from NSSM. To configure this:

   1. Open CMD as Administrator

   2. cd to the directory where nssm.exe has been placed
      e.g.cd D:\NSSM

   3. Run nssm.exe install

   4. This will open the GUI to configure the service, input the following information
      

      

      Application tab
      Path: Click the … and browse for ollama.exe (notollama app.exe)
      Startup directory: Auto-populates based on the Path
      Arguments:serve
      Service name: Ollama (or a name of your choice)
      

      

      Process tab
      Priority: High
      Console window: unchecked

   5. Click Install service

5. Open the Services window, you will now have an Ollama service (or whatever you named the service in previous steps). Start it now:
   

   

   
   

6. Confirm Ollama is running by opening your web browser and going to

   http://localhost:11434 

   If Ollama is running, you will see

   

7. The Ollama service will start automatically with the computer without requiring a sign-in

Bind to 0.0.0.0 for external server access

To allow Shinydocs Pro to connect to Ollama:

1. Press Win + S, search for Environment Variables, and open the system settings

2. Under System Variables, click New

   • Variable name: OLLAMA_HOST

   • Variable value: 0.0.0.0

3. Click OK on all dialogs to apply

4. Restart the Ollama app

Ollama will now listen on all interfaces, allowing Shinydocs Pro to connect using your machine's IP address on port 11434.

Increase default context window

1. Press Win + S, search for Environment Variables, and open the system settings

2. Under System Variables, click New

   • Variable name: OLLAMA_CONTEXT_LENGTH

   • Variable value: 10240

3. Click OK on all dialogs to apply

4. Restart the Ollama app

Linux

Install Ollama

1. Visit Download Ollama on Linux for the latest version

2. Start the Ollama service:

   sudo systemctl start ollama 

3. Enable on boot:

   sudo systemctl enable ollama 

4. Test it:

   ollama run llama3 

Bind to 0.0.0.0 for external server access

To allow network access for Shinydocs Pro:

1. Run:

   sudo systemctl edit ollama.service 

2. Add the following lines:

   [Service] Environment="OLLAMA_HOST=0.0.0.0" 

3. Save and close

4. Reload the systemd daemon and restart Ollama:

   sudo systemctl daemon-reload sudo systemctl restart ollama 

Now Ollama will bind to all interfaces.

Increase default context window

1. Run:

   sudo systemctl edit ollama.service 

2. Add the following lines:

   [Service] Environment="OLLAMA_CONTEXT_LENGTH=10240" 

3. Save and close

4. Reload the systemd daemon and restart Ollama:

   sudo systemctl daemon-reload sudo systemctl restart ollama 

Pull some models for use in Shinydocs Pro

Ollama supported models for different use cases

You will need to make sure the model you download will fit into your GPU’s VRAM.

Feel free to explore Ollama’s models and how they might fit your use cases: Ollama Models