TIL: ollama notes

01-Nov-2024 196 words One minute

TIL - that docker for mac doesn’t allow you to access the GPUs that you have on your mac running apple Si.

this is annoying, it means that i need to run ollama with some wrappers, which isn’t the end of the world, but a bit more hassle than i was looking for.

misc. notes in no particular order

default ollama port 11434
making it such that i can connect to this from off localhost (i.e., from the ipad using enchanted) requires a bit more chicanery.
1. stop ollama
2. launchctl setenv OLLAMA_HOST "0.0.0.0"
3. (re)start ollama
if using the non-cask version from homebrew it’s possible to handle this via a modification to the plist file that homebrew creates for the service entry.

remember this for future reference.

documentation pointers

https://github.com/ollama/ollama/issues/3581 - a rather snarky bit of discussion on this topic.

update the plist and move on with life

brew services stop ollama - to make sure nothing bumps the config
the following should be placed in /opt/homebrew/opt/ollama/homebrew.mxcl.ollama.plist
brew services start ollama - to start ollama again.

there’s probably some “allowing remote connections” prompts to work thru on the GUI.

	<?xml version="1.0" encoding="UTF-8"?>
	<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
	<plist version="1.0">
	<dict>
	<key>KeepAlive</key>
	<true/>
	<key>Label</key>
	<string>homebrew.mxcl.ollama</string>
	<key>LimitLoadToSessionType</key>
	<array>
	<string>Aqua</string>
	<string>Background</string>
	<string>LoginWindow</string>
	<string>StandardIO</string>
	<string>System</string>
	</array>
	<key>ProgramArguments</key>
	<array>
	<string>/opt/homebrew/opt/ollama/bin/ollama</string>
	<string>serve</string>
	</array>
	<key>RunAtLoad</key>
	<true/>
	<key>StandardErrorPath</key>
	<string>/opt/homebrew/var/log/ollama.log</string>
	<key>StandardOutPath</key>
	<string>/opt/homebrew/var/log/ollama.log</string>
	<key>WorkingDirectory</key>
	<string>/opt/homebrew/var</string>
	<key>EnvironmentVariables</key>
	<dict>
	<key>OLLAMA_HOST</key>
	<string>0.0.0.0</string>
	</dict>
	</dict>
	</plist>

view raw homebrew.mxcl.ollama.plist hosted with ❤ by GitHub

TIL: ollama notes

misc. notes in no particular order

documentation pointers

update the plist and move on with life

meta