A Life? Cool! Where can I download one of those?

LLM Adventures on the deck

 

 

Notes:

Context is limited to 32k for testing stability but with ~3gb of headroom I should be able to give it more. Testing ensues...

To force the screen off I had to manually login and use: setterm --blank force which leaves a session open. I am trying to find a way to do this via systemd or via a remote ssh session.

Model(s) Currently testing: 

 

BIOS:

Video UMA: 1gb

Bluetooth: Disabled

Memory Power down: Disabled

 

Debian Testing (as of 10/28/2026)

Vulkan SDK from source: 1.4.363.0

llama.CPP from git - 10-28-2026

only packages required for above + tmux / btop

Minimal services / no GUI

llama.cpp runs as a user systemd service

 

/etc/default/grub:

GRUB_CMDLINE_LINUX_DEFAULT="amdgpu.gttsize=110592 amdttm.page_pool_size=27648000 amdttm.pages_limit=27648000 ttm.pages_limit=27648000"

 

~/bin/llama-server.sh (started from llama-server.service)

export GGML_CUDA_ENABLE_UNIFIED_MEMORY=1
export GGML_VK_FORCE_MAX_ALLOCATION_SIZE=4294967295
export GGML_VK_SUBALLOCATION_BLOCK_SIZE=4294967295
export GGML_VK_NODES_PER_SUBMIT=32

llama-server --api-key-file /home/kidacro/.llama-api-keys -c 32768 --jinja -fa on -ngl 32 --host 0.0.0.0 --port 18080 --log-file /home/kidacro/logs/llama.log -v --log-timestamps --ui-mcp-proxy --models-dir /home/kidacro/models

 

Results...

... profit! For now...