Sleepy Macs Need Caffeine for LLMs

Macs were designed for power efficiency! They sleep when the lid is down and wake instantly when opened. They sleep when a timer runs down and nobody pressed the shift key for a bit. Fine if you’re working on a laptop all day and didn’t bring your charger. But run an LLM on a Mac Mini in a closet it feels like your Mac caught a case of digital narcolepsy. For what I do in my hobbies and work a sleepy Mac just won’t do; it must remain wide awake and perpetually caffeinated.

All the parts work, but not together.

Running a local LLM on my Macs was drop dead simple: I downloaded Ollama, opened the GUI and started a chat with an LLM. Yet I needed them to behave like proper headless servers, and like every good PM I began to think about the requirements. That starts with minimizing the need for the keyboard, monitor and mouse but more’s required.

1. What are the optimized power settings to maximize performance and minimize the chance it may go to sleep mid-inference?
2. How can I connect an external volume and easily point inference services there to cache models so the boot disk doesn’t fill up?
3. What if an unattended update caused a reboot in the middle of an inferencing run causing the long running process to die?
4. Given the available RAM on the machine what models are a reasonably good fit?
5. If an agent or harness wants to check server busyness and running models how?
6. Can we easily setup services beyond Ollama like Exo, Rapid-MLX, Infinity, and so on?

So off I went all optimistic and hopeful, believing someone else likely solved these problems, but actually not so much.

I found a problem many were grappling with, but few solved completely.

A decent amount of work already exists, but it’s never been assembled under the same tent nor the same way twice.

anurmatov/mac-studio-server, and two others...

By far this is the closest single competitive open source project. Composed of shell scripts and plists it focuses on headless Ollama on Apple Silicon. mac-studio-server contains features like sleep prevention, LaunchDaemon autostart, SSH remote management, and service suppression that claims to cut memory use from 11GB to 3GB. It's well recognized with 352 stars and 39 forks. Yet, it’s Ollama-only, plain shell, seven commits total with the most recent sometime last year, lacks pre-check/verify/restore lifecycle, no storage-volume automation, and no hardware telemetry. A couple of other projects, jpalat/local-llm-setup and mrmoe28/mac-mini-llm-setup, cover similar ground with less structure, closer to notes than to tools.

llmserve, LlamaStash, Olla

A second category of projects/tools solve a different part of the problem.

• llmserve (Rust, 388 stars) and LlamaStash (v0.2.0, 137 stars) both auto-detect and launch multiple backends, Ollama, MLX, vLLM, llama.cpp, KoboldCpp, LocalAI, LM Studio, from a TUI.
• Olla is a load balancer that sits in front of backends you’ve already setup.

But all of these miss OS hardening and setting the system up to be resilient when restarted meaning: no pmset, no sysctl, no SIP handling, a missing service account, no SSH hardening, no LaunchDaemon management, and no storage-volume work.

The competitive view said build it.

There were clear gaps where my needs diverged from the field. I wanted an appliance-like system with associated hardening (i.e., pmset, sysctl, SIP, SSH, a dedicated service account, LaunchDaemon conventions). I needed it idempotent in case we needed to stop mid-way and restart, auditability plus reversibility to revert back, multi-tool serving orchestration, storage-volume automation, and hardware telemetry. My sense was these all needed to be in one tool, for one machine and not loosely linked together in my own head. So off I went, with my new friend Claude and some critical thinking, to build the thing stitching them all together.

What resulted from my requirements and friendly + not so friendly chatting.

headless-macs is a Go binary with an interactive terminal UI (TUI) plus equivalent CLI subcommands. This way it works as you're remotely SSH'ed in and want to start from a menu driven experience or you want to remotely execute from a central system. Here’s a walk through of general steps it takes.

• Precheck — a read-only audit before anything is touched
• Storage setup — find, validate, and wire up an external volume for model storage covering required ownership changes, symlinks, fstab, and a re-mount process to ensure it comes up after reboot.
• System baseline — including pmset, sysctl, SSH, and daemon settings overriding sleepy defaults to let a Mac actually behave more like a 24/7 server.
• Install tools — installs and auto-tunes the serving stack (things like Ollama’s MAX_LOADED_MODELS, NUM_PARALLEL, and context size) based on the RAM it detects rather than a generic default.
• Verify — to perform a health check across everything you’ve enabled and installed.
• Update tools — in-place upgrades that re-verify each service actually responds afterward, instead of assuming success and walking away.
• Restore — undo all of it, cleanly.

There’s even a live dashboard to quickly show key service state, PIDs, memory, CPU%, plus hardware telemetry (CPU/GPU power draw, temperature) if you turn on macmon. The config lives in a single JSON file, seeded from a template on first run. By design there's no CLI flag to edit config values directly, instead you edit the file or use the TUI editor, and the subcommands only run things. That separation enshrines the config values cleanly and prevents me from forgetting a flag I typed six months ago.

Press enter or click to view image in full size

I didn’t Go for cleanliness in the beginning.

The first version wasn’t based on Go at all, instead it was two bash scripts (i.e., pmset_to_ollama.sh & setup_colima.sh) to perform power management and Docker/Colima setup with no shared structure between them. That quickly change growing into a proper bash pipeline (i.e., precheck.sh, setup.sh, install-tools.sh, verify.sh and restore.sh) and a shared library with consistent messaging. This was a significant improvement, but not good enough for me. Eventually the shell pipeline became was cleaned up by using Go and a proper TUI plus CLI was generated. If the reader is curious the old bash scripts are still in the repo, printing a deprecation notice and exit 0 rather than vanishing. At some point they will be removed, but I leave them just in case there’s a copy running someplace I don’t know about. I suppose breaking a thing is worse than leaving a fossil behind.

Realizing inferencing service deployment choice + a little observability.

headless-macs doesn't marry you to a single inferencing service, and that’s because no single one wins every time. If you simply need something for embeddings/reranking, Infinity is the right fit. Want to run a large model and have a collection of Macs with the right interconnect, then Exo is your best bet. This was a primary design point for headless-macs: give choice to the administrator to tailor the system to match the need at hand.

• Ollama — the default, most mature, and easiest way to manage models. However, when running as a service logs can grow without bounds which was a known problem as of V0.33.2. Therefore, headless-macs employs a brew install of logrotate to manage log files.
• Rapid-MLX — purpose-built for coding agents, with 17 tool-call parsers, and prompt caching for hybrid attention models. It used to be 2–4.2× faster than Ollama; now that Ollama has MLX support, that gap is down to roughly 15–30%. The trade-off: it holds its full model resident in memory for the life of the daemon which reduces model selectivity, but may result in extra efficiency.
• mlx-lm — Apple’s own lower-level MLX serving layer, useful when you need a specific HuggingFace model path the higher-level tools don’t alias.
• Infinity — an embeddings and reranking server, the half of a RAG pipeline that generation tools don’t cover.
• Exo — the multi-machine memory pooling service, with some caveats. Right now, the downloadable app runs as a user-level LaunchAgent, not a LaunchDaemon, meaning it needs auto-login to survive a headless reboot. This isn’t a hard architectural limit, just the current state. Pulling the Exo source and wrapping it to run as a proper background service looks doable, I just haven’t done it yet. Until then, I’d rather tell you about the gap than have you discover it after a power outage or system update.
• macmon — hardware telemetry, not an inference tool, which is off by default, but can come in pretty handy for remote observabiliyt, debugging and automation when enabled.

Suggesting models that are a likely fit on your Mac.

Instead of letting you “hunt and peck” with/for various models, headless-macs, includes a list of several models to consider as a starting point. This list is not an endorsement just some suggestions for models that will fit. To get to the right model, context window, concurrency approach, etc. you have to understand your usage scenario, Mac, workload, etc.

• MacBook Air M3/M4, 16 GB — qwen3:8b at Q4 (5 GB), one model at a time.
• MacBook Air M3 / Mac Mini M4, 24 GB — qwen3:14b at Q4, or the 30B MoE qwen3-coder:30b at Q4 (19 GB).
• MacBook Pro M4 / Mac Mini M4 Pro, 32 GB — qwen3:32b at Q4 (20 GB) or deepseek-r1:32b, two models.
• MacBook Pro M4 Max / Mac Studio M4 Max, 64 GB — llama3.3:70b or deepseek-r1:70b at Q4 (43 GB), with a 32B model at Q8 alongside if you need a second.
• MacBook Pro M4 Max / Mac Studio M4 Max (Mac16,9), 128 GB — a 70B model at Q8 (near lossless) or qwen3.5:122b at Q4 (81 GB); a 70B Q4 plus a 32B Q8 pair also fits. Note: qwen3:235b at Q4 (142 GB) does not fit here, that needs the tier below.
• Mac Studio M3 Ultra, up to 256 GB — qwen3:235b at Q4 (142 GB), or multiple 70B models running simultaneously.
• Mac Pro M2 Ultra, 192 GB — a 70B model at Q8 plus a second 70B at Q4 simultaneously, or a single ~230B-class model at Q4.

Apple announced M5 Max and M5 Ultra in August 2026, with units shipping by September 22. This update restores higher RAM ceilings, with a 512 GB configuration following the September release in October. I’m sure there will be new news about how models behave on these newer versions. Additionally, for both work and my hobbies I’ll be updating the list with real use cases, workloads and behaviors on my Mac home lab. So stay tuned…

The warts that remain and those that were overcome.

Apple’s opinionated design makes usability and personal productivity super first class. Practically, it means that for several configuration changes manual intervention is needed and in some cases physical access is required. Most of these remaining warts can’t be overcome through automation.

• FileVault — if left enabled will hang a headless reboot at the password prompt. Unfortunately, a remedy to fix this automatically isn’t yet implemented, so at this stage this has to be turned off manually, via sudo fdesetup disable over SSH or via physical access to the box.
• System Integrity Protection (SIP) — blocks persistent service disabling unless disabled through Recovery mode via physical access to the box.
• Turning on RDMA — Exo can use RDMA over Thunderbolt to pool memory with low latency, and turning it on needs explicit setup via Recovery mode with physical access to the box.
• High power mode — there’s no pmset key or CLI equivalent to execute; it’s GUI-only. However, unlike SIP and RDMA, this happens after MacOS is fully booted, so if you’ve already got Screen Sharing enabled or physical access to the box it can be enabled.
• Lid-close sleep — on MacBooks is better solved with a physical HDMI/USB-C dummy plug than with the tempting pmset -a lidwake 0 workaround, which carries real thermal risk if the machine’s vents end up blocked while closed.

In other cases limitations were observed but automation was still possible. That means these warts were overcome.

• panic: $HOME is not defined — is a real crash you’ll hit if HOME isn’t set to a default like /Library/LLMServer. launchd doesn’t set it for you the way an interactive shell does.
• Logs grow without bounds — Ollama was observed with significant log growth in /var/log, caused by a bug and by required verbosity coming from llama.cpp.. logrotate was deployed, instead of MacOS newsyslog, because its treatment of log files is congruent with launchd.
• xcode-select — install fails headless — the installer expects a display to click through a GUI dialog, which a headless box doesn’t have. softwareupdate’s own package list was used instead to find and install the Command Line Tools package directly, without a dialog.

What’s next

For cases like SIP, RDMA, and High Power Mode, Apple would need to take action: either bring forward their Private Cloud Compute server infrastructure, or build a proper server distribution of MacOS. Recovery Mode access could also use something like Dell’s iDRAC for enterprise remote management. We’ll see what Apple does.