LocalLM Lab 1.0.0-beta: one model-calling API across Apple's on-device model, Claude and local open-weight models (in your app)

Community Showcase
,
1

Here's another update on LocalLM Lab. This is release for macOS 27 Golden Gate.

1.0.0-beta.1 adds what I've been calling the model layer: one API surface across Apple's on-device FoundationModels model, Claude (via a host-supplied API key) and locally-run open-weight models (Qwen/Deepseek/Gemma/...). Complicated stuff like routing and model residency is owned by the SDK instead of every app rolling its own provider abstraction.

The front door is LocalLMLab, an optional convenience object wiring a model registry, the MCP manager, and the connector/workspace facades together:

```swift

let lab = LocalLMLab(configuration: .init(providers: [

    SystemModelProvider(),

    ClaudeModelProvider(auth: .apiKey(myKey)),

    MLXModelProvider(residentModelLimit: 1),   // from LocalLMLabSDKInference

]))

lab.models.route(.heavy, to: ModelID("mlx:mlx-community/Qwen3-8B-4bit")!)

lab.models.route(.light, to: .system)

let session = try lab.makeSession(route: .heavy, tools: myTools, instructions: sys)

```

ModelProvider is a protocol keyed by scheme : SystemModelProvider (system), ClaudeModelProvider (claude), MLXModelProvider (mlx, the separate LocalLMLabSDKInference binary), and PCCModelProvider (pcc, Apple's Private Cloud Compute). PCCModelProvider ships but **isn't functional in `1.0.0-beta.1`** — a session routed to .pcc fails outright. Check out Private Cloud Compute - Apple Developer for the details.

RouteName is a name your app maps to a ModelID (.heavy, .light, .draft, or any string). The SDK owns residency but never the routing policy: which route (aka model) a task uses is entirely your call as a developer.

The MLXModelProvider lifecycle is the part most worth detailing, since it's new: validate(_:) preflights a repo before pulling any weights. Preflight includes: reachability, MLX format, architecture support and weight size against MLXPreflightLimits.maxWeightFractionOfRAM (default 0.7 of physical RAM). download(_:) streams progress. capabilityProbe(_:) runs a real prompt plus a trivial tool call after download. This is the authoritative signal for whether a given model can reliably tool-call, not something to assume from its name. residencyEventStream emits .warmed/.evicted(reason:)/.loadProgress for a status line. residentModelLimit caps how many models stay resident in RAM at once; switching routes evicts the other.

Weights land in a standard Hugging Face cache ~/.cache/huggingface/hub/ for a bare CLI, redirected automatically to ~/Library/Containers/<bundle-id>/Data/Library/Caches/huggingface/hub/ under App Sandbox. A sandboxed app also needs com.apple.security.network.client for the download itself. This is the same silently-hanging-without-it entitlement gotcha called out in the SDK launch.

makeSession(route:tools:instructions:includeMCPTools:) resolves the route to a model and merges your tools with the enabled MCP session tools automatically (opt out with includeMCPTools: false). session.events carries the side-channel Apple's streaming doesn't give you (.toolCallStarted/.toolCallFinished/.contextCompacted/.modelLoadProgress) and session.contextBudget plus an optional retryOnContextOverflow compact-hook retry are there for sessions long enough to fill a context window.

Two new reference apps show this end to end: code-buddy, a CLI coding agent with .heavy/.light MLX routes, Core's Workspace tools, a live MCP docs server, and streamed output; and repo-qa-local, the minimal version of the same idea. workspace-buddy-local is the sandboxed-MLX filesystem example. There are 8 reference apps in the repo now. All for your reviewing pleasure!

Requires macOS 27 beta, Apple Intelligence enabled, Swift 6 tools, Xcode 27 beta for SDK development.

SDK guide: locallm/docs/sdk-guide.md at main · ancientcomputing/locallm · GitHub

Feature page: thisbrain.ai/locallm/1.0.0-beta

1 Like