Forem: Tatsuya Tanaka

LocalLLMClient: A Swift Package for Local LLMs Using llama.cpp and MLX

Tatsuya Tanaka — Thu, 15 May 2025 16:37:21 +0000

LocalLLMClient: A Swift package for Local LLMs

Hi!

Recently, Large Language Models (LLMs) that can run locally on your device have been gaining a lot of attention. As someone who frequently works with Apple platforms, I wanted an easy way to use various local LLMs from Swift. After researching existing solutions and not finding one that fit my needs, I decided to create my own.

In this article, I'm introducing LocalLLMClient, a library that makes it simple to use local LLMs from Swift. Of course, you can use it in both macOS and iOS apps!

tattn / LocalLLMClient

Swift package to run local LLMs on iOS, macOS, Linux

LocalLLMClient

A Swift package to interact with local Large Language Models (LLMs) on Apple platforms.

Demo / Multimodal

MobileVLM-3B (llama.cpp)	Qwen2.5 VL 3B (MLX)
llamacpp-mobilevlm.mov	mlx-qwen2.5.mov

iPhone 16 Pro

Example app

Important

This project is still experimental. The API is subject to change.

Tip

To run larger models more reliably, consider adding com.apple.developer.kernel.increased-memory-limit entitlement to your app.

Features

Support for GGUF / MLX models / FoundationModels framework
Support for iOS, macOS and Linux
Streaming API
Multimodal (experimental)
Tool calling (experimental)

Installation

Add the following dependency to your Package.swift file:

dependencies: [
    .package(url: "https://github.com/tattn/LocalLLMClient.git", branch: "main")
]

Usage

The API documentation is available here.

Quick Start

import LocalLLMClient
import LocalLLMClientLlama
let session = LLMSession(model: .llama(
    id: "lmstudio-community/gemma-3-4B-it-qat-GGUF",
    model: "gemma-3-4B-it-QAT-Q4_0.gguf"
))

print(try await session.

…

View on GitHub

What LocalLLMClient Can Do

Multiple Backend Support
- Uses llama.cpp and Apple MLX under the hood, with the same interface for both backends
iOS/macOS Support
- Runs on both iPhones and Macs
Streaming API
- Leverages Swift Concurrency for a nice streaming experience
Multimodal Support
- Handles not just text but images as well

I built this because I often find myself in situations where I want to use MLX for its faster performance, but still need llama.cpp for newer models that MLX doesn't support yet.

The library also supports VLMs (Visual Language Models) from both backends, allowing you to ask questions like "What's in this photo?" Even on iOS, the Qwen 2.5 VL 3B 4bit model just barely runs on an iPhone 16 Pro.

How to Use It

For the latest information, please check the GitHub repository.

The library is provided as a Swift Package. It's modularized so you can import only what you need:

LocalLLMClient: Common interfaces
LocalLLMClientLlama: llama.cpp backend
LocalLLMClientMLX: Apple MLX backend
LocalLLMClientUtility: Utilities like LLM model downloaders

Text Generation

Here's a simple example:

import LocalLLMClient
import LocalLLMClientLlama
import LocalLLMClientUtility

// Download the model (e.g., Gemma 3)
let ggufName = "gemma-3-4B-it-QAT-Q4_0.gguf"
let downloader = FileDownloader(source: .huggingFace(
    id: "lmstudio-community/gemma-3-4B-it-qat-GGUF",
    globs: [ggufName]
))

try await downloader.download { progress in
    print("Download progress: \(progress)")
}

// Initialize the client
let modelURL = downloader.destination.appending(component: ggufName)
let client = try await LocalLLMClient.llama(url: modelURL, parameter: .init(
    context: 4096,      // Text context size
    temperature: 0.7,   // Randomness (0.0-1.0)
    topK: 40,           // Top-K sampling
    topP: 0.9,          // Top-P (nucleus) sampling
    options: .init(responseFormat: .json) // Response format
))

let prompt = """
Create the opening of an epic story where a cat is the protagonist.
Format as JSON like this:
{
    "title": "<title>",
    "content": "<content>",
}
"""

// Generate text
let input = LLMInput.chat([
    .system("You are a helpful assistant."),
    .user(prompt)
])

for try await text in try await client.textStream(from: input) {
    print(text, terminator: "")
}

Here's an example result:

{
  "title": "Shadow of the Moon's Claw",
  "content": "Long ago, when humans still dreamed of stars, the world was ruled by cats. With their intelligence and grace, cats governed kingdoms across the land, keeping humans as their adorable pets. But even in the world of cats, conspiracies and secrets swirled. The 'Moon Shadow Cats,' a lineage dominated by powerful magic, plotted to use their powers to control the world. The protagonist, Mika, decides to embark on a journey to stop this conspiracy. She meets a legendary cat sage and acquires ancient knowledge and magical powers. However, Moon Shadow Cats' pursuers relentlessly chase Mika, starting a battle that will shake the world where cats and humans coexist. To stop the Moon Shadow Cats' conspiracy, Mika must accept her destiny and set off on an epic adventure to save the world."
}

Multimodal

Here's an example that includes an image input:

import LocalLLMClient
import LocalLLMClientMLX
import LocalLLMClientUtility

// Download the model files (e.g., Qwen2.5 VL 3B)
let downloader = FileDownloader(source: .huggingFace(
    id: "mlx-community/Qwen2.5-VL-3B-Instruct-abliterated-4bit",
    globs: .mlx
))

try await downloader.download { progress in
    print("Download progress: \(progress)")
}

let client = try await LocalLLMClient.mlx(url: downloader.destination)

// Create input with an image
let input = LLMInput.chat([
    .user("Describe what's in this photo as a song", attachments: [.image(<image>)]),
])

// Get the text all at once without streaming
print(try await client.generateText(from: input))

When I tested with a photo of a stone angel statue, I got this result:

Lyrics for this photo:

Wings folded on the tombstone
In the distant beyond
Binding love
Where wishes dwell

This song expresses the image of an angel with folded wings on a tombstone. It represents the wishes dwelling on the tombstone that connect to a distant love. The image portrays an angel figure carved on a gravestone with wishes dwelling within it.

Additional Features

The FileDownloader in LocalLLMClientUtility includes features like skipping downloads for models that are already stored and background downloading capability that allows iOS apps to continue downloading models even when the app is in the background..

Check out the sample app to see it in action!

Conclusion

The Apple platforms I usually work with have a philosophy of prioritizing privacy and trying to run things on-device as much as possible, which I strongly support. Also, as someone who's afraid of accidentally burning through money when playing with AI services, I'm incredibly grateful to everyone working on developing, providing, and utilizing local LLMs. Thank you!

I'd be happy if LocalLLMClient becomes one of your options when you want to play with AI in Swift.

tattn / LocalLLMClient

Swift package to run local LLMs on iOS, macOS, Linux

LocalLLMClient

A Swift package to interact with local Large Language Models (LLMs) on Apple platforms.

Demo / Multimodal

MobileVLM-3B (llama.cpp)	Qwen2.5 VL 3B (MLX)
llamacpp-mobilevlm.mov	mlx-qwen2.5.mov

iPhone 16 Pro

Example app

Important

This project is still experimental. The API is subject to change.

Tip

To run larger models more reliably, consider adding com.apple.developer.kernel.increased-memory-limit entitlement to your app.

Features

Support for GGUF / MLX models / FoundationModels framework
Support for iOS, macOS and Linux
Streaming API
Multimodal (experimental)
Tool calling (experimental)

Installation

Add the following dependency to your Package.swift file:

dependencies: [
    .package(url: "https://github.com/tattn/LocalLLMClient.git", branch: "main")
]

Usage

The API documentation is available here.

Quick Start

import LocalLLMClient
import LocalLLMClientLlama
let session = LLMSession(model: .llama(
    id: "lmstudio-community/gemma-3-4B-it-qat-GGUF",
    model: "gemma-3-4B-it-QAT-Q4_0.gguf"
))

print(try await session.

…

View on GitHub

Pull requests are very welcome!

If you like it, a ⭐ would be appreciated too!

My favorite ❤️ Swift extensions

Tatsuya Tanaka — Mon, 18 Dec 2017 16:50:09 +0000

This is a collection of Swift's convenience extensions I always use. Protocols and operators are also included.

All of the extensions are in the following repository.
https://github.com/tattn/SwiftExtensions

The version of Swift is 4.0.
This is a translated version of the article I wrote in Japanese here.

Get a class name

public protocol ClassNameProtocol {
    static var className: String { get }
    var className: String { get }
}

public extension ClassNameProtocol {
    public static var className: String {
        return String(describing: self)
    }

    public var className: String {
        return type(of: self).className
    }
}

extension NSObject: ClassNameProtocol {}

UIView.className   //=> "UIView"
UILabel().className //=> "UILabel"

Register / Dequeue XIB

UITableView

public extension UITableView {
    public func register<T: UITableViewCell>(cellType: T.Type, bundle: Bundle? = nil) {
        let className = cellType.className
        let nib = UINib(nibName: className, bundle: bundle)
        register(nib, forCellReuseIdentifier: className)
    }

    public func register<T: UITableViewCell>(cellTypes: [T.Type], bundle: Bundle? = nil) {
        cellTypes.forEach { register(cellType: $0, bundle: bundle) }
    }

    public func dequeueReusableCell<T: UITableViewCell>(with type: T.Type, for indexPath: IndexPath) -> T {
        return self.dequeueReusableCell(withIdentifier: type.className, for: indexPath) as! T
    }
}

tableView.register(cellType: MyCell.self)
tableView.register(cellTypes: [MyCell1.self, MyCell2.self])

let cell = tableView.dequeueReusableCell(with: MyCell.self, for: indexPath)

UICollectionView

public extension UICollectionView {
    public func register<T: UICollectionViewCell>(cellType: T.Type, bundle: Bundle? = nil) {
        let className = cellType.className
        let nib = UINib(nibName: className, bundle: bundle)
        register(nib, forCellWithReuseIdentifier: className)
    }

    public func register<T: UICollectionViewCell>(cellTypes: [T.Type], bundle: Bundle? = nil) {
        cellTypes.forEach { register(cellType: $0, bundle: bundle) }
    }

    public func register<T: UICollectionReusableView>(reusableViewType: T.Type,
                                                      ofKind kind: String = UICollectionElementKindSectionHeader,
                                                      bundle: Bundle? = nil) {
        let className = reusableViewType.className
        let nib = UINib(nibName: className, bundle: bundle)
        register(nib, forSupplementaryViewOfKind: kind, withReuseIdentifier: className)
    }

    public func register<T: UICollectionReusableView>(reusableViewTypes: [T.Type],
                                                      ofKind kind: String = UICollectionElementKindSectionHeader,
                                                      bundle: Bundle? = nil) {
        reusableViewTypes.forEach { register(reusableViewType: $0, ofKind: kind, bundle: bundle) }
    }

    public func dequeueReusableCell<T: UICollectionViewCell>(with type: T.Type,
                                                             for indexPath: IndexPath) -> T {
        return dequeueReusableCell(withReuseIdentifier: type.className, for: indexPath) as! T
    }

    public func dequeueReusableView<T: UICollectionReusableView>(with type: T.Type,
                                                                 for indexPath: IndexPath,
                                                                 ofKind kind: String = UICollectionElementKindSectionHeader) -> T {
        return dequeueReusableSupplementaryView(ofKind: kind, withReuseIdentifier: type.className, for: indexPath) as! T
    }
}

collectionView.register(cellType: MyCell.self)
collectionView.register(cellTypes: [MyCell1.self, MyCell2.self])
let cell = collectionView.dequeueReusableCell(with: MyCell.self, for: indexPath)

collectionView.register(reusableViewType: MyReusableView.self)
collectionView.register(reusableViewTypes: [MyReusableView1.self, MyReusableView2.self])
let view = collectionView.dequeueReusableView(with: MyReusableView.self, for: indexPath)

Instantiate a viewController in a storyboard

public enum StoryboardInstantiateType {
    case initial
    case identifier(String)
}

public protocol StoryboardInstantiatable {
    static var storyboardName: String { get }
    static var storyboardBundle: Bundle { get }
    static var instantiateType: StoryboardInstantiateType { get }
}

public extension StoryboardInstantiatable where Self: NSObject {
    public static var storyboardName: String {
        return className
    }

    public static var storyboardBundle: Bundle {
        return Bundle(for: self)
    }

    private static var storyboard: UIStoryboard {
        return UIStoryboard(name: storyboardName, bundle: storyboardBundle)
    }

    public static var instantiateType: StoryboardInstantiateType {
        return .identifier(className)
    }
}

public extension StoryboardInstantiatable where Self: UIViewController {
    public static func instantiate() -> Self {
        switch instantiateType {
        case .initial:
            return storyboard.instantiateInitialViewController() as! Self
        case .identifier(let identifier):
            return storyboard.instantiateViewController(withIdentifier: identifier) as! Self
        }
    }
}

// Class name and Storyboard name, same as Storyboard ID
final class ViewController: UIViewController, StoryboardInstantiatable {
}

ViewController.instantiate()

// Initial view controller & class name and Storyboard name are the same
final class InitialViewController: UIViewController, StoryboardInstantiatable {
    static var instantiateType: StoryboardInstantiateType {
        return .initial
    }
}

InitialViewController.instantiate()

Instantiate a view in XIB

public protocol NibInstantiatable {
    static var nibName: String { get }
    static var nibBundle: Bundle { get }
    static var nibOwner: Any? { get }
    static var nibOptions: [AnyHashable: Any]? { get }
    static var instantiateIndex: Int { get }
}

public extension NibInstantiatable where Self: NSObject {
    public static var nibName: String { return className }
    public static var nibBundle: Bundle { return Bundle(for: self) }
    public static var nibOwner: Any? { return self }
    public static var nibOptions: [AnyHashable: Any]? { return nil }
    public static var instantiateIndex: Int { return 0 }
}

public extension NibInstantiatable where Self: UIView {
    public static func instantiate() -> Self {
        let nib = UINib(nibName: nibName, bundle: nibBundle)
        return nib.instantiate(withOwner: nibOwner, options: nibOptions)[instantiateIndex] as! Self
    }
}

// XIB name and class name are the same & 0th View
final class View: UIView, NibInstantiatable {
}

View.instantiate()

// XIB name and class name are different & & 2nd View
final class View2: UIView, NibInstantiatable {
    static var nibName: String { return "Foo" } // Foo.xib
    static var instantiateIndex: Int { return 2 }
}

View2.instantiate()

Put a custom view created by Xib in Interface Builder

public protocol EmbeddedNibInstantiatable {
    associatedtype Embedded: NibInstantiatable
}

public extension EmbeddedNibInstantiatable where Self: UIView, Embedded: UIView {
    public var embedded: Embedded { return subviews[0] as! Embedded }

    public func configureEmbededView() {
        let view = Embedded.instantiate()
        insertSubview(view, at: 0)
        view.fillSuperview() // described later
    }
}

final class EmbeddedView: UIView, NibInstantiatable {
}

@IBDesignable
final class IBEmbeddedView: UIView, EmbeddedNibInstantiatable {
    typealias Embedded = EmbeddedView

    #if TARGET_INTERFACE_BUILDER
    override func prepareForInterfaceBuilder() {
        super.prepareForInterfaceBuilder()
        configureEmbededView()
    }
    #endif

    override func awakeFromNib() {
        super.awakeFromNib()
        configureEmbededView()
    }
}

Make a view the same size as the superview

public extension UIView {
    public func fillSuperview() {
        guard let superview = self.superview else { return }
        translatesAutoresizingMaskIntoConstraints = superview.translatesAutoresizingMaskIntoConstraints
        if translatesAutoresizingMaskIntoConstraints {
            autoresizingMask = [.flexibleWidth, .flexibleHeight]
            frame = superview.bounds
        } else {
            topAnchor.constraint(equalTo: superview.topAnchor).isActive = true
            bottomAnchor.constraint(equalTo: superview.bottomAnchor).isActive = true
            leftAnchor.constraint(equalTo: superview.leftAnchor).isActive = true
            rightAnchor.constraint(equalTo: superview.rightAnchor).isActive = true
        }
    }
}

superView.addSubview(view)
view.fillSuperview()

Get the top most UIViewController / UINavigationController

public extension UIApplication {
    public var topViewController: UIViewController? {
        guard var topViewController = UIApplication.shared.keyWindow?.rootViewController else { return nil }

        while let presentedViewController = topViewController.presentedViewController {
            topViewController = presentedViewController
        }
        return topViewController
    }

    public var topNavigationController: UINavigationController? {
        return topViewController as? UINavigationController
    }
}

UIApplication.shared.topViewController

Get a view controller which has the view

public extension UIView {
    public var viewController: UIViewController? {
        var parent: UIResponder? = self
        while parent != nil {
            parent = parent?.next
            if let viewController = parent as? UIViewController {
                return viewController
            }
        }
        return nil
    }
}

view.viewController

Create a UIColor by hexadecimal

public extension UIColor {
    public convenience init(hex: Int, alpha: CGFloat = 1.0) {
        let r = CGFloat((hex & 0xFF0000) >> 16) / 255.0
        let g = CGFloat((hex & 0x00FF00) >> 8) / 255.0
        let b = CGFloat(hex & 0x0000FF) / 255.0
        self.init(red: r, green: g, blue: b, alpha: alpha)
    }
}

let color = UIColor(hex: 0xAABBCC)

Remove elements by the instances in an array

public extension Array where Element: Equatable {
    @discardableResult
    public mutating func remove(element: Element) -> Index? {
        guard let index = index(of: element) else { return nil }
        remove(at: index)
        return index
    }

    @discardableResult
    public mutating func remove(elements: [Element]) -> [Index] {
        return elements.flatMap { remove(element: $0) }
    }
}

let array = ["foo", "bar"]
array.remove(element: "foo")
array //=> ["bar"]

Remove the same value from the array

public extension Array where Element: Hashable {
    public mutating func unify() {
        self = unified()
    }
}

public extension Collection where Element: Hashable {
    public func unified() -> [Element] {
        return Array(Set(self))
    }
}

let array = [1, 2, 3, 3, 2, 1, 4]
array.unify() // [1, 2, 3, 4]

Get an element in safety.

public extension Collection {
    public subscript(safe index: Index) -> Element? {
        return startIndex <= index && index < endIndex ? self[index] : nil
    }
}

let array = [0, 1, 2]
if let item = array[safe: 5] {
    print("unreachable")
}

Get substrings with various ranges

public extension String {
    subscript (bounds: CountableClosedRange<Int>) -> String {
        let start = index(startIndex, offsetBy: bounds.lowerBound)
        let end = index(startIndex, offsetBy: bounds.upperBound)
        return String(self[start...end])
    }

    subscript (bounds: CountableRange<Int>) -> String {
        let start = index(startIndex, offsetBy: bounds.lowerBound)
        let end = index(startIndex, offsetBy: bounds.upperBound)
        return String(self[start..<end])
    }

    subscript (bounds: PartialRangeUpTo<Int>) -> String {
        let end = index(startIndex, offsetBy: bounds.upperBound)
        return String(self[startIndex..<end])
    }

    subscript (bounds: PartialRangeThrough<Int>) -> String {
        let end = index(startIndex, offsetBy: bounds.upperBound)
        return String(self[startIndex...end])
    }

    subscript (bounds: CountablePartialRangeFrom<Int>) -> String {
        let start = index(startIndex, offsetBy: bounds.lowerBound)
        return String(self[start..<endIndex])
    }
}

let string = "0123456789"
string[0...5] //=> "012345"
string[1...3] //=> "123"
string[3..<7] //=> "3456"
string[...4]  //=> "01234
string[..<4]  //=> "0123"
string[4...]  //=> "456789"

Create URL from a string / Create URL from a string literal

public extension String {
    public var url: URL? {
        return URL(string: self)
    }
}

extension URL: ExpressibleByStringLiteral {
    public init(stringLiteral value: String) {
        guard let url = URL(string: value) else {
            fatalError("\(value) is an invalid url")
        }
        self = url
    }

    public init(extendedGraphemeClusterLiteral value: String) {
        self.init(stringLiteral: value)
    }

    public init(unicodeScalarLiteral value: String) {
        self.init(stringLiteral: value)
    }
}

if let url = "https://example.com".url {
}

let url: URL = "https://example.com"

Use Kotlin-like scope function

public protocol Appliable {}

public extension Appliable {
    @discardableResult
    public func apply(closure: (Self) -> Void) -> Self {
        closure(self)
        return self
    }
}

public protocol Runnable {}

public extension Runnable {
    @discardableResult
    public func run<T>(closure: (Self) -> T) -> T {
        return closure(self)
    }
}

extension NSObject: Appliable {}
extension NSObject: Runnable {}

let view = UIView().apply {
    $0.backgroundColor = .red
    $0.frame = .init(x: 0, y: 0, width: 200, height: 200)
}

Throw an error by `???` if the value is nil

infix operator ???

public func ???<T>(lhs: T?,
                   error: @autoclosure () -> Error) throws -> T {
    guard let value = lhs else { throw error() }
    return value
}

let value: String? = nil

struct OptionalError: Error {}

do {
    let v = try value ??? OptionalError()
    print(v) // unreachable
} catch {
    print(error) //=> OptionalError
}

Throw an error if there is no key when getting the value from a dictionary

public struct DictionaryTryValueError: Error {
    public init() {}
}

public extension Dictionary {
    func tryValue(forKey key: Key, error: Error = DictionaryTryValueError()) throws -> Value {
        guard let value = self[key] else { throw error }
        return value
    }
}

var dictionary: [String: Int] = [:]
do {
    let value = try dictionary.tryValue(forKey: "foo")
    print(value) // unreachable
} catch {
    print(error) //=> DictionaryTryValueError
}

Create properties and methods accessible with `.ex`

public struct TargetedExtension<Base> {
    let base: Base
    init (_ base: Base) {
        self.base = base
    }
}

public protocol TargetedExtensionCompatible {
    associatedtype Compatible
    static var ex: TargetedExtension<Compatible>.Type { get }
    var ex: TargetedExtension<Compatible> { get }
}

public extension TargetedExtensionCompatible {
    public static var ex: TargetedExtension<Self>.Type {
        return TargetedExtension<Self>.self
    }

    public var ex: TargetedExtension<Self> {
        return TargetedExtension(self)
    }
}

extension UIView: TargetedExtensionCompatible {}

private extension TargetedExtension where Base: UIView {
    func foo() {}
}

UIView().ex.foo()

Forem: Tatsuya Tanaka

LocalLLMClient: A Swift Package for Local LLMs Using llama.cpp and MLX

LocalLLMClient: A Swift package for Local LLMs

tattn / LocalLLMClient

Swift package to run local LLMs on iOS, macOS, Linux

LocalLLMClient

Features

Installation

Usage

Quick Start

What LocalLLMClient Can Do

How to Use It

Text Generation

Multimodal

Additional Features

Conclusion

tattn / LocalLLMClient

Swift package to run local LLMs on iOS, macOS, Linux

LocalLLMClient

Features

Installation

Usage

Quick Start

My favorite ❤️ Swift extensions

Get a class name

Register / Dequeue XIB

UITableView

UICollectionView

Instantiate a viewController in a storyboard

Instantiate a view in XIB

Put a custom view created by Xib in Interface Builder

Make a view the same size as the superview

Get the top most UIViewController / UINavigationController

Get a view controller which has the view

Create a UIColor by hexadecimal

Remove elements by the instances in an array

Remove the same value from the array

Get an element in safety.

Get substrings with various ranges

Create URL from a string / Create URL from a string literal

Use Kotlin-like scope function

Throw an error by ??? if the value is nil

Throw an error if there is no key when getting the value from a dictionary

Create properties and methods accessible with .ex

Throw an error by `???` if the value is nil

Create properties and methods accessible with `.ex`