Cactus
AI
on-device llm framework
SHOW HN2025
How HN took it
11 positive, 39 neutral or asking, 1 mixed, 12 critical, out of 63 comments about the project. Thread score +0.04.
Idea: 3 for, 4 againstDesign: 2 for, 2 againstPerformance: 3 for, 1 againstExecution: 0 for, 2 againstDocs: 0 for, 2 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Simple, clean API that is easy to integrate
From 1 comment
This is actually crazy. The API is so simple! I tried to do this on Swift using LLM.swift and it went okay, excited to try this on RN
awaseem on HN
Good performance on phones, fast local responses
From 2 comments
I've installed the Android version from https://play.google.com/store/apps/details?id=com.rshemetsubuser.myapp It is fantastic. Compared to another program I had installed a year…
nunobrito on HNAmazing, this is so so useful. Thank you especially for the phone model vs tok/s breakdown. Do you have such tables for more models? For models even leaner than Gemma3 1B. How…
pj_mukh on HN
Objections
Wraps llama.cpp, so the framework is thin and the 'Ollama for phones' framing is questioned
From 2 comments
They literally vendored llama.cpp and they STILL called it "Ollama for *". Georgi cannot be vindicated hard enough.
max-privatevoid on HN[flagged]
refulgentis on HN
Model sharing across apps not yet possible; each app downloads its own copy
From 1 comment
Great question. Currently, each app is sandboxed - so each model file is downloaded inside each app's sandbox. We're working on enabling file sharing across multiple apps so you…
rshemet on HN
Demo app lacks dark mode and crashes on some custom GGUF models
From 1 comment
This is a cool app! I'm happy to play with it. Some feedback: 1. The lack of a dark mode is an accessibility issue for me. I have a genetic condition that causes severe light…
pxc on HN
Requests
Dark mode and better contrast in the demo app
From 1 comment
This is a cool app! I'm happy to play with it. Some feedback: 1. The lack of a dark mode is an accessibility issue for me. I have a genetic condition that causes severe light…
pxc on HN
Support for GPU/NPU acceleration on Android
From 2 comments
Very nice, good work. I think you should add the chat app links on the readme, so that visitors get a good idea of what the framework is capable of. The performance is quite…
politelemon on HNIs this using only llama.cpp as inference engine? How is this days support there on NPU and GPU? Not sure if LLM can run on NPU but many models like STT and TTS and vision often…
pzo on HN
Download API separate from model initialization
From 1 comment
Does this download models at runtime? I would have expected a different API for that. I understand that you don’t want to include a multi-gig model in your app. But the mobile…
teaearlgraycold on HN
APK that does not require Google Play services
From 1 comment
appreciate if you can provide a apk that does not require google play services to run...
ekianjo on HN
Questions
How does model sharing and serving to other apps work
From 2 comments
How does it work? How does one model on the device get shared to many apps? Does each app have it's own inference sdk running or is there one inference engine shared to many apps…
jadbox on HNHow does writing a model into an App Group container enable your framework to enable an app to enable a local LLM server that 3rd party apps can make calls to on iOS?[^1] How…
refulgentis on HN
How does it compare to existing apps like ChatterUI
From 1 comment
I understand this is targetted towarda devwlopers, bur Can someone explain why should I go through this complex install process instead of just using ChatterUI? It can handle…
wanderingmind on HN
Alternatives named
Ollama
From 2 comments
Ollama runs on Android just fine via Termux. I use it with 5GB models. They even recently added ollama package, there is no longer need to compile it from source code.
throw777373 on HNThey literally vendored llama.cpp and they STILL called it "Ollama for *". Georgi cannot be vindicated hard enough.
max-privatevoid on HN
ChatterUI
From 1 comment
I understand this is targetted towarda devwlopers, bur Can someone explain why should I go through this complex install process instead of just using ChatterUI? It can handle…
wanderingmind on HN
Google AI Edge / MediaPipe
From 2 comments
[flagged]
deepdarkforest on HNThanks for the feedback. You're right to point out that Google AI Edge is cross-platform and more flexible than our phrasing suggested. The core distinction is in the ecosystem:…
rshemet on HN
llama.cpp
From 1 comment
didn't Ollama vendor Llama cpp too? Most projects typically start with llama.cpp and then move away to proprietary kernels
rshemet on HN