Secret Llama

This is absolutely wonderful, I am a HUGE fan of local first apps. Running models locally is such a powerful thing I wish more companies could leverage it to build smarter apps…

This is amazing! I always wanted something like this, thank you so much!

Amazing! It's surprisingly fast to load and run given the size of the downloaded models. Do you think it would be feasible to extend it to support web browsing? I'd like to help…

Nice demo! I briefly tried it out and the demo felt much better than the original WebLLM one! On a side note, i've been trying to do something similar too for similar reasons…

After the model is supposedly fully downloaded (about 4GB) I get: Could not load the model because Error: ArtifactIndexedDBCache failed to fetch:…

Tested on Ubuntu 22.04 with Chrome, sure enough, "Could not load the model because Error: Cannot find adapter that matches the request". It really is too bad WebGPU isn't…

On Firefox Nightly on my Steam Deck it "cannot find WebGPU in the environment".

Could not load the model because Error: Cannot find WebGPU in the environment

Is this downloading a ~5gb model to my machine and storing it locally for subsequent use?

It's great but I hope it don't catch on because then every website will make me download models. My hard drive will be full, too much bloat. I think the web is not good for this.…

Phi 1.5 isn't quite there is it? Caught it having some inner dialog. What does that mean? Instruct: I'm sorry, but I'm not sure what you mean by "wutt". Output: Oh, I'm sorry. I…

Llama 3 seems to not be there yet at all. Can it read earlier prompts and what you have written before? It doesn't seem like it. I asked it to help come up with a song name for a…

Very cool! I wish there was chat history. Also if you click the "New Chat" button while an answer is generating I think some of the output gets fed back into the model, it causes…

Amazing! It's surprisingly fast to load and run given the size of the downloaded models. Do you think it would be feasible to extend it to support web browsing? I'd like to help…

Could we use an already downloaded .gguf file?

It's great but I hope it don't catch on because then every website will make me download models. My hard drive will be full, too much bloat. I think the web is not good for this.…

This is amazing but can we please set the .prose width to be dynamic? the text column in 3 inches wide on my monitor, it should take up a % of the browser window.

How do people use something like this as coach or therapist? This is genuine question. Side note, impressive project. Future of AI is offline mostly with few APIs in the cloud…

What therapy prompts have you found useful?

Question - Do I compromise on quality on answers if I use models using WebLLM (like this) compare to using them on system console.

This is very cool, it's something I wish existed since Llama came out, having to install Ollama + Cuda to get locally working LLM didn't felt right to me when there's all what's…

Likewise (same error) with Chrome on Windows. Currently running Ollama / Open WebUI and finding lama3:8B quite useful for writing snippets of powershell, javascript, golang etc.

I've managed to avoid ollama and just toyed with lmstudio. It's non-free software, but extremely easy to get into, uses llama.cpp under the hood, cross-platform, yada yada.…

Window AI (https://windowai.io/) is an attempt to do something like this with a browser extension.

This is very cool, it's something I wish existed since Llama came out, having to install Ollama + Cuda to get locally working LLM didn't felt right to me when there's all what's…