Llama-dl
AI
model weights downloader
SHOW HN2023
How HN took it
9 positive, 46 neutral or asking, 0 mixed, 4 critical, out of 59 comments about the project. Thread score +0.07.
Execution: 4 for, 2 againstPerformance: 4 for, 2 againstIdea: 1 for, 0 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Running LLaMA on a single A100 gives output as good as davinci.
From 1 comment
The output is at least as good as davinci. I think some early results are using bad repetition penalty and/or temperature settings. I had to set both fairly high to get the best…
MacsHeadroom on HN
Writeup and sharing are seen as valuable.
From 2 comments
thanks for doing this, honestly your writeup seems more valuable than the model weights lol > But for what it's worth, my personal opinion is that LLaMA probably isn't…
swyx on HNSeconded. I see vast.ai listing interruptible instance with a single A100 80GB at $1/hour which is pretty reasonable. ChatGPT plus is $20/month which would be roughly 20 hours of…
rnosov on HN
Objections
Cloud GPU cost makes running the 65B model expensive.
From 3 comments
Hmmm, the Github repo suggests that you might be able to run the 65B model on a single A100 80gb card. At the moment, the spot price on Google cloud for this card is $1.25/hour…
rnosov on HN$1.25/hour is roughly a year of GPU time until it exceeds the price of A100 80GB card.
nabla9 on HNI think OP meant that $1.25/hr makes this accessible for people try it out themselves cost effectively, without having to spend thousands or tens of thousands up front to obtain…
metadat on HN
LLaMA is a raw completion model, so comparisons to ChatGPT are unfair.
From 2 comments
Note that unlike ChatGPT, these models are pure text completers and have not been trained to be prompted. The llama FAQ [1] mentions this and gives tips for how to get out of the…
v64 on HN> not really competitive with ChatGPT That's impossible to judge. LLama is a foundational model. It has received neither instructional fine tuning (davinci-3) nor RLHF (ChatGPT).…
minxomat on HN
Downloading weights from a leaked source raises legal and ethical concerns.
From 2 comments
I womder, could Facebook take legal action here? While some (most of) the data used to train the model is copyrighted, I don't think the model is. It's the result of a…
EMIRELADERO on HNAre we celebrating theft from tech companies now?
ahahahahah on HN
Requests
Write up a beginner-friendly guide to running the model.
From 2 comments
Seconded. I see vast.ai listing interruptible instance with a single A100 80GB at $1/hour which is pretty reasonable. ChatGPT plus is $20/month which would be roughly 20 hours of…
rnosov on HNWould you mind publishing your notes/learnings once you gain enough understanding of this model?
linearalgebra45 on HN
Share sampling settings and prompts used.
From 1 comment
That's great to hear. Thank you very much, both for reporting this, and especially for the crucial note about temperature. In fact, sampling settings are so important and so…
sillysaurusx on HN
Support for CPU or lower-memory hardware.
From 1 comment
I would like to support this request for AI challenged developers :) For things like these, I always wonder: How much slower would it be to run such a model on a CPU? I mean,…
davrosthedalek on HN
Questions
How to run the model without ML expertise.
From 1 comment
How or what can someone do with this who isn't a ML expert? Is there some docker app that leverages this? To the average dev, is this useful to me? I know there's lots of "plug…
vanillax on HN
Minimum GPU needed for the smallest model.
From 1 comment
What’s the minimum single GPU that’ll work for the smallest model?
yumraj on HN
How big is the model on disk.
From 1 comment
How big is this model? (i.e. disk space to store it)
arjvik on HN
Alternatives named
llama-int8 for running LLaMA in half the memory.
From 1 comment
If anyone is interested in running this at home, please follow the llama-int8 project [1]. LLM.int8() is a recent development allowing LLMs to run in half the memory without loss…
v64 on HN
llama-cpu for running LLaMA on CPU.
From 1 comment
Eagerly awaiting the int8 vs 4 benchmarks. Also, it can run on CPU https://github.com/markasoftware/llama-cpu So, an 8 bit patch could allow the 65B to run on a standard 128 GB…
downvotetruth on HN