Unsloth
AI
llm finetuning
SHOW HN2023
How HN took it
4 positive, 21 neutral or asking, 0 mixed, 5 critical, out of 30 comments about the project. Thread score +0.03.
Pricing: 1 for, 3 againstPerformance: 0 for, 1 againstIdea: 1 for, 0 againstDesign: 1 for, 0 againstExecution: 0 for, 1 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Interesting results and excitement to try it.
From 2 comments
promising results, excited to try it out! question on the perf benchmarks: why do all the results with 2 GPUs & DDP take longer than the single GPU case? Both benchmarks do the…
neilmovva on HNwow i wish i could do this with all my M1 pro max neural cores
TuringNYC on HN
Maker is refreshingly open about monetization.
From 1 comment
Refreshing to see, especially on this forum! Best of luck.
ShamelessC on HN
Objections
Multi-GPU benchmarks look slower than single GPU, questioning the test.
From 2 comments
I agree that synchronization causes overhead, so 2x GPUs won't achieve the ideal 0.5x total runtime. But here, taking your Alpaca benchmark as an example, we are seeing 2x GPUs…
neilmovva on HNpromising results, excited to try it out! question on the perf benchmarks: why do all the results with 2 GPUs & DDP take longer than the single GPU case? Both benchmarks do the…
neilmovva on HN
Paid tier gating of the fastest version seems odd.
From 1 comment
This seems very interesting, but I’m very confused why you have gated the /max/imum speedup version for enterprise-only? It would make more sense to have only the Free and Paid…
yunohn on HN
Open-core model seen as shareware-like.
From 1 comment
Shareware. See the open source preview, pay a million dollars for the full version.
sbierwagen on HN
Requests
Support for older GPUs such as 1080 Ti and P100.
From 2 comments
I see this mentions 2018+ GPUs, but I'm curious why this wouldn't work on say a 1080Ti. My cursory look at the hardware specs shows this has support for CUDA8+ and this states…
0x6c6f6c on HNSomewhat related: is it worth it to use a P100 or P40? I was gonna get one but seems like pascal isn't being supported by more and more projects.
graphe on HN
Apple M1 support.
From 2 comments
wow i wish i could do this with all my M1 pro max neural cores
TuringNYC on HNM1 support is probably coming in the future if there is enough support - could you make an Issue at https://github.com/unslothai/unsloth - that would be much appreciated! I think…
danielhanchen on HN
Share benchmark code.
From 1 comment
I agree that synchronization causes overhead, so 2x GPUs won't achieve the ideal 0.5x total runtime. But here, taking your Alpaca benchmark as an example, we are seeing 2x GPUs…
neilmovva on HN
Questions
How does it compare to PyTorch inference optimizations?
From 1 comment
How does this compare to PyTorch labs optimizations for Sam and llama2 ? https://github.com/pytorch-labs/segment-anything-fast https://github.com/pytorch-labs/gpt-fast
naveen99 on HN
What are minimum hardware requirements?
From 1 comment
Any source recommendations on howto find out the minimum hw requirementsto train LLM with Xbit and YB parameters?
3abiton on HN
Alternatives named
Hugging Face's original training implementation.
From 1 comment
Hi HN! I'm just sharing a project I've been working on during the LLM Efficiency Challenge - you can now finetune Llama with QLoRA 5x faster than Huggingface's original…
danielhanchen on HN
PyTorch gpt-fast and segment-anything-fast (inference only).
From 1 comment
How does this compare to PyTorch labs optimizations for Sam and llama2 ? https://github.com/pytorch-labs/segment-anything-fast https://github.com/pytorch-labs/gpt-fast
naveen99 on HN