[{"data":1,"prerenderedAt":482},["ShallowReactive",2],{"project-35256769":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":29,"sentiment":30,"stance_mix":31,"scored_comments":36,"aspects":37,"strip":44,"post_text":46,"what":47,"said":90,"maker_in_thread":161,"shape":173,"grid":177,"attempts":445,"same_week":446,"maker_shows":45,"maker_hidden":474,"reader_version":475,"card":476},"35256769","simple-llama-finetuner","Finetune LLaMA-7B on commodity GPUs using your own text","title","Show HN: Finetune LLaMA-7B on commodity GPUs using your own text","https:\u002F\u002Fgithub.com\u002Flxe\u002Fsimple-llama-finetuner","github.com","lxe",1679458529,2023,449,98,66,3109,3,"legend","ai","model finetuning tool","web_app",[24,25,26,27,28],"llama finetuning","lora","peft","gradio","llm training","loved",0.263,{"positive":32,"neutral":33,"mixed":34,"critical":35},7,10,0,1,18,{"idea":38,"performance":40,"design":41,"execution":42,"docs":43},{"positive":39,"critical":35},5,{"positive":35,"critical":34},{"positive":34,"critical":34},{"positive":34,"critical":34},{"positive":34,"critical":34},[34,34,34,34,34,34,34,34,34,34,34,39,34,18,39,34,34,34,34,34,34,34,34,34,34,34,18,18,39,18,34,34,34,39,18,45,18,18,34,34],4,"I've been playing around with https:\u002F\u002Fgithub.com\u002Fzphang\u002Fminimal-llama\u002F and https:\u002F\u002Fgithub.com\u002Ftloen\u002Falpaca-lora\u002Fblob\u002Fmain\u002Ffinetune.py, and wanted to create a simple UI where you can just paste text, tweak the parameters, and finetune the model quickly using a modern GPU.\n\nTo prepare the data, simply separate your text with two blank lines.\n\nThere's an inference tab, so you can test how the tuned model behaves.\n\nThis is my first foray into the world of LLM finetuning, Python, Torch, Transformers, LoRA, PEFT, and Gradio.\n\nEnjoy!",{"problem":48,"audience":54,"pricing":54,"motivation":55,"time_to_build":54,"stack":62,"openness":87,"stage":88,"team_size":89},{"value":49,"source":50,"evidence":51},"Finetuning LLaMA requires scripts and setup; this offers a simple UI to paste text and tune","maker",[52],{"id":53},"post",null,{"value":56,"source":50,"evidence":57},"Maker was learning LLM finetuning and wanted a simple UI for quick experiments",[58,59],{"id":53},{"id":60,"by":11,"text":61},"35257291","> I can't help but question the actual value of such a tool. Is it just an attempt to dumb down the process for newcomers? Actually, you've hit the nail on the head here. I…",[63,67,71,75,79,83],{"value":64,"source":50,"evidence":65},"Python",[66],{"id":53},{"value":68,"source":50,"evidence":69},"PyTorch",[70],{"id":53},{"value":72,"source":50,"evidence":73},"Transformers",[74],{"id":53},{"value":76,"source":50,"evidence":77},"LoRA",[78],{"id":53},{"value":80,"source":50,"evidence":81},"PEFT",[82],{"id":53},{"value":84,"source":50,"evidence":85},"Gradio",[86],{"id":53},"open_source","prototype","unknown",{"praise":91,"objections":104,"requests":123,"questions":135,"alternatives":150},[92],{"value":93,"source":94,"evidence":95},"Useful beginner-friendly project","commenter",[96,100],{"id":97,"by":98,"text":99},"35261893","mattfrommars","Great job OP. Yesterday after managing to run local instance of alpaca.cpp and reading more about what alpaca is and how it got fine tuned on LLaMA, I began to wonder what will…",{"id":101,"by":102,"text":103},"35257654","Tepix","Very interesting project, thanks. I see that DeepSpeed is on your TODO list. I wonder what the biggest LLaMA model is that can be fine-tuned on 2x RTX 3090 & 128GB RAM.",[105,112],{"value":106,"source":94,"evidence":107},"Questions the value of a tool that dumbs down finetuning",[108],{"id":109,"by":110,"text":111},"35257245","yousnail","my instance doesn't seem impressed: So, I stumbled upon this Simple LLaMA FineTuner project by Aleksey Smolenchuk, claiming to be a beginner-friendly tool for fine-tuning the…",{"value":113,"source":94,"evidence":114},"Unclear what training screenshots show",[115,119],{"id":116,"by":117,"text":118},"35257101","antman","What do the training screenshot examples try to accomplish? In what way would the model be different after the example fine-tuning?",{"id":120,"by":121,"text":122},"35257116","rafaguin","Same question here!",[124,128],{"value":125,"source":94,"evidence":126},"Support for DeepSpeed",[127],{"id":101,"by":102,"text":103},{"value":129,"source":94,"evidence":130},"Run on smaller GPUs like a 3080 with 16GB VRAM",[131],{"id":132,"by":133,"text":134},"35258186","reissbaker","This is awesome! I noticed it said a prereq is >16GB VRAM — is that >= 16GB, or is it really explicitly greater than 16? Would be sweet to be able to finetune locally on, say, a…",[136,143],{"value":137,"source":94,"evidence":138},"How much VRAM is needed to finetune the 65B model",[139],{"id":140,"by":141,"text":142},"35257190","Taek","What would it take to fine tune the 65B?",{"value":144,"source":94,"evidence":145},"Can it be finetuned on CPU only",[146],{"id":147,"by":148,"text":149},"35257021","sea_temple","Is it possible to fine-tune using CPU only?",[151,155],{"value":152,"source":94,"evidence":153},"Alpaca-LoRA",[154],{"id":109,"by":110,"text":111},{"value":156,"source":50,"evidence":157},"text-generation-webui",[158],{"id":159,"by":11,"text":160},"35262800","> With no real knowledge of LLM and only recently started to understand what LLM terms mean, such as 'model, inference, LLM model, intruction set, fine tuning' whatelse do you…",{"comments":162,"answered":33,"key":163},12,[164,167,170],{"id":165,"text":166},"35261196","So hard I’ve been using lambda labs, vast.ai, and runpod to rent machines. A 3090 is about 30 cents an hour.",{"id":168,"text":169},"35268458","If there's context limitations (your document cant' fit in the context, or your session quickly loses the context), fine-tuning is a great option.",{"id":171,"text":172},"35257203","Lots of VRAM, but the method is the same. Here's someone who finetuned llama-30b on alpaca dataset for example: https:\u002F\u002Fgithub.com\u002Fdeep-diver\u002FAlpaca-LoRA-Serve",{"commenters":174,"max_depth":32,"maker_replies":162,"first_comment_mins":175,"lifespan_hours":176},64,46,67,[178,182,185,188,191,195,198,200,201,203,205,207,210,213,217,221,224,226,229,232,236,239,240,244,247,250,253,256,260,262,265,268,271,274,276,279,281,283,284,286,288,291,294,297,300,303,306,308,311,314,317,320,323,325,328,330,332,334,338,341,343,347,350,353,356,358,360,361,364,367,368,370,373,377,379,381,383,386,389,392,394,396,399,403,407,410,414,418,420,422,425,428,430,432,435,437,440,442],[179,34,34,180,181,34],"35257901","other\u002Fneutral\u002Fnone","syntaxing",[183,34,34,180,184,34],"35258318","speedgoose",[186,34,34,180,187,34],"35258545","te_chris",[189,34,34,180,190,34],"35259026","tysam_and",[192,34,34,193,194,34],"35259406","other\u002Fneutral\u002Fpricing","nwoli",[196,34,34,193,197,34],"35260105","capableweb",[199,34,34,193,190,34],"35268775",[165,34,34,193,11,35],[202,34,34,180,187,34],"35261640",[204,34,34,180,190,34],"35268799",[206,34,34,180,187,34],"35271861",[208,34,34,193,209,34],"35267491","tempaccount420",[211,34,34,193,212,34],"35269058","qeternity",[214,34,34,215,216,34],"35270399","other\u002Fneutral\u002Fprivacy","flemhans",[218,35,34,219,220,34],"35263883","other\u002Fneutral\u002Fidea","spunker540",[222,35,34,180,223,34],"35263989","wazer5",[225,35,34,219,220,34],"35264421",[227,35,34,219,228,34],"35265826","ttoinou",[230,35,34,219,231,34],"35267075","underlines",[233,35,34,234,235,34],"35296744","other\u002Fneutral\u002Fdocs","curl-up",[237,35,34,219,238,34],"35265247","stu2b50",[168,35,34,219,11,35],[241,242,34,219,243,34],"35258153",2,"callesgg",[245,242,34,219,246,34],"35258217","jointpdf",[248,242,34,219,249,34],"35258346","Drakim",[251,242,34,180,252,34],"35258506","Dwedit",[254,242,34,180,255,34],"35258679","MattPalmer1086",[257,242,39,258,259,34],"35258769","project\u002Fpositive\u002Fidea","newswasboring",[261,242,34,219,243,34],"35260030",[263,242,34,219,264,34],"35260461","aqme28",[266,242,34,267,190,34],"35259046","other\u002Fcritical\u002Fidea",[269,242,34,219,270,34],"35260211","8jy89hui",[272,242,34,219,273,34],"35260341","jkeisling",[132,18,18,275,133,34],"project\u002Fneutral\u002Fperformance",[277,18,39,278,197,34],"35261033","project\u002Fpositive\u002Fperformance",[280,18,18,275,197,34],"35263502",[140,45,34,282,141,34],"other\u002Fneutral\u002Fperformance",[171,45,34,180,11,35],[285,45,34,282,11,35],"35257230",[287,45,34,282,141,34],"35257320",[289,45,34,282,290,34],"35257471","arpowers",[292,45,34,282,293,34],"35258242","ioedward",[295,45,34,282,296,34],"35258531","albertzeyer",[298,45,34,282,299,34],"35258959","joshxyz",[301,39,34,219,302,34],"35258239","Hedepig",[304,39,34,180,305,34],"35258365","Phemist",[307,39,34,180,302,34],"35258402",[309,39,34,180,310,34],"35258688","NavinF",[312,39,34,180,313,34],"35258788","YetAnotherNick",[315,39,34,180,316,34],"35262442","throwaway1851",[318,39,34,180,319,34],"35275606","satvikpendem",[321,39,34,180,322,34],"35259665","amrb",[324,39,34,180,305,34],"35259279",[326,39,34,180,327,34],"35258470","Metus",[329,39,34,180,302,34],"35258536",[331,39,34,180,322,34],"35259591",[333,39,34,180,11,35],"35268983",[335,336,34,180,337,34],"35258435",6,"holoduke",[339,336,34,234,340,34],"35258507","Method-X",[342,336,34,234,11,35],"35268977",[344,336,34,345,346,34],"35258821","commenter\u002Fneutral\u002Fnone","benob",[348,336,34,345,349,34],"35258923","wsgeorge",[351,336,34,345,352,34],"35260377","inportb",[354,336,34,345,355,34],"35260501","Closi",[357,336,34,180,337,34],"35271815",[116,32,18,359,117,34],"project\u002Fneutral\u002Fdesign",[120,32,18,359,121,34],[362,32,34,363,11,35],"35257147","maker\u002Fneutral\u002Fnone",[97,365,39,366,98,34],8,"maker\u002Fpositive\u002Fnone",[159,365,34,363,11,35],[101,369,39,258,102,34],9,[371,369,34,372,11,35],"35257699","maker\u002Fneutral\u002Fperformance",[374,369,18,375,376,34],"35258998","project\u002Fneutral\u002Fexecution","meghan_rain",[378,369,34,345,322,34],"35259582",[380,369,34,345,376,34],"35260206",[382,369,34,345,11,35],"35263382",[384,33,34,345,385,34],"35260007","larodi",[387,33,34,345,388,34],"35260520","selfhoster11",[390,33,34,345,391,34],"35261176","happycube",[393,33,34,345,385,34],"35282102",[395,33,34,345,385,34],"35282121",[397,33,34,345,398,34],"35261295","6gvONxR4sf7o",[400,401,39,258,402,34],"35258106",11,"ALittleLight",[404,162,18,405,406,34],"35260037","project\u002Fneutral\u002Fidea","la64710",[408,162,18,409,197,34],"35261056","project\u002Fneutral\u002Fdocs",[411,412,18,405,413,34],"35267214",13,"akrymski",[415,416,39,258,417,34],"35267025",14,"nathanasmith",[147,419,18,275,148,34],15,[421,419,34,275,11,35],"35257105",[423,419,18,275,424,34],"35257684","kiraaa",[109,426,35,427,110,34],16,"project\u002Fcritical\u002Fidea",[60,426,34,429,11,35],"maker\u002Fneutral\u002Fidea",[431,426,39,258,141,34],"35257333",[433,426,34,345,434,34],"35257442","fbdab103",[436,426,34,345,110,34],"35257616",[438,426,34,345,439,34],"35257828","bbor",[441,426,34,345,110,34],"35258066",[443,426,34,345,444,34],"35259664","bjord",[],[447,451,455,460,464,469],{"id":448,"name":449,"points":450,"category":20},"35184985","Alpaca.cpp",673,{"id":452,"name":453,"points":454,"category":20},"35258553","ChatLLaMA",402,{"id":456,"name":457,"points":458,"category":459},"35191303","GPT Repo Loader",373,"dev_tools",{"id":461,"name":462,"points":463,"category":459},"35223759","Chatblade",315,{"id":465,"name":466,"points":467,"category":468},"35263542","Zapier Natural Language Actions",235,"infrastructure",{"id":470,"name":471,"points":472,"category":473},"35179871","Dadagrams",232,"games",false,"v2",{"id":477,"name":5,"line":6,"ai":474,"cat":20,"year":13,"points":14,"comments":15,"rec":29,"s":478,"top":18,"kw":479,"maker":11,"domain":10,"tier":19,"attempts":34,"strip":480,"url":9,"mix":481,"praise":93,"objection":106,"sub":21},35256769,0.26,[24,25,26,27,28],"0000000000050350000000000033530005343300",[32,33,34,35],1794036632860]