[{"data":1,"prerenderedAt":364},["ShallowReactive",2],{"project-37251969":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":29,"sentiment":30,"stance_mix":31,"scored_comments":36,"aspects":37,"strip":45,"post_text":47,"what":48,"said":75,"maker_in_thread":186,"shape":195,"grid":200,"attempts":328,"same_week":329,"maker_shows":35,"maker_hidden":356,"reader_version":357,"card":358},"37251969","Continue","Use Code Llama as Drop-In Replacement for Copilot Chat","title","Show HN: Use Code Llama as Drop-In Replacement for Copilot Chat","https:\u002F\u002Fcontinue.dev\u002Fdocs\u002Fwalkthroughs\u002Fcodellama","continue.dev","sestinj",1692898392,2023,187,52,302,3109,10,"notable","dev_tools","ai coding assistant","other",[24,25,26,27,28],"code llama","copilot alternative","local llm","ollama","ai code assistant","curious",-0.046,{"positive":32,"neutral":33,"mixed":34,"critical":35},3,13,0,4,20,{"performance":38,"idea":40,"pricing":41,"docs":42,"execution":43},{"positive":39,"critical":39},1,{"positive":39,"critical":34},{"positive":34,"critical":39},{"positive":39,"critical":34},{"positive":34,"critical":44},2,[32,34,34,34,39,46,32,32,34,32,34,34,32,32,34,34,34,34,34,34,34,34,34,34,32,34,32,32,32,39,32,34,32,46,34,34,34,34,34,39],5,"Hi HN,\n\nCode Llama was released, but we noticed a ton of questions in the main thread about how\u002Fwhere to use it — not just from an API or the terminal, but in your own codebase as a drop-in replacement for Copilot Chat. Without this, developers don't get much utility from the model.\n\nThis concern is also important because benchmarks like HumanEval don't perfectly reflect the quality of responses. There's likely to be a flurry of improvements to coding models in the coming months, and rather than relying on the benchmarks to evaluate them, the community will get better feedback from people actu",{"problem":49,"audience":55,"pricing":60,"motivation":66,"time_to_build":70,"stack":71,"openness":72,"stage":73,"team_size":74},{"value":50,"source":51,"evidence":52},"Using Code Llama inside your own codebase as a Copilot Chat replacement","maker",[53],{"id":54},"post",{"value":56,"source":57,"evidence":58},"Developers wanting to use Code Llama in their editor","inferred",[59],{"id":54},{"value":61,"source":51,"evidence":62},"Ollama local use is free; Replicate costs about $0.005 per second",[63],{"id":64,"by":11,"text":65},"37253133","If you run Ollama locally, the cost is zero. With Replicate, you pay about $0.005 per second, so with an average of say 15 seconds per request to generate the full response, and…",{"value":67,"source":51,"evidence":68},"Developers lacked good ways to use Code Llama in everyday workflows",[69],{"id":54},null,[],"open_source","launched","unknown",{"praise":76,"objections":96,"requests":130,"questions":145,"alternatives":156},[77,89],{"value":78,"source":79,"evidence":80},"Local Code Llama via Ollama is capable and cheap to run","commenter",[81,85],{"id":82,"by":83,"text":84},"37340417","jasonjmcghee","not true anymore. give codellama-7b-instruct a try. Just install Ollama. Pretty mind-blowing performance for the ram it uses and how fast it is. It's in the ballpark of…",{"id":86,"by":87,"text":88},"37256369","freckletonj","UniteAI (https:\u002F\u002Fgithub.com\u002Ffreckletonj\u002Funiteai) I think fits the bill for you. This is my project, where the goal is to Unite your AI-stack inside your editor (so,…",{"value":90,"source":79,"evidence":91},"Excitement about open code models",[92],{"id":93,"by":94,"text":95},"37252989","ramesh31","I just love this so much. It took what, maybe 6 hours for these model binaries to be publicly available? Never seen so much excitement in tech.",[97,108,119],{"value":98,"source":79,"evidence":99},"Running a capable model locally needs expensive hardware",[100,104],{"id":101,"by":102,"text":103},"37252960","say_it_as_it_is","This would be great if I had a beefy machine with 32GB ram and 16 cores. The alternatives presented are to use either Together or Standard api's to run llama-code. Both have a…",{"id":105,"by":106,"text":107},"37288725","yangcheng","but for code llm to be useful , the local machine need to have some very powerful GPU",{"value":109,"source":79,"evidence":110},"Local models may lag GPT-4 in quality",[111,115],{"id":112,"by":113,"text":114},"37256338","gettodachoppa","Why is this post flagged? I've yet to try code stuff with AI (even Copilot). How well do local models like Code Llama work compared to GPT? When I use ChatGPT, GPT 3.5 feels a…",{"id":116,"by":117,"text":118},"37255553","int_19h","You can't run anything even remotely close to GPT-4 locally. Large enterprises can run a private instance of it, but OpenAI is paid for that (indirectly via Microsoft).",{"value":120,"source":79,"evidence":121},"Not yet available on Windows or Linux",[122,126],{"id":123,"by":124,"text":125},"37252784","Xiol32","I don't have a Mac. So it's either have a Mac or don't run locally if you want to use this service?",{"id":127,"by":128,"text":129},"37252833","mchiang","Sorry about that. We are working on building the Windows and Linux versions. May I ask which specific OS you are on?",[131,138],{"value":132,"source":79,"evidence":133},"JetBrains plugin",[134],{"id":135,"by":136,"text":137},"37252598","smith7018","I wish someone would make an IntelliJ\u002FAndroid Studio plugin for Code Llama (or another local llm). I know both platforms have their own AI features but that involves sending my…",{"value":139,"source":79,"evidence":140},"Linux and Windows support",[141],{"id":142,"by":143,"text":144},"37277512","hawk_","Not OP, but would be great to have it working on Linux. How can one help?",[146,150],{"value":147,"source":79,"evidence":148},"How well does Code Llama compare to GPT-4?",[149],{"id":112,"by":113,"text":114},{"value":151,"source":79,"evidence":152},"Why are two external services needed if running locally?",[153],{"id":154,"by":124,"text":155},"37252548","Why is it necessary to sign up for two external services if this is running the LLM locally?",[157,161,168,175,182],{"value":158,"source":79,"evidence":159},"UniteAI",[160],{"id":86,"by":87,"text":88},{"value":162,"source":79,"evidence":163},"Refact",[164],{"id":165,"by":166,"text":167},"37261005","kateklink","at Refact we have Jetbrains plugin that works with a bunch of local code models https:\u002F\u002Fgithub.com\u002Fsmallcloudai\u002Frefact\u002F",{"value":169,"source":79,"evidence":170},"Cursor",[171],{"id":172,"by":173,"text":174},"37253193","dcre","I don't see how things like Cursor[0] (a VS Code fork with a nicer Codex UI) have a moat or chance of profitability in the face of things like this. If you can run models locally…",{"value":176,"source":79,"evidence":177},"Sourcegraph Cody",[178],{"id":179,"by":180,"text":181},"37254084","_steve_yegge_","I guess I've been summoned. We're primarily focusing on VSCode, IntelliJ and Neovim for Cody. Of course I'll be working on an Emacs version, but that's kinda best-effort for now.…",{"value":183,"source":79,"evidence":184},"GitHub Copilot",[185],{"id":64,"by":11,"text":65},{"comments":46,"answered":32,"key":187},[188,191,192],{"id":189,"text":190},"37253171","We've done work to set Continue up to be a JetBrains plugin—it's a matter of just a) figuring out the basics of putting a webview in a JetBrains plugin and b) implementing this…",{"id":64,"text":65},{"id":193,"text":194},"37252415","It's definitely a knowledgable model. I'm seeing a bit of trouble with stopping at the right point and managing chat context, but this is all a matter of prompt engineering. Very…",{"commenters":196,"max_depth":197,"maker_replies":46,"first_comment_mins":198,"lifespan_hours":199},35,8,23,168,[201,203,207,208,209,210,212,214,216,218,221,223,227,231,235,238,241,242,246,249,252,253,256,258,260,263,265,267,270,272,275,278,281,283,286,288,291,293,295,297,298,300,303,306,308,311,313,316,318,322,324,327],[135,34,32,202,136,34],"project\u002Fneutral\u002Fnone",[204,34,34,205,206,34],"37252928","other\u002Fneutral\u002Fnone","javajosh",[189,34,34,202,11,39],[86,34,34,205,87,34],[165,34,34,205,166,34],[105,34,39,211,106,34],"project\u002Fcritical\u002Fperformance",[82,34,46,213,83,34],"project\u002Fpositive\u002Fperformance",[93,39,46,215,94,34],"project\u002Fpositive\u002Fidea",[101,44,39,217,102,34],"project\u002Fcritical\u002Fpricing",[219,44,32,220,94,34],"37253089","project\u002Fneutral\u002Fperformance",[64,44,34,222,11,39],"project\u002Fneutral\u002Fpricing",[224,44,32,225,226,34],"37266660","maker\u002Fneutral\u002Fnone","agcat",[228,44,34,229,230,34],"37253300","commenter\u002Fneutral\u002Fpricing","whimsicalism",[232,44,34,233,234,34],"37260044","other\u002Fneutral\u002Fpricing","Havoc",[236,44,34,205,237,34],"37253491","freedomben",[239,44,32,220,240,34],"37294502","hmottestad",[112,32,32,220,113,34],[243,35,32,244,245,34],"37252470","project\u002Fneutral\u002Fidea","rtolsma",[247,35,34,205,248,34],"37252782","baggiponte",[250,35,34,205,251,34],"37253306","BaculumMeumEst",[179,35,34,205,180,34],[254,35,34,205,255,34],"37255551","tjdevries",[257,35,34,205,251,34],"37300319",[259,35,34,205,251,34],"37300309",[261,35,34,205,262,34],"37253346","w0m",[264,35,34,205,237,34],"37253454",[266,35,34,205,262,34],"37260909",[268,35,34,205,269,34],"37300235","tomr75",[271,35,34,205,262,34],"37307004",[273,35,34,205,274,34],"37314806","richchurcher",[276,46,34,205,277,34],"37254656","kristianp",[279,280,32,225,128,34],"37252341",6,[193,280,34,282,11,39],"project\u002Fmixed\u002Fperformance",[284,280,32,225,285,34],"37253412","jstummbillig",[287,280,34,211,11,39],"37254330",[154,289,32,290,124,34],7,"project\u002Fneutral\u002Fdocs",[292,289,32,290,128,34],"37252744",[123,289,39,294,124,34],"project\u002Fcritical\u002Fexecution",[127,289,32,296,128,34],"project\u002Fneutral\u002Fexecution",[142,289,32,296,143,34],[299,289,34,202,11,39],"37252800",[301,289,32,290,302,34],"37253465","daemonologist",[304,289,46,305,234,34],"37255802","project\u002Fpositive\u002Fdocs",[172,197,34,307,173,34],"other\u002Fneutral\u002Fidea",[309,197,34,205,310,34],"37253260","blackkettle",[312,197,34,233,173,34],"37253365",[314,197,34,205,315,34],"37253284","zaptrem",[317,197,34,205,173,34],"37253378",[319,197,34,320,321,34],"37253499","other\u002Fneutral\u002Fdesign","mlsu",[323,197,34,205,173,34],"37253634",[325,197,39,294,326,34],"37253777","lacoolj",[116,197,34,205,117,34],[],[330,334,339,344,348,351],{"id":331,"name":332,"points":333,"category":20},"37326870","compress",815,{"id":335,"name":336,"points":337,"category":338},"37247767","obsidian-sync",416,"infrastructure",{"id":340,"name":341,"points":342,"category":343},"37252231","Shimmer",229,"consumer_social",{"id":345,"name":346,"points":347,"category":338},"37263473","Release AI",143,{"id":349,"name":350,"points":347,"category":20},"37310070","Langfuse",{"id":352,"name":353,"points":354,"category":355},"37274882","Crumb",138,"languages",false,"v2",{"id":359,"name":5,"line":6,"ai":356,"cat":20,"year":13,"points":14,"comments":15,"rec":29,"s":360,"top":18,"kw":361,"maker":11,"domain":10,"tier":19,"attempts":34,"strip":362,"url":9,"mix":363,"praise":78,"objection":98,"sub":21},37251969,-0.05,[24,25,26,27,28],"3000153303003300000000003033313035000001",[32,33,34,35],1794036640687]