[{"data":1,"prerenderedAt":252},["ShallowReactive",2],{"project-36409201":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":34,"aspects":35,"strip":39,"post_text":42,"what":43,"said":72,"maker_in_thread":148,"shape":157,"grid":162,"attempts":214,"same_week":215,"maker_shows":37,"maker_hidden":244,"reader_version":245,"card":246},"36409201","Autolabel","A Python library to label and enrich text data with LLMs","title","Show HN: Autolabel, a Python library to label and enrich text data with LLMs","https:\u002F\u002Fgithub.com\u002Frefuel-ai\u002Fautolabel","github.com","nihit-desai",1687289162,2023,153,22,406,3109,14,"notable","data","data labeling library","library",[24,25,26,27],"llm data labeling","text labeling","dataset enrichment","confidence scores","curious",-0.061,{"positive":31,"neutral":32,"mixed":33,"critical":31},2,6,0,10,{"idea":36,"privacy":38},{"positive":37,"critical":33},1,{"positive":37,"critical":33},[40,33,40,33,33,33,41,41,40,37,40,33,33,33,33,33,40,33,33,40,37,33],3,5,"Hi HN! I'm excited to share Autolabel, an open-source Python library to label and enrich text datasets with any Large Language Model (LLM) of your choice.\n\nWe built Autolabel because access to clean, labeled data is a huge bottleneck for most ML\u002Fdata science teams. The most capable LLMs are able to label data with high accuracy, and at a fraction of the cost and time compared to manual labeling. With Autolabel, you can leverage LLMs to label any text dataset with \u003C5 lines of code.\n\nWe’re eager for your feedback!",{"problem":44,"audience":50,"pricing":54,"motivation":55,"time_to_build":54,"stack":59,"openness":70,"stage":71,"team_size":71},{"value":45,"source":46,"evidence":47},"Access to clean labeled data is a bottleneck for ML teams; manual labeling is slow and costly","maker",[48],{"id":49},"post",{"value":51,"source":46,"evidence":52},"ML and data science teams labeling text datasets",[53],{"id":49},null,{"value":56,"source":46,"evidence":57},"Labeled data is a bottleneck for ML teams; LLMs can label accurately and cheaply",[58],{"id":49},[60,64],{"value":61,"source":46,"evidence":62},"Python",[63],{"id":49},{"value":65,"source":46,"evidence":66},"Hugging Face pipelines",[67],{"id":68,"by":11,"text":69},"36417333","Yep! I totally understand the concerns around not being able to share data externally - the library currently supports open source, self-hosted LLMs through huggingface pipelines…","open_source","unknown",{"praise":73,"objections":86,"requests":105,"questions":121,"alternatives":136},[74,82],{"value":75,"source":76,"evidence":77},"Confidence estimation lets LLM labels be used where reliable and manual labeling for the rest","commenter",[78],{"id":79,"by":80,"text":81},"36411433","viswajithiii","Thank you for open sourcing this! This seems very useful, especially because of the confidence estimation, which lets you use LLMs for the points they can do well and fall back…",{"value":83,"source":46,"evidence":84},"Support for self-hosted open-source models addresses privacy concerns",[85],{"id":68,"by":11,"text":69},[87,98],{"value":88,"source":76,"evidence":89},"OpenAI terms may forbid using outputs to build competing datasets or models",[90,94],{"id":91,"by":92,"text":93},"36409582","isawczuk","You should read carefully OpenAI terms and conditions before using it to build custom datasets.",{"id":95,"by":96,"text":97},"36410626","iillexial","For anyone wondering it's here https:\u002F\u002Fopenai.com\u002Fpolicies\u002Fterms-of-use: >use output from the Services to develop models that compete with OpenAI; Well, I still can use ChatGPT…",{"value":99,"source":76,"evidence":100},"Spamming HN with promotional posts",[101],{"id":102,"by":103,"text":104},"36410705","voz_","You just posted this here https:\u002F\u002Fnews.ycombinator.com\u002Fitem?id=36384015 It's one thing to show HN \u002F share, its another thing to spam it with your ads.",[106,117],{"value":107,"source":76,"evidence":108},"Explain how confidence scores work",[109,113],{"id":110,"by":111,"text":112},"36417025","msp26",">Refuel provides LLMs that can compute confidence scores for every label, if the LLM you've chosen doesn't provide token-level log probabilities. How does this work exactly?",{"id":114,"by":115,"text":116},"36424463","applgo443","How does the confidence scores work?",{"value":118,"source":46,"evidence":119},"Support for llama.cpp self-hosted models",[120],{"id":68,"by":11,"text":69},[122,129],{"value":123,"source":76,"evidence":124},"What does it do that OpenAI function calling does not?",[125],{"id":126,"by":127,"text":128},"36414619","bomewish","What can this do that the new ‘calling functions’ feature can’t? It seems to be roughly the same thing?",{"value":130,"source":76,"evidence":131},"Can data be used without sending it outside the organisation?",[132],{"id":133,"by":134,"text":135},"36415147","devjab","This is very interesting to me. We spent a significant time “labelling” data when I was in the public sector digitalisation. Basically what was done, was to do the LLM part…",[137,141],{"value":138,"source":76,"evidence":139},"OpenAI function calling",[140],{"id":126,"by":127,"text":128},{"value":142,"source":76,"evidence":143},"llama.cpp self-hosted models",[144],{"id":145,"by":146,"text":147},"36415859","oli5679","You can self-host an open-source model. Llama CCP is a very popular project with great docs. https:\u002F\u002Fgithub.com\u002Fggerganov\u002Fllama.cpp You need to be careful about liscencing - some…",{"comments":40,"answered":40,"key":149},[150,153,154],{"id":151,"text":152},"36417479","function calling, as I understand it, makes LLM outputs easier to consume by downstream APIs\u002Ffunctions (https:\u002F\u002Fopenai.com\u002Fblog\u002Ffunction-calling-and-other-api-updates). Autolabel…",{"id":68,"text":69},{"id":155,"text":156},"36411062","Hi! The earlier post was a report summarizing LLM labeling benchmarking results. This post shares the open source library. Neither is intended to be an ad. Our hope with sharing…",{"commenters":158,"max_depth":159,"maker_replies":40,"first_comment_mins":160,"lifespan_hours":161},16,7,28,45,[163,165,166,168,170,173,174,177,179,180,183,187,191,195,198,200,202,203,206,209,210,212],[126,33,40,164,127,33],"project\u002Fneutral\u002Fidea",[151,33,33,164,11,37],[133,37,40,167,134,33],"project\u002Fneutral\u002Fprivacy",[145,37,33,169,146,33],"commenter\u002Fneutral\u002Fprivacy",[171,37,33,172,134,33],"36428380","other\u002Fneutral\u002Fprivacy",[68,37,33,167,11,37],[175,37,41,176,134,33],"36428383","project\u002Fpositive\u002Fprivacy",[79,31,41,178,80,33],"project\u002Fpositive\u002Fidea",[110,40,40,164,111,33],[91,181,37,182,92,33],4,"project\u002Fcritical\u002Fnone",[184,181,40,185,186,33],"36409703","project\u002Fneutral\u002Fnone","victorbjorklund",[188,181,33,189,190,33],"36409935","other\u002Fneutral\u002Fnone","moffkalast",[192,181,33,193,194,33],"36410589","commenter\u002Fneutral\u002Fnone","chillbill",[196,181,33,193,197,33],"36412213","nickstinemates",[199,181,33,193,194,33],"36430856",[201,181,33,189,190,33],"36434864",[95,181,40,185,96,33],[204,181,33,189,205,33],"36411695","binarymax",[207,181,33,189,208,33],"36413952","Takennickname",[114,41,40,164,115,33],[102,32,37,211,103,33],"maker\u002Fcritical\u002Fnone",[155,32,33,213,11,37],"maker\u002Fneutral\u002Fnone",[],[216,221,226,230,235,240],{"id":217,"name":218,"points":219,"category":220},"36365511","PropBox",520,"web_saas",{"id":222,"name":223,"points":224,"category":225},"36360789","Novel",370,"productivity",{"id":227,"name":228,"points":229,"category":20},"36383773","Answer Overflow",333,{"id":231,"name":232,"points":233,"category":234},"36391655","HN Recommend",314,"learning_science",{"id":236,"name":237,"points":238,"category":239},"36388821","Slint",291,"dev_tools",{"id":241,"name":242,"points":243,"category":239},"36422730","gpt-engineer",178,false,"v2",{"id":247,"name":5,"line":6,"ai":244,"cat":20,"year":13,"points":14,"comments":15,"rec":28,"s":248,"top":18,"kw":249,"maker":11,"domain":10,"tier":19,"attempts":33,"strip":250,"url":9,"mix":251,"praise":75,"objection":88,"sub":21},36409201,-0.06,[24,25,26,27],"3030005531300000300310",[31,32,33,31],1794036637194]