[{"data":1,"prerenderedAt":509},["ShallowReactive",2],{"project-39619053":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":29,"sentiment":30,"stance_mix":31,"scored_comments":35,"aspects":36,"strip":43,"post_text":46,"what":47,"said":71,"maker_in_thread":207,"shape":218,"grid":223,"attempts":471,"same_week":472,"maker_shows":39,"maker_hidden":501,"reader_version":502,"card":503},"39619053","Fructose","LLM calls as strongly typed functions","title","Show HN: Fructose – LLM calls as strongly typed functions","https:\u002F\u002Fgithub.com\u002Fbananaml\u002Ffructose","github.com","edunteman",1709749062,2024,218,99,355,3515,11,"notable","ai","llm function calling library","library",[24,25,26,27,28],"llm","python","structured output","typed functions","json mode","curious",-0.159,{"positive":32,"neutral":33,"mixed":34,"critical":18},4,22,1,38,{"idea":37,"design":40,"execution":41},{"positive":38,"critical":39},2,8,{"positive":38,"critical":38},{"positive":42,"critical":34},0,[42,42,42,42,34,42,42,42,34,42,34,34,34,34,44,44,44,45,45,44,44,42,42,34,42,34,45,45,44,44,44,44,42,44,42,44,44,44,44,34],3,5,"Hi HN! Erik here from Banana (formerly the serverless GPU platform), excited to show you what we’ve been working on next:\n\nFructose\n\nFructose is a python package to call LLMs as strongly typed functions. It uses function type signatures to guide the generation and guarantee a correctly typed output, in whatever basic\u002Fcomplex python datatype requested.\n\nBy guaranteeing output structure, we believe this will enable more complex applications to be built, interweaving code with LLMs with code. For now, we’ve shipped Fructose as a client-only library simply calling gpt-4 (by default) with json mode",{"problem":48,"audience":54,"pricing":54,"motivation":55,"time_to_build":54,"stack":59,"openness":68,"stage":69,"team_size":70},{"value":49,"source":50,"evidence":51},"Getting LLM output that reliably matches typed Python data structures","maker",[52],{"id":53},"post",null,{"value":56,"source":50,"evidence":57},"Enable more complex applications that interweave code and LLM calls by guaranteeing output structure",[58],{"id":53},[60,64],{"value":61,"source":50,"evidence":62},"Python",[63],{"id":53},{"value":65,"source":50,"evidence":66},"OpenAI gpt-4",[67],{"id":53},"open_source","beta","company",{"praise":72,"objections":91,"requests":131,"questions":157,"alternatives":178},[73,81],{"value":74,"source":75,"evidence":76},"Clean, neat aesthetics of the API","commenter",[77],{"id":78,"by":79,"text":80},"39620566","AmanKishore","This is much nicer than calling GPT in the middle of my code. Honestly the aesthetics of Fructose just make the code so much neater",{"value":82,"source":75,"evidence":83},"Functional programming feel fits generative programming",[84,88],{"id":85,"by":86,"text":87},"39620622","ravenstine","Yeah, functional programming and pure functions seem perfect for generative programming. Granted, I think they're perfect in general, but as long as human programmers are still…",{"id":89,"by":11,"text":90},"39620210","yeah I had a moment working with fructose where I realized \"oh this is more like functional programming than I expected\"",[92,107,118,124],{"value":93,"source":75,"evidence":94},"Too similar to Marvin and other existing libraries",[95,99,103],{"id":96,"by":97,"text":98},"39620241","itfollowsthen","> not unlike other packages such as marvin This feels pretty much identical to Marvin? Like the entire API? From a genuine place of curiosity: I get that your prompts are…",{"id":100,"by":101,"text":102},"39620337","nate_nowack","yeah this seems to be pretty much the same interface as `fn` from marvin, except w\u002Fo pydantic (see…",{"id":104,"by":105,"text":106},"39623107","lefttoreader","Wait. So why not just contribute to an existing open source project if you’re going to implement an identical API? If you run your own models as a part of it, surely you could…",{"value":108,"source":75,"evidence":109},"Relies on prompt-and-parse JSON rather than native function calling or grammar constraints",[110,114],{"id":111,"by":112,"text":113},"39619533","minimaxir","This approach may be too high-level \"magic\" to the point of being difficult to work with and iterate upon. Looking at the prompt templates…",{"id":115,"by":116,"text":117},"39624066","icyfox","Big proponent of guaranteed outputs for LLMs. I wrote a library awhile back (gpt-json) that did something similar by querying the OpenAI API. At the end of the day though while…",{"value":119,"source":50,"evidence":120},"Output structure not guaranteed; parse failures throw exceptions",[121],{"id":122,"by":11,"text":123},"39619610","It's not 100% yet. Route to that: 1. Clientside, retry strategy on failed parse. Not yet implemented, we throw an exception on parse fail right now, but soon to be implemented.…",{"value":125,"source":75,"evidence":126},"Only works with OpenAI models despite LLM framing",[127],{"id":128,"by":129,"text":130},"39621612","ilaksh","I find it grating that all of these types of things say \"LLMs\" when in fact they literally only work with OpenAI. There are hundreds of variations of LLM models. When it works…",[132,143,150],{"value":133,"source":75,"evidence":134},"Support for local LLMs like Ollama and LocalAI",[135,139],{"id":136,"by":137,"text":138},"39619444","jsight","I love the concept, but I'd really prefer being able to use it against local llms (localai, ollama, etc).",{"id":140,"by":141,"text":142},"39620248","yieldcrv","fyi: LM Studio can host a server that uses the OpenAI api to whatever model you are using locally So as long as this library can be directed to localhost or configured, it can…",{"value":144,"source":75,"evidence":145},"Support for Claude, Llama2 and other models",[146],{"id":147,"by":148,"text":149},"39619783","politelemon","Are you planning to add other types like Claude or Llama2?",{"value":151,"source":75,"evidence":152},"Pydantic support",[153],{"id":154,"by":155,"text":156},"39619468","jonathan-adly","Very Cool! Would it work for Pydantic out of the box? Or that's something coming along?",[158,173],{"value":159,"source":75,"evidence":160},"How does it compare to Instructor, Marvin, Outlines, LMQL, Guidance",[161,165,169],{"id":162,"by":163,"text":164},"39619798","hedgehog","How does Fructose relate or compare to Instructor (https:\u002F\u002Fgithub.com\u002Fjxnl\u002Finstructor)?",{"id":166,"by":167,"text":168},"39619568","biddit","I've done a lot of work over the last year wrangling LLM outputs - both from the OpenAI API as well as local LLMs. What are the benefits of using Fructose over LMQL, Guidance or…",{"id":170,"by":171,"text":172},"39625368","justanotheratom","Obvious question - how is this better than marvin, instructor, outlines.",{"value":174,"source":75,"evidence":175},"Why not contribute to existing open source projects instead",[176,177],{"id":96,"by":97,"text":98},{"id":104,"by":105,"text":106},[179,187,192,196,200],{"value":180,"source":75,"evidence":181},"Instructor",[182,186],{"id":183,"by":184,"text":185},"39620221","mattew","Good stuff. How does this compare to Instructor? I’ve been using this extensively https:\u002F\u002Fjxnl.github.io\u002Finstructor\u002F",{"id":162,"by":163,"text":164},{"value":188,"source":75,"evidence":189},"Marvin",[190,191],{"id":96,"by":97,"text":98},{"id":100,"by":101,"text":102},{"value":193,"source":75,"evidence":194},"Outlines",[195],{"id":111,"by":112,"text":113},{"value":197,"source":75,"evidence":198},"LMQL",[199],{"id":166,"by":167,"text":168},{"value":201,"source":75,"evidence":202},"Guidance",[203],{"id":204,"by":205,"text":206},"39619684","iAkashPaul","TGI just integrated Guidance in 1.4.3, that by itself can support both grammar\u002FJSON\u002FPydantic & tool invocation\u002Ffunction calling. Langchain & Llamaindex plus Fructose really need…",{"comments":208,"answered":209,"key":210},23,10,[211,212,215],{"id":89,"text":90},{"id":213,"text":214},"39619647","Many of our early users have said this as well. I don't want this to turn into an abstraction monstrosity: the more unadulterated the prompt, the better. We're looking to…",{"id":216,"text":217},"39624546","it seems this is in the context of \"extraction\" where all of the data is already present in the input text, and all that's needed is the reformatting. This is something we've…",{"commenters":219,"max_depth":220,"maker_replies":208,"first_comment_mins":221,"lifespan_hours":222},58,6,25,17,[224,228,230,234,236,239,241,243,245,246,250,253,256,258,261,264,267,269,272,275,277,279,280,282,285,288,291,292,293,295,297,300,302,304,307,309,312,314,316,319,322,323,325,327,330,334,335,338,342,343,345,346,349,351,353,356,358,361,363,366,368,370,372,376,380,384,389,393,397,400,401,403,405,408,411,413,416,418,420,423,425,427,429,431,433,436,438,442,445,447,449,451,452,453,455,457,460,463,467],[225,42,42,226,227,42],"39620062","other\u002Fneutral\u002Fidea","nextos",[89,42,42,229,11,34],"project\u002Fpositive\u002Fdesign",[231,42,42,232,233,42],"39620319","other\u002Fneutral\u002Fnone","obeavs",[235,42,42,232,227,42],"39620341",[237,42,42,226,238,42],"39620524","yaj54",[240,42,42,226,227,42],"39620819",[242,42,42,226,238,42],"39621073",[244,42,42,232,233,42],"39622810",[85,42,42,226,86,42],[247,42,34,248,249,42],"39620986","project\u002Fcritical\u002Fidea","fwip",[251,42,42,232,252,42],"39621294","nickpsecurity",[254,42,42,232,255,42],"39621534","ddellacosta",[257,42,42,232,227,42],"39621566",[259,42,42,260,255,42],"39621578","commenter\u002Fneutral\u002Fnone",[262,42,42,232,263,42],"39623421","amiantos",[265,42,42,260,266,42],"39624059","AgentOrange1234",[268,42,42,260,227,42],"39624260",[270,42,42,260,271,42],"39626354","trenchgun",[273,42,42,232,274,42],"39624375","pyinstallwoes",[111,34,34,276,112,42],"project\u002Fcritical\u002Fdesign",[213,34,42,278,11,34],"project\u002Fneutral\u002Fdesign",[115,38,42,226,116,42],[216,38,42,281,11,34],"project\u002Fneutral\u002Fidea",[283,38,42,232,284,42],"39624692","KTibow",[286,38,42,287,11,34],"39624730","project\u002Fneutral\u002Fexecution",[289,38,42,226,290,42],"39625194","disconcision",[96,44,34,248,97,42],[100,44,34,248,101,42],[294,44,42,281,11,34],"39620789",[104,44,34,296,105,42],"maker\u002Fcritical\u002Fidea",[298,44,42,260,299,42],"39623935","barapa",[301,44,42,260,105,42],"39624536",[303,44,42,260,299,42],"39624653",[305,32,34,276,306,42],"39621318","babyshake",[308,32,42,278,11,34],"39621449",[310,32,44,278,311,42],"39623652","dragonwriter",[313,32,42,229,11,34],"39624752",[315,32,44,278,311,42],"39623633",[317,32,42,232,318,42],"39624641","pedrovhb",[320,32,42,232,321,42],"39626315","thekombustor",[183,45,44,281,184,42],[324,45,42,281,11,34],"39620242",[326,45,45,229,238,42],"39620691",[328,220,44,281,329,42],"39620015","imtringued",[331,332,42,232,333,42],"39620749",7,"shykes",[78,39,45,229,79,42],[336,39,42,337,11,34],"39620587","maker\u002Fpositive\u002Fnone",[339,340,44,287,341,42],"39619485",9,"bagels",[122,340,42,287,11,34],[344,340,44,287,341,42],"39622561",[204,209,42,232,205,42],[347,209,42,232,348,42],"39619922","memothon",[350,209,42,232,11,34],"39620148",[352,209,42,260,348,42],"39620215",[354,209,42,260,355,42],"39620604","reliableturing",[357,209,42,260,11,34],"39621123",[359,209,42,260,360,42],"39620125","Terretta",[128,18,34,362,129,42],"project\u002Fcritical\u002Fexecution",[364,18,42,260,365,42],"39621686","vineetch",[367,18,42,260,129,42],"39623895",[369,18,42,260,11,34],"39624473",[371,18,34,248,205,42],"39624917",[373,374,34,248,375,42],"39626701",12,"Sai_",[377,378,42,232,379,42],"39624479",13,"campers",[381,378,44,382,383,42],"39624511","project\u002Fneutral\u002Fnone","Jonovono",[385,386,45,387,388,42],"39624693",14,"project\u002Fpositive\u002Fidea","nico",[390,391,44,382,392,42],"39620313",15,"msp26",[394,395,45,387,396,42],"39622631",16,"judicious",[398,395,42,399,11,34],"39624426","maker\u002Fneutral\u002Fnone",[136,222,44,382,137,42],[402,222,42,399,11,34],"39619510",[404,222,44,382,329,42],"39620035",[406,222,44,382,407,42],"39620107","ramses0",[409,222,42,260,410,42],"39622891","yunohn",[412,222,44,382,101,42],"39620277",[414,415,42,337,11,34],"39624954",18,[166,417,44,382,167,42],19,[419,417,42,281,11,34],"39619741",[421,417,44,382,422,42],"39620257","nerpderp82",[424,417,42,232,11,34],"39620286",[426,417,42,232,422,42],"39620705",[428,417,42,281,11,34],"39620892",[162,430,44,382,163,42],20,[432,430,42,281,11,34],"39620119",[434,430,42,232,435,42],"39623547","jxnlco",[437,430,42,232,11,34],"39624481",[439,430,44,440,441,42],"39627577","project\u002Fmixed\u002Fidea","ganarajpr",[443,430,44,382,444,42],"39620153","dennisy",[154,446,44,382,155,42],21,[448,446,42,382,11,34],"39620014",[450,446,44,287,238,42],"39620332",[170,33,44,382,171,42],[140,208,44,382,141,42],[147,454,44,382,148,42],24,[456,454,42,382,11,34],"39620171",[458,221,44,382,459,42],"39619499","FrustratedMonky",[461,221,34,248,462,42],"39621482","Smaug123",[464,465,42,232,466,42],"39623659",26,"yumidarwi001",[468,469,34,248,470,42],"39619879",27,"creatonez",[],[473,478,482,487,492,496],{"id":474,"name":475,"points":476,"category":477},"39597030","Puter",1274,"dev_tools",{"id":479,"name":480,"points":481,"category":477},"39620540","dockerc",374,{"id":483,"name":484,"points":485,"category":486},"39557188","Struct",298,"productivity",{"id":488,"name":489,"points":490,"category":491},"39591529","Astro App",291,"learning_science",{"id":493,"name":494,"points":495,"category":486},"39551064","Row Zero",274,{"id":497,"name":498,"points":499,"category":500},"39549194","Workout Tracker",263,"infrastructure",false,"v2",{"id":504,"name":5,"line":6,"ai":501,"cat":20,"year":13,"points":14,"comments":15,"rec":29,"s":505,"top":18,"kw":506,"maker":11,"domain":10,"tier":19,"attempts":42,"strip":507,"url":9,"mix":508,"praise":74,"objection":93,"sub":21},39619053,-0.16,[24,25,26,27,28],"0000100010111133355330010155333303033331",[32,33,34,18],1793991086049]