[{"data":1,"prerenderedAt":320},["ShallowReactive",2],{"project-47212576":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":29,"sentiment":30,"stance_mix":31,"scored_comments":34,"aspects":35,"strip":43,"post_text":44,"what":45,"said":80,"maker_in_thread":192,"shape":202,"grid":207,"attempts":284,"same_week":285,"maker_shows":40,"maker_hidden":313,"reader_version":314,"card":315},"47212576","Timber","Ollama for classical ML models, 336x faster than Python","title","Show HN: Timber – Ollama for classical ML models, 336x faster than Python","https:\u002F\u002Fgithub.com\u002Fkossisoroyce\u002Ftimber","github.com","kossisoroyce",1772413060,2026,207,33,277,4366,7,"notable","ai","classical ml inference","cli",[24,25,26,27,28],"classical ml","model serving","scikit-learn inference","onnx","ml deployment","debated",0,{"positive":32,"neutral":32,"mixed":30,"critical":33},6,5,17,{"idea":36,"performance":39,"design":41,"execution":42},{"positive":37,"critical":38},3,2,{"positive":40,"critical":40},1,{"positive":38,"critical":40},{"positive":30,"critical":40},[37,30,30,30,30,40,30,30,37,30,37,30,40,30,37,37,30,40,33,30,33,30,33,30,37,33,40,33,30,33,40,30,30,30],null,{"problem":46,"audience":56,"pricing":44,"motivation":60,"time_to_build":44,"stack":64,"openness":79,"stage":79,"team_size":79},{"value":47,"source":48,"evidence":49},"Python runtime overhead makes classical ML inference slow in latency-critical deployments","maker",[50,53],{"id":51,"by":11,"text":52},"47215655","The motivation was edge and latency-critical use cases on a product I consulted on. Feature vectors arrived pre-formed and a Python runtime in the hot path wass a non-starter.…",{"id":54,"by":11,"text":55},"47215675","Exactly. The objective is to abstract away completely. Shared libraries just add too much overhead.",{"value":57,"source":48,"evidence":58},"teams deploying classical ML models in edge or latency-critical systems",[59],{"id":51,"by":11,"text":52},{"value":61,"source":48,"evidence":62},"Edge and latency-critical use cases where a Python runtime in the hot path was not viable",[63],{"id":51,"by":11,"text":52},[65,71],{"value":66,"source":48,"evidence":67},"C",[68],{"id":69,"by":11,"text":70},"47215608","Fair point in general, but the risk profile here is actually quite low. The generated C is purely computational, with no heap allocation, no pointer arithmetic, no…",{"value":72,"source":73,"evidence":74},"ONNX","commenter",[75],{"id":76,"by":77,"text":78},"47254775","STARGA","The 336x speedup claim makes sense if the baseline is scikit-learn with Python overhead on every prediction call. Python's interpreter loop is the bottleneck for inference on…","unknown",{"praise":81,"objections":105,"requests":135,"questions":154,"alternatives":173},[82,97],{"value":83,"source":73,"evidence":84},"Good concept and timely for classical ML serving",[85,89,93],{"id":86,"by":87,"text":88},"47213467","Dansvidania","Can’t check it out yet, but the concept alone sounds great. Thank you for sharing.",{"id":90,"by":91,"text":92},"47213239","jnstrdm05","I have been waiting for this! Nice",{"id":94,"by":95,"text":96},"47214612","palashkulsh","Nice idea, i needed something like it",{"value":98,"source":73,"evidence":99},"Speedup claim makes sense versus Python per-call overhead",[100,101],{"id":76,"by":77,"text":78},{"id":102,"by":103,"text":104},"47213771","sriram_malhar","Perhaps because the performance is good enough and this approach is much simpler and portable than shared libraries across platforms.",[106,117,128],{"value":107,"source":73,"evidence":108},"Ollama is a weak analogy; classical ML models are narrow and not interchangeable like LLMs",[109,113],{"id":110,"by":111,"text":112},"47213671","brokensegue","\"classical ML\" models typically have a more narrow range of applicability. in my mind the value of ollama is that you can easily download and swap-out different models with the…",{"id":114,"by":115,"text":116},"47213507","mehdibl","Ollama is quite a bad example here. Despite popular, it's a simple wrapper and more and more pushed by the app it wraps llama.cpp. Don't understand here the parallel.",{"value":118,"source":73,"evidence":119},"Inference is rarely the bottleneck; data preprocessing dominates pipelines",[120,124],{"id":121,"by":122,"text":123},"47214198","o10449366","Can you tell us more about the motivation for this project? I'm very curious if it was driven by a specific use case. I know there are specialized trading firms that have…",{"id":125,"by":126,"text":127},"47227711","bbstats","inference is not usually slow for classical ML (save lightgbm)",{"value":129,"source":73,"evidence":130},"Generated C is memory-unsafe; Rust, Zig or Nim would be safer",[131],{"id":132,"by":133,"text":134},"47214831","OutOfHere","It would be safer to use a Zig or Rust or Nim target. C risks memory-unsafe behavior.",[136,143,150],{"value":137,"source":73,"evidence":138},"Benchmarks against vanilla inference latency",[139],{"id":140,"by":141,"text":142},"47215442","marcyb5st","You can do that with Onnx. You can graft the preprocessing layers to the actual model [1] and then serve that. Honestly, I already thought that ONNX (CPU at least) was already…",{"value":144,"source":73,"evidence":145},"Support for raw input preprocessing rather than feature vectors",[146],{"id":147,"by":148,"text":149},"47214906","theanonymousone","Wouldn't it be much more useful if the request received raw input (i.e. before feature extraction), and not the feature vector?",{"value":151,"source":73,"evidence":152},"Plugin system for custom or legacy model formats such as PMML",[153],{"id":76,"by":77,"text":78},[155,159,166],{"value":156,"source":73,"evidence":157},"What was the motivation and specific use case",[158],{"id":121,"by":122,"text":123},{"value":160,"source":73,"evidence":161},"Does it use XNNPACK under the hood",[162],{"id":163,"by":164,"text":165},"47218145","deepsquirrelnet","Does this use something like xnnpack under the hood?",{"value":167,"source":73,"evidence":168},"Why a separate process instead of a shared library",[169],{"id":170,"by":171,"text":172},"47213724","rudhdb773b","If the focus is performance, why use a separate process and have to deal with data serialization overhead? Why not a typical shared library that can be loaded in python, R,…",[174,178,185],{"value":175,"source":73,"evidence":176},"ONNX Runtime",[177],{"id":140,"by":141,"text":142},{"value":179,"source":73,"evidence":180},"llama.cpp server",[181],{"id":182,"by":183,"text":184},"47215375","sieve","llama-cpp provides an API server as well via llama-server (and a competent webgui too).",{"value":186,"source":73,"evidence":187},"vLLM",[188],{"id":189,"by":190,"text":191},"47215374","eleventyseven","This is the vLLM of classic ML, not Ollama.",{"comments":193,"answered":38,"key":194},8,[195,198,199],{"id":196,"text":197},"47215742","TBVH I didn't think about naming it too much. I defaulted to Ollama because of the perceive simplicity and I wanted that same perceived simplicity to help adoption.",{"id":51,"text":52},{"id":200,"text":201},"47215691","I'll check this out as soon as I am at my desk.",{"commenters":203,"max_depth":204,"maker_replies":193,"first_comment_mins":205,"lifespan_hours":206},24,4,103,69,[208,212,216,220,222,225,227,231,232,233,235,236,237,238,241,242,244,245,247,249,251,254,257,259,261,262,263,266,268,270,273,276,278,281],[209,30,37,210,211,30],"47213614","project\u002Fneutral\u002Fidea","tl2do",[213,30,30,214,215,30],"47214905","other\u002Fneutral\u002Fidea","visarga",[217,30,30,218,219,30],"47215576","commenter\u002Fneutral\u002Fidea","benrutter",[221,30,30,218,215,30],"47235056",[223,30,30,218,224,30],"47216077","mirsadm",[114,40,40,226,115,30],"project\u002Fcritical\u002Fidea",[228,40,30,229,230,30],"47214442","commenter\u002Fneutral\u002Fnone","ekianjo",[182,40,30,229,183,30],[189,40,37,210,190,30],[196,40,30,234,11,40],"maker\u002Fneutral\u002Fnone",[121,38,37,210,122,30],[51,38,30,210,11,40],[110,37,40,226,111,30],[239,37,30,240,190,30],"47220183","commenter\u002Fpositive\u002Fperformance",[147,204,37,210,148,30],[140,204,37,243,141,30],"project\u002Fneutral\u002Fperformance",[200,204,30,234,11,40],[170,33,40,246,171,30],"project\u002Fcritical\u002Fdesign",[102,33,33,248,103,30],"project\u002Fpositive\u002Fdesign",[54,33,30,250,11,40],"project\u002Fneutral\u002Fdesign",[252,33,33,248,253,30],"47214469","bob001",[255,33,30,256,11,40],"47215665","maker\u002Fpositive\u002Fnone",[86,32,33,258,87,30],"project\u002Fpositive\u002Fidea",[260,32,30,234,11,40],"47215661",[163,18,37,243,164,30],[94,193,33,258,95,30],[125,264,40,265,126,30],9,"project\u002Fcritical\u002Fperformance",[90,267,33,258,91,30],10,[269,267,30,234,11,40],"47215575",[76,271,33,272,77,30],11,"project\u002Fpositive\u002Fperformance",[132,274,40,275,133,30],12,"project\u002Fcritical\u002Fexecution",[69,274,30,277,11,40],"maker\u002Fneutral\u002Fexecution",[279,274,30,280,133,30],"47218574","commenter\u002Fneutral\u002Fexecution",[282,274,30,280,283,30],"47228917","reverius42",[],[286,290,295,299,304,309],{"id":287,"name":288,"points":289,"category":20},"47224295","I built a sub-500ms latency voice agent from scratch",570,{"id":291,"name":292,"points":293,"category":294},"47164270","Terminal Phone",322,"security_privacy",{"id":296,"name":297,"points":298,"category":20},"47195123","Now I Get It",305,{"id":300,"name":301,"points":302,"category":303},"47220809","Govbase",219,"productivity",{"id":305,"name":306,"points":307,"category":308},"47180083","RetroTick",195,"dev_tools",{"id":310,"name":311,"points":307,"category":312},"47197218","Stacked Game of Life","learning_science",false,"v2",{"id":316,"name":5,"line":6,"ai":313,"cat":20,"year":13,"points":14,"comments":15,"rec":29,"s":30,"top":18,"kw":317,"maker":11,"domain":10,"tier":19,"attempts":30,"strip":318,"url":9,"mix":319,"praise":83,"objection":107,"sub":21},47212576,[24,25,26,27,28],"3000010030301033015050503515051000",[32,32,30,33],1794036694874]