[{"data":1,"prerenderedAt":525},["ShallowReactive",2],{"project-33877893":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":29,"sentiment":30,"stance_mix":31,"scored_comments":35,"aspects":36,"strip":47,"post_text":48,"what":49,"said":93,"maker_in_thread":209,"shape":220,"grid":223,"attempts":487,"same_week":488,"maker_shows":211,"maker_hidden":517,"reader_version":518,"card":519},"33877893","Whisper","Port of OpenAI's Whisper model in C\u002FC++","title","Show HN: Port of OpenAI's Whisper model in C\u002FC++","https:\u002F\u002Fgithub.com\u002Fggerganov\u002Fwhisper.cpp","github.com","ggerganov",1670323585,2022,399,87,85,2570,4,"legend","ai","speech recognition","library",[24,25,26,27,28],"speech to text","whisper","transcription","c++ inference","offline speech recognition","loved",0.455,{"positive":32,"neutral":33,"mixed":34,"critical":18},29,21,2,56,{"design":37,"docs":40,"execution":42,"performance":43,"idea":45},{"positive":38,"critical":39},3,0,{"positive":39,"critical":41},1,{"positive":18,"critical":39},{"positive":44,"critical":41},5,{"positive":46,"critical":34},9,[44,39,44,38,44,44,38,44,38,44,44,44,44,38,38,39,44,38,44,38,44,38,44,44,44,38,38,44,44,41,44,38,44,44,44,39,39,39,41,39],null,{"problem":50,"audience":57,"pricing":48,"motivation":70,"time_to_build":48,"stack":77,"openness":90,"stage":91,"team_size":92},{"value":51,"source":52,"evidence":53},"Running Whisper speech recognition without PyTorch or heavy CUDA dependencies","maker",[54],{"id":55,"by":11,"text":56},"33877895","Hi HN, OpenAI recently released a model for automatic speech recognition called Whisper [0]. I decided to reimplement the inference of the model from scratch using C\u002FC++. To…",{"value":58,"source":59,"evidence":60},"Developers wanting offline speech recognition on CPUs, Apple Silicon, Raspberry Pi or in browsers","inferred",[61,62,66],{"id":55,"by":11,"text":56},{"id":63,"by":64,"text":65},"33892666","stephc_int13","Offline models, especially speech recognition, is a game changer for many apps. A fully CPU based implementation, simple enough with minimal dependencies is also something that…",{"id":67,"by":68,"text":69},"33898644","Tystros","Very nice! Especially that it can run with good speed on the CPU without any external dependencies. I wish something like this would also exist for Stable Diffusion - a simple,…",{"value":71,"source":52,"evidence":72},"Reimplemented Whisper from scratch to learn how transformers work and practice low-level optimization",[73,74],{"id":55,"by":11,"text":56},{"id":75,"by":11,"text":76},"33892297","Thanks! Indeed, I agree that the project has an educational aspect and value. For me, it helped me get a better understanding of the neural network layers involved in the…",[78,82,86],{"value":79,"source":52,"evidence":80},"C",[81],{"id":55,"by":11,"text":56},{"value":83,"source":52,"evidence":84},"C++",[85],{"id":55,"by":11,"text":56},{"value":87,"source":52,"evidence":88},"WebAssembly",[89],{"id":55,"by":11,"text":56},"open_source","launched","solo",{"praise":94,"objections":124,"requests":145,"questions":164,"alternatives":190},[95,108,116],{"value":96,"source":97,"evidence":98},"Fast CPU inference, no dependencies, easy to build","commenter",[99,100,104],{"id":63,"by":64,"text":65},{"id":101,"by":102,"text":103},"33899031","klim2media","Works well on my old Intel 2020 Macbook Pro. Having no dependencies meant I was able to just clone the repo and build (as long as you have the C++ tooling). There's something…",{"id":105,"by":106,"text":107},"33895629","iamflimflam1","Really amazing work - just cloned the repo on my Apple Silicon Mac and ran the streaming sample. Worked flawlessly.",{"value":109,"source":97,"evidence":110},"Enables offline speech recognition and accessibility uses",[111,115],{"id":112,"by":113,"text":114},"33892956","gdz","I've been watching this repo pretty much since the beginning, and the amount of work you've achieved is incredible. I've started tinkering with the code about last week and…",{"id":63,"by":64,"text":65},{"value":117,"source":97,"evidence":118},"Clean, minimal codebase",[119,123],{"id":120,"by":121,"text":122},"33893603","GistNoesis","Great work. It's a real bowl of fresh air coming from huge framework that use cuda. So many different cuda version, with each framework using its own, that all rely on a…",{"id":112,"by":113,"text":114},[125,132,138],{"value":126,"source":97,"evidence":127},"Not clearly better than built-in or other open-source speech models",[128],{"id":129,"by":130,"text":131},"33895095","jarbus","There are tons of open source STT models, what makes whisper so valuable? I especially don't get it on mobile, where the native STT built into the OS is now real-time and…",{"value":133,"source":52,"evidence":134},"Greedy decoder gives more mistakes than beam search",[135],{"id":136,"by":11,"text":137},"33896363","I have only implemented the Greedy decoder which is worse compared to the BeamSearch decoder in most cases.",{"value":139,"source":97,"evidence":140},"Mixed C\u002FC++ naming criticized as not a real language",[141],{"id":142,"by":143,"text":144},"33898579","moloch-hai","There is no language called \"C\u002FC++\". If, to use it, you need C++, it's a C++ system, period. If parts are in C, that is an implementation detail, and is furthermore pointless;…",[146,153,157],{"value":147,"source":97,"evidence":148},"Speaker identification in output",[149],{"id":150,"by":151,"text":152},"33898699","eigenvalue","This is really awesome, great work! I downloaded a long video from YouTube that is very challenging to transcribe (it is an interview with Hayek, who was both very soft spoken…",{"value":154,"source":97,"evidence":155},"Built-in audio format conversion via ffmpeg",[156],{"id":150,"by":151,"text":152},{"value":158,"source":97,"evidence":159},"Feature roadmap",[160],{"id":161,"by":162,"text":163},"33898164","password4321","Is there a feature roadmap? OpenAI quietly launched Whisper V2 https:\u002F\u002Fnews.ycombinator.com\u002Fitem?id=33884716",[165,176,183],{"value":166,"source":97,"evidence":167},"GPU or Neural Engine support",[168,172],{"id":169,"by":170,"text":171},"33894349","conradev","This is really cool, and I have been meaning to get my hands dirty with Whisper I looks like you’ve definitely maximized the parallelization on the CPU\u002FAMX here, but have you…",{"id":173,"by":174,"text":175},"33895590","fareesh","Is there any improvement to GPU inference at all? CPU inference is not cost effective for my use-case",{"value":177,"source":97,"evidence":178},"How to deploy in a web app",[179],{"id":180,"by":181,"text":182},"33910260","TheEndIsNigh","How could I deploy and use this in my web app? I don't know much about Web Assembly if someone can ELI5 please thanks. Is the result some API that I send requests to? Thanks a ton!",{"value":184,"source":97,"evidence":185},"Confidence levels for restricted vocabulary",[186],{"id":187,"by":188,"text":189},"33894902","wfcollins","Hi Georgi, I am experimenting with your code now. Is there a way to force Whisper to only consider a limited vocabulary and then respond with confidence levels? I am working on…",[191,198,205],{"value":192,"source":97,"evidence":193},"PyTorch Whisper implementation",[194],{"id":195,"by":196,"text":197},"33892746","sva_","> about 2-3 times faster compared to the current PyTorch implementation This is surprising to me. Is this about CPU only? Then it would make sense.",{"value":199,"source":97,"evidence":200},"tinygrad",[201],{"id":202,"by":203,"text":204},"33892466","vletal","Awesome work. Curious whether the whisper model could be ported to tinygrad and how the performance would compare to your implementation.",{"value":206,"source":97,"evidence":207},"Built-in OS speech recognition",[208],{"id":129,"by":130,"text":131},{"comments":210,"answered":211,"key":212},14,10,[213,214,217],{"id":75,"text":76},{"id":215,"text":216},"33894184","I don't have other recommendations as I am a novice myself when it comes to SIMD. I think the multiplication routines in `whisper.cpp` are relatively basic - dot product and…",{"id":218,"text":219},"33898814","Btw, there is the `yt-wsp.sh` helper script to download, convert and transcribe a video by given url: ``` .\u002Fexamples\u002Fyt-wsp.sh \u003Cvideo-url> ```",{"commenters":221,"max_depth":18,"maker_replies":210,"first_comment_mins":39,"lifespan_hours":222},61,190,[224,226,229,233,237,241,244,248,250,251,254,256,260,261,263,265,268,271,273,276,279,282,286,291,293,296,299,302,305,308,310,312,317,319,322,325,327,329,332,334,338,340,342,344,348,350,354,358,362,363,366,368,370,374,378,382,384,386,388,391,395,399,401,406,410,414,417,420,423,426,428,430,434,439,443,447,451,453,456,459,461,465,467,469,472,474,478,481,484],[120,39,44,225,121,39],"project\u002Fpositive\u002Fdesign",[227,39,39,228,64,39],"33894751","commenter\u002Fpositive\u002Fnone",[230,39,39,231,232,39],"33897751","commenter\u002Fneutral\u002Fnone","ShamelessC",[234,39,39,235,236,39],"33904633","other\u002Fneutral\u002Fnone","ProtoAES256",[238,41,44,239,240,39],"33894568","project\u002Fpositive\u002Fnone","AMICABoard",[242,41,44,225,243,39],"33896109","stevenhuang",[245,41,38,246,247,39],"33936992","project\u002Fneutral\u002Fdocs","NWoodsman",[249,41,38,246,247,39],"33937592",[112,34,44,239,113,39],[252,38,44,239,253,39],"33892177","thot_experiment",[75,38,39,255,11,41],"maker\u002Fneutral\u002Fnone",[257,38,44,258,259,39],"33893221","maker\u002Fpositive\u002Fdesign","ahgamut",[215,38,39,255,11,41],[262,18,38,246,247,39],"33937809",[150,44,38,264,151,39],"project\u002Fmixed\u002Fexecution",[266,44,44,267,151,39],"33898754","project\u002Fpositive\u002Fperformance",[269,44,38,270,151,39],"33898856","project\u002Fneutral\u002Fnone",[218,44,39,272,11,41],"maker\u002Fneutral\u002Fdocs",[274,44,44,275,151,39],"33898863","project\u002Fpositive\u002Fidea",[277,44,41,278,151,39],"33899423","project\u002Fcritical\u002Fdocs",[280,44,38,246,281,39],"33901969","zimpenfish",[283,284,44,275,285,39],"33892984",6,"codetrotter",[287,288,44,289,290,39],"33892330",7,"project\u002Fpositive\u002Fexecution","fimdomeio",[63,292,44,275,64,39],8,[294,46,38,270,295,39],"33893478","buzzier",[297,46,44,267,298,39],"33894666","naillo",[300,211,44,275,301,39],"33892842","IshKebab",[187,303,38,304,188,39],11,"project\u002Fneutral\u002Fidea",[306,303,38,304,307,39],"33898305","jjwiseman",[309,303,39,255,11,41],"33905282",[311,303,39,255,11,41],"33981314",[313,314,38,315,316,39],"33913570",12,"project\u002Fmixed\u002Fperformance","Const-me",[202,318,38,304,203,39],13,[320,318,39,321,11,41],"33892565","maker\u002Fneutral\u002Fperformance",[323,210,39,235,324,39],"33899781","andrewstuart",[169,326,44,267,170,39],15,[328,326,39,321,11,41],"33894487",[330,326,38,331,240,39],"33894584","project\u002Fneutral\u002Fperformance",[333,326,39,231,170,39],"33897513",[335,326,38,336,337,39],"33900206","project\u002Fneutral\u002Fexecution","mirsadm",[67,339,44,275,68,39],16,[195,341,38,331,196,39],17,[343,341,39,321,11,41],"33892865",[345,346,38,304,347,39],"33894531",18,"doctorwho42",[349,346,39,321,11,41],"33895480",[351,352,44,239,353,39],"33893037",19,"lyu07282",[355,356,44,275,357,39],"33894287",20,"kfihihc",[359,33,41,360,361,39],"33896287","project\u002Fcritical\u002Fperformance","IceHegel",[136,33,39,321,11,41],[364,33,38,336,365,39],"33897160","pbowyer",[105,367,44,289,106,39],22,[101,369,44,289,102,39],23,[371,372,44,267,373,39],"33897802",24,"ocdtrekkie",[375,376,44,239,377,39],"33894471",25,"FL33TW00D",[379,380,44,275,381,39],"33904645",26,"pkz",[161,383,38,304,162,39],27,[385,383,39,255,11,41],"33898580",[180,387,38,246,181,39],28,[389,32,38,304,390,39],"33912695","schoubey",[392,393,38,304,394,39],"33896162",30,"jojohack",[396,397,44,275,398,39],"33893027",31,"borbulon",[400,397,39,255,11,41],"33893257",[402,403,44,404,405,39],"33894996",32,"maker\u002Fpositive\u002Fexecution","binarymax",[407,408,38,331,409,39],"33896346",33,"gok",[411,412,39,235,413,39],"33897121",34,"KKKKkkkk1",[129,415,41,416,130,39],35,"project\u002Fcritical\u002Fidea",[418,415,44,267,419,39],"33895194","fastball",[421,415,39,422,130,39],"33895945","commenter\u002Fneutral\u002Fperformance",[424,415,39,422,425,39],"33897376","Jayakumark",[173,427,38,331,174,39],36,[429,427,39,331,11,41],"33895849",[431,432,44,275,433,39],"33896502",37,"dicroce",[435,436,44,437,438,39],"33892775",38,"maker\u002Fpositive\u002Fnone","anatoly",[440,441,44,437,442,39],"33892949",39,"checker659",[444,445,38,270,446,39],"33897007",40,"2devnull",[448,449,44,437,450,39],"33895265",41,"ripperdoc",[55,452,39,289,11,41],42,[142,454,39,455,143,39],43,"commenter\u002Fcritical\u002Fdocs",[457,454,39,231,458,39],"33899259","MaxBarraclough",[460,454,39,231,143,39],"33899798",[462,454,39,463,464,39],"33900339","commenter\u002Fneutral\u002Fdocs","kgwgk",[466,454,39,455,143,39],"33915543",[468,454,39,463,464,39],"33918767",[470,454,39,231,471,39],"33901045","mh7",[473,454,39,231,143,39],"33902274",[475,476,41,416,477,39],"33892510",44,"xchkr1337",[479,476,39,231,480,39],"33893141","ducktective",[482,476,39,235,483,39],"33894393","allisdust",[485,476,39,486,64,39],"33896972","commenter\u002Fcritical\u002Fnone",[],[489,494,499,503,507,512],{"id":490,"name":491,"points":492,"category":493},"33833836","SQL for Devs",865,"learning_science",{"id":495,"name":496,"points":497,"category":498},"33853773","chat-gpt-google-extension",431,"dev_tools",{"id":500,"name":501,"points":502,"category":20},"33882497","SmartyNames",313,{"id":504,"name":505,"points":506,"category":493},"33815146","NLP Demystified",286,{"id":508,"name":509,"points":510,"category":511},"33842406","Wikiwho",261,"data",{"id":513,"name":514,"points":515,"category":516},"33845946","This Hacker News Does Not Exist",252,"creative",false,"v2",{"id":520,"name":5,"line":6,"ai":517,"cat":20,"year":13,"points":14,"comments":15,"rec":29,"s":521,"top":18,"kw":522,"maker":11,"domain":10,"tier":19,"attempts":39,"strip":523,"url":9,"mix":524,"praise":96,"objection":126,"sub":21},33877893,0.46,[24,25,26,27,28],"5053553535555330535353555335515355500010",[32,33,34,18],1793991056085]