[{"data":1,"prerenderedAt":344},["ShallowReactive",2],{"project-34434249":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":27,"sentiment":28,"stance_mix":29,"scored_comments":34,"aspects":35,"strip":40,"post_text":43,"what":44,"said":91,"maker_in_thread":207,"shape":212,"grid":216,"attempts":305,"same_week":306,"maker_shows":41,"maker_hidden":336,"reader_version":337,"card":338},"34434249","Unblob","extraction suite for 30+ file formats","title","Show HN: Unblob – extraction suite for 30+ file formats","https:\u002F\u002Fgithub.com\u002Fonekey-sec\u002Funblob","github.com","kissgyorgy",1674082272,2023,240,42,207,3109,7,"notable","dev_tools","file extraction","cli",[21,24,25,26],"firmware analysis","binwalk alternative","file carving","loved",0.287,{"positive":30,"neutral":31,"mixed":32,"critical":33},6,13,1,0,20,{"design":36,"idea":37,"docs":39},{"positive":33,"critical":33},{"positive":38,"critical":33},4,{"positive":33,"critical":33},[41,42,33,33,33,33,41,33,41,33,42,33,42,33,33,42,33,42,33,41,41,33,33,33,33,42,33,33,33,42,42,42,33,42,33,42,42,41,42,42],5,3,null,{"problem":45,"audience":56,"pricing":43,"motivation":70,"time_to_build":43,"stack":76,"openness":88,"stage":89,"team_size":90},{"value":46,"source":47,"evidence":48},"Finding and extracting valid archive and format chunks within arbitrary files such as firmware images, where binwalk is noisy and slow","maker",[49,52],{"id":50,"by":11,"text":51},"34435380","The main difference is that binwalk goes through a file linearly, searching for patterns like magic bytes, and tries to extract everything it finds. The problems with this: -…",{"id":53,"by":54,"text":55},"34439036","qkaiser","The atool suite is great but only supports well formatted files. The idea with unblob is to precisely identify valid chunks of data within arbitrary files, carve them out, then…",{"value":57,"source":58,"evidence":59},"Firmware and embedded systems analysts, reverse engineers, forensic and data recovery users","commenter",[60,64,67],{"id":61,"by":62,"text":63},"34435263","wrayjustin","Awesome tool\u002Fproject. These blind (or \"magic\"; https:\u002F\u002Fen.m.wikipedia.org\u002Fwiki\u002FList_of_file_signatures) extractors can be handy is a very wide range of applications. From…",{"id":65,"by":11,"text":66},"34435405","Yes, we developed a framework \"unblob core\" which can be extended easily [1] to any use-case. For example, a separate Python package could contain format specifications for…",{"id":68,"by":62,"text":69},"34435244","Adding a comparison, or at least the clear differentiators, to the documentation would be very helpful. As someone who uses `binwalk` extensively in a professional setting, with…",{"value":71,"source":47,"evidence":72},"binwalk was not a good fit for their needs, so they built a smarter alternative",[73],{"id":74,"by":11,"text":75},"34434986","Indeed, it's a smarter alternative for binwalk, which we started because binwalk was not a good fit for us :) Should probably include comparison somewhere in the docs.",[77,84],{"value":78,"source":47,"evidence":79},"Python",[80,81],{"id":65,"by":11,"text":66},{"id":82,"by":11,"text":83},"34435431","There are multiple reasons: 1. it already supports more formats [1] than atool. 2. It's written in Python and easily extendable by anyone interested supporting their own formats.…",{"value":85,"source":47,"evidence":86},"Hyperscan",[87],{"id":50,"by":11,"text":51},"open_source","launched","unknown",{"praise":92,"objections":110,"requests":131,"questions":150,"alternatives":169},[93,105],{"value":94,"source":58,"evidence":95},"Useful tool and a smarter take than binwalk",[96,100,101],{"id":97,"by":98,"text":99},"34434902","alschwalm","Looks nice! Kind of reminds me of binwalk: https:\u002F\u002Fgithub.com\u002FReFirmLabs\u002Fbinwalk",{"id":61,"by":62,"text":63},{"id":102,"by":103,"text":104},"34435532","arcticbull","Very cool - something I don't expect to need very often but likely critical when I do. I can't help but feel there was a missed opportunity to name it 'unpackman'",{"value":106,"source":47,"evidence":107},"Extensible handler framework",[108,109],{"id":65,"by":11,"text":66},{"id":82,"by":11,"text":83},[111,121],{"value":112,"source":58,"evidence":113},"No support for passwords on archive extractors",[114,118],{"id":115,"by":116,"text":117},"34437507","phoyd","I am not seeing an option to pass a password to the archive extractors. Is the any?",{"id":119,"by":11,"text":120},"34437560","No, not yet, however the problem came up several times. Feel free to create a GitHub issue if you would like this feature.",{"value":122,"source":58,"evidence":123},"Does not build on aarch64 or non-Linux systems",[124,128],{"id":125,"by":126,"text":127},"34436684","fellowmartian","This looks awesome! Looking forward to trying it over binwalk! I'd be great if you could get it building on aarch64 and non-Linux system, I've tried adding the flake to my nix…",{"id":129,"by":11,"text":130},"34437185","hyperscan is supported on Intel 64 bit only, but there is another project wchich supports ARM called vectorscan. My colleague wrote a Python wrapper for vectorscan:…",[132,140],{"value":133,"source":58,"evidence":134},"Comparison with binwalk in the docs",[135,136],{"id":68,"by":62,"text":69},{"id":137,"by":138,"text":139},"34435739","infotogivenm","+1 on a comparison. Binwalk has served me faithfully for years but you never know what you don’t know until the day it fails",{"value":141,"source":58,"evidence":142},"Ability to rename top-level directory when extracting",[143,147],{"id":144,"by":145,"text":146},"34438936","amelius","Can it rename the toplevel directory name before extracting? (My main problem with tar)",{"id":148,"by":145,"text":149},"34439049","No, sorry for being unclear, I mean if the archive contains a toplevel folder e.g. binutils-1.0.0, that I can rename it to binutils before extracting (so I don't end up with the…",[151,162],{"value":152,"source":58,"evidence":153},"How does it differ from aunpack or atool",[154,158],{"id":155,"by":156,"text":157},"34435235","alanbernstein","Why use this over aunpack? https:\u002F\u002Flinux.die.net\u002Fman\u002F1\u002Faunpack",{"id":159,"by":160,"text":161},"34436508","mintplant","apack\u002Faunpack from the atool suite for me [0]. Funny how many solutions exist for this problem. Though I think Unblob is aiming more for binwalk's niche [1]. [0]…",{"value":163,"source":58,"evidence":164},"Can it handle media container formats like jpg, png, mp4",[165],{"id":166,"by":167,"text":168},"34437370","mixmastamyk","Thought it might help with media containers but I guess not. Can anyone recommend a good jpg png mp4 mkv parser lib in python?",[170,175,182,186,193,200],{"value":171,"source":58,"evidence":172},"binwalk",[173,174],{"id":97,"by":98,"text":99},{"id":50,"by":11,"text":51},{"value":176,"source":58,"evidence":177},"dtrx",[178],{"id":179,"by":180,"text":181},"34435544","kbd","For years and years I've used `dtrx` (\"do the right extraction\") (https:\u002F\u002Fgithub.com\u002Fdtrx-py\u002Fdtrx\u002F). Maybe I should switch to unblob? It looks like unblob has the right behavior…",{"value":183,"source":58,"evidence":184},"atool (aunpack)",[185],{"id":159,"by":160,"text":161},{"value":187,"source":58,"evidence":188},"unp",[189],{"id":190,"by":191,"text":192},"34435690","nikeee","I've been using unp (https:\u002F\u002Fmanpages.ubuntu.com\u002Fmanpages\u002Ffocal\u002Fman1\u002Funp.1.html), which is just a wrapper around standard cli tools for unpacking things (tar, xz, unzip, etc). It…",{"value":194,"source":58,"evidence":195},"bsdtar",[196],{"id":197,"by":198,"text":199},"34437251","JonathonW","If you’re looking for a general-purpose extractor for known, common archive formats, bsdtar is really nice these days; it’s libarchive-based and does way more than just tarballs…",{"value":201,"source":58,"evidence":202},"OFRAK",[203],{"id":204,"by":205,"text":206},"34436846","jstrieb","Unblob looks cool! If you're interested in something similar that can put things back together after you've modified them, check out OFRAK:…",{"comments":18,"answered":42,"key":208},[209,210,211],{"id":65,"text":66},{"id":74,"text":75},{"id":50,"text":51},{"commenters":213,"max_depth":30,"maker_replies":18,"first_comment_mins":214,"lifespan_hours":215},25,32,23,[217,219,221,223,224,226,228,231,233,235,238,240,241,243,245,247,249,251,252,253,256,258,262,265,267,270,272,274,276,278,280,282,283,286,288,291,293,297,300,302,303],[204,33,41,218,205,33],"project\u002Fpositive\u002Fnone",[179,32,42,220,180,33],"project\u002Fneutral\u002Fdesign",[190,32,33,222,191,33],"commenter\u002Fneutral\u002Fnone",[159,32,33,222,160,33],[53,32,33,225,54,33],"commenter\u002Fneutral\u002Fidea",[197,32,33,227,198,33],"other\u002Fneutral\u002Fnone",[61,229,41,230,62,33],2,"project\u002Fpositive\u002Fidea",[65,229,33,232,11,32],"maker\u002Fneutral\u002Fnone",[234,229,41,230,62,33],"34435477",[236,42,33,222,237,33],"34438907","scrollaway",[97,38,42,239,98,33],"project\u002Fneutral\u002Fidea",[74,38,33,232,11,32],[68,38,42,242,62,33],"project\u002Fneutral\u002Fdocs",[137,38,33,244,138,33],"commenter\u002Fneutral\u002Fdocs",[50,38,33,246,11,32],"project\u002Fneutral\u002Fperformance",[125,41,42,248,126,33],"project\u002Fmixed\u002Fnone",[129,41,33,250,11,32],"project\u002Fneutral\u002Fnone",[115,30,42,250,116,33],[119,30,33,232,11,32],[254,18,41,230,255,33],"34435806","enos_feedler",[102,257,41,230,103,33],8,[259,260,33,227,261,33],"34436970",9,"snthpy",[263,260,33,227,264,33],"34437338","phs318u",[266,260,33,227,261,33],"34445779",[268,260,33,227,269,33],"34437796","saulpw",[271,260,42,250,54,33],"34439041",[166,273,33,227,167,33],10,[275,273,33,232,11,32],"34437411",[277,273,33,227,167,33],"34437822",[144,279,42,250,145,33],11,[281,279,42,250,54,33],"34438996",[148,279,42,250,145,33],[284,279,33,227,285,33],"34439153","jenscow",[287,279,42,250,145,33],"34439427",[289,279,33,227,290,33],"34440378","nerdponx",[292,279,42,250,54,33],"34439731",[294,295,42,242,296,33],"34437323",12,"eigenvalue",[298,31,41,218,299,33],"34434601","Centigonal",[155,301,42,239,156,33],14,[82,301,33,230,11,32],[304,301,42,239,62,33],"34435464",[],[307,312,317,321,326,331],{"id":308,"name":309,"points":310,"category":311},"34416386","Plus",307,"web_saas",{"id":313,"name":314,"points":315,"category":316},"34400443","Sketch",252,"ai",{"id":318,"name":319,"points":320,"category":316},"34389041","neuralframes.com",247,{"id":322,"name":323,"points":324,"category":325},"34396026","Terra Firma",229,"games",{"id":327,"name":328,"points":329,"category":330},"34406340","Vento",224,"productivity",{"id":332,"name":333,"points":334,"category":335},"34407924","Otterkit",192,"languages",false,"v2",{"id":339,"name":5,"line":6,"ai":336,"cat":20,"year":13,"points":14,"comments":15,"rec":27,"s":340,"top":18,"kw":341,"maker":11,"domain":10,"tier":19,"attempts":33,"strip":342,"url":9,"mix":343,"praise":94,"objection":112,"sub":21},34434249,0.29,[21,24,25,26],"5300005050303003030550000300033303033533",[30,31,32,33],1793991058952]