[{"data":1,"prerenderedAt":468},["ShallowReactive",2],{"project-27523014":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":35,"aspects":36,"strip":44,"post_text":46,"what":47,"said":72,"maker_in_thread":195,"shape":204,"grid":208,"attempts":430,"same_week":431,"maker_shows":38,"maker_hidden":460,"reader_version":461,"card":462},"27523014","fcp","A significantly faster alternative to cp(1), written in Rust","title","Show HN: fcp – A significantly faster alternative to cp(1), written in Rust","https:\u002F\u002Fgithub.com\u002FSvetlitski\u002Ffcp","github.com","Svetlitski",1623800182,2021,88,86,589,2533,24,"notable","dev_tools","file copy utility","cli",[24,25,26,27],"cp alternative","parallel file copy","rust cli","copy-on-write","curious",-0.297,{"positive":31,"neutral":32,"mixed":33,"critical":34},2,23,0,13,38,{"performance":37,"docs":40,"idea":41,"execution":43},{"positive":38,"critical":39},1,8,{"positive":33,"critical":33},{"positive":38,"critical":42},3,{"positive":33,"critical":33},[42,42,38,38,38,42,42,42,42,42,38,42,42,33,33,33,38,38,33,33,33,33,38,33,38,42,42,42,42,33,33,33,33,38,45,38,38,42,42,33],5,null,{"problem":48,"audience":46,"pricing":46,"motivation":55,"time_to_build":46,"stack":59,"openness":70,"stage":71,"team_size":71},{"value":49,"source":50,"evidence":51},"Copying large files and directories is slow with classic cp","maker",[52],{"id":53,"by":11,"text":54},"27523086","Inspired by many of the amazing new-wave Unix tools that have come about over the past several years (fzf, ripgrep, fd, lsd, etc.), I set out to create a more modern and…",{"value":56,"source":50,"evidence":57},"Build a more modern and performant cp, inspired by new Unix tools",[58],{"id":53,"by":11,"text":54},[60,64],{"value":61,"source":50,"evidence":62},"Rust",[63],{"id":53,"by":11,"text":54},{"value":65,"source":50,"evidence":66},"macOS fclonefileat and fcopyfile syscalls",[67],{"id":68,"by":11,"text":69},"27523491","The primary reason is that it uses all of your computer's available cores to walk through directories and copy files, effectively speeding things up by a factor directly…","open_source","unknown",{"praise":73,"objections":82,"requests":122,"questions":147,"alternatives":169},[74],{"value":75,"source":76,"evidence":77},"Large speedup over cp on NFS and some benchmarks","commenter",[78],{"id":79,"by":80,"text":81},"27598776","virtuallynathan","I just tried this going between my 2 servers with NFS, as compared to cp, rsync, and rclone. This tool wins by >2x. Great work! ~1.5Gbps vs ~3-4Gbps.",[83,94,104,111],{"value":84,"source":76,"evidence":85},"Multi-threaded copying is slower on spinning hard drives",[86,90],{"id":87,"by":88,"text":89},"27526315","the8472","It looks like this is using multiple threads to copy files. That's fine for NVMe which tends to achieve more throughput with higher queue depths. But it will degrade performance…",{"id":91,"by":92,"text":93},"27527429","idoubtit","My experiments confirms your analyze. On a SATA hard drive using ext4, I copied a directory with a large hierarchy of file and 216MB overall. After 2 warm-ups, I launch this…",{"value":95,"source":76,"evidence":96},"Lacks most of cp's options",[97,101],{"id":98,"by":99,"text":100},"27527284","formerly_proven","For SSDs this is probably the right thing (maybe use uring tho?) For hard drives this is the opposite of what you want. For hard drives, especially when copying large trees, you…",{"id":102,"by":11,"text":103},"27529469","Definitely valid points, as I mentioned in a comment below I'll be updating the README to mention this is geared towards systems with SSDs. As for lacking most of cp(1)'s…",{"value":105,"source":76,"evidence":106},"Saturating CPU with threads is undesirable for utility tools",[107],{"id":108,"by":109,"text":110},"27531248","kccqzy","Why do people think it is acceptable for utility programs like file copies to launch many threads and saturate my CPU? It's usually the case that there are many things being done…",{"value":112,"source":76,"evidence":113},"Speed claim based on core count is misleading since disk IO is the bottleneck",[114,118],{"id":115,"by":116,"text":117},"27532826","jjav","> The primary reason is that it uses all of your computer's available cores to walk through directories and copy files, effectively speeding things up by a factor directly…",{"id":119,"by":120,"text":121},"27526023","gregoriol","The only thing I'd like \"new\" in cp is an optional progress indicator, for large files Btw, copying files is mostly bound by disk speed (or network if share), not so much by CPU,…",[123,130,137,143],{"value":124,"source":76,"evidence":125},"Benchmarks on other filesystems",[126],{"id":127,"by":128,"text":129},"27523044","cratermoon","What filesystem on Linux? Show us some benchmarks on other filesystems.",{"value":131,"source":76,"evidence":132},"Man page documentation",[133],{"id":134,"by":135,"text":136},"27525512","nextaccountic","My only gripe with those new unix tools is that many of them don't come with a man page, even though they have essentially the same information elsewhere So, perhaps, take a look…",{"value":138,"source":76,"evidence":139},"Support for reflinks and deduplication",[140],{"id":141,"by":135,"text":142},"27525488","Does this support reflinks on filesystems that support deduplication? (like cp --reflink)",{"value":144,"source":76,"evidence":145},"Progress indicator for large files",[146],{"id":119,"by":120,"text":121},[148,155,162],{"value":149,"source":76,"evidence":150},"Why is it faster than cp?",[151],{"id":152,"by":153,"text":154},"27523454","jepler","_Why_ is it faster? (particularly on Linux, I see you're using syscalls on mac that mac's cp doesn't, since apple doesn't really maintain their unix userspace)",{"value":156,"source":76,"evidence":157},"Does it sacrifice fsync durability?",[158],{"id":159,"by":160,"text":161},"27527364","mnw21cam","Does this software gain any speed by sacrificing any safety? As in, does it switch off fsync in any way? Are the files guaranteed durable by the time the software terminates?",{"value":163,"source":76,"evidence":164},"Which filesystems are supported?",[165],{"id":166,"by":167,"text":168},"27537662","slava_kiose","What file systems are supported here?",[170,178,182,188],{"value":171,"source":76,"evidence":172},"GNU cp with reflink options",[173,177],{"id":174,"by":175,"text":176},"27546890","kazinator","> The massive difference in performance in this case is due to fcp using fclonefileat and fcopyfile under the hood GNU Coreutils' cp, the thing you use on most GNU\u002FLinux systems,…",{"id":141,"by":135,"text":142},{"value":179,"source":76,"evidence":180},"rsync, rclone",[181],{"id":79,"by":80,"text":81},{"value":183,"source":50,"evidence":184},"uutils coreutils rewrite",[185],{"id":186,"by":11,"text":187},"27529424","I would argue that \"the easy parts\" of cp account for a very significant fraction of its usage (i.e. copy some files\u002Fdirectories without any other options). The idea with fcp is…",{"value":189,"source":76,"evidence":190},"wcp (io_uring based copy tool)",[191],{"id":192,"by":193,"text":194},"27527444","wheybags","Not a question, but I made a similar tool in c++[1], for Linux only using io_uring, and a blog post explaining its internals [2]. I'll definitely have a look some time soon, I'd…",{"comments":196,"answered":197,"key":198},10,7,[199,200,201],{"id":53,"text":54},{"id":68,"text":69},{"id":202,"text":203},"27529515","No, it is not. I would've submitted a Homebrew formula for it, but that would go directly against Homebrew's guidelines, which frown upon authors submitting formulas for their…",{"commenters":205,"max_depth":39,"maker_replies":196,"first_comment_mins":206,"lifespan_hours":207},57,4,168,[209,211,213,215,219,220,224,227,229,230,231,234,236,239,242,245,248,251,254,257,258,261,263,264,267,269,271,275,278,281,284,287,290,292,295,297,298,299,302,306,309,312,315,318,320,322,324,326,328,331,334,337,339,341,344,348,352,354,356,357,360,363,365,368,370,372,375,377,379,382,385,387,389,390,392,397,398,402,405,408,412,415,417,421,424,427],[53,33,33,210,11,38],"maker\u002Fneutral\u002Fnone",[152,33,42,212,153,33],"project\u002Fneutral\u002Fperformance",[68,33,33,214,11,38],"maker\u002Fpositive\u002Fperformance",[216,33,42,217,218,33],"27524914","project\u002Fneutral\u002Fnone","CalChris",[202,33,33,210,11,38],[221,33,38,222,223,33],"27524942","project\u002Fcritical\u002Fperformance","fulafel",[225,33,42,212,226,33],"27526151","yokaze",[228,33,38,222,223,33],"27536072",[159,33,42,217,160,33],[115,33,38,222,116,33],[232,33,42,212,233,33],"27526209","Siira",[134,33,42,235,135,33],"project\u002Fneutral\u002Fdocs",[237,33,42,235,238,33],"27527306","drran",[240,33,42,235,241,33],"27527971","attractivechaos",[243,33,33,244,238,33],"27530710","commenter\u002Fneutral\u002Fnone",[246,33,33,244,247,33],"27534412","selfhoster11",[249,33,42,235,250,33],"27527530","Y_Y",[252,33,42,212,253,33],"27526041","nicoburns",[255,33,42,217,256,33],"27526423","robthebrew",[192,33,42,212,193,33],[259,33,33,260,11,38],"27529358","maker\u002Fneutral\u002Fexecution",[262,33,42,212,193,33],"27531332",[98,38,38,222,99,33],[265,38,42,212,266,33],"27527682","vthriller",[268,38,33,244,99,33],"27528279",[270,38,42,212,88,33],"27531590",[272,38,33,273,274,33],"27528531","other\u002Fneutral\u002Fnone","CWuestefeld",[276,38,33,273,277,33],"27528567","duped",[279,38,33,273,280,33],"27528568","amluto",[282,38,33,273,283,33],"27528614","lvkv",[285,38,33,273,286,33],"27529517","andrewmcwatters",[288,38,33,273,289,33],"27531328","unknown_error",[291,38,33,273,286,33],"27533398",[293,38,33,273,294,33],"27529357","dekhn",[102,38,33,296,11,38],"maker\u002Fneutral\u002Fperformance",[87,31,38,222,88,33],[91,31,38,222,92,33],[300,31,42,212,301,33],"27527492","kwdc",[303,31,33,304,305,33],"27527563","commenter\u002Fneutral\u002Fperformance","hpb42",[307,31,33,244,308,33],"27528120","snet0",[310,31,33,244,311,33],"27530672","juancampa",[313,31,33,273,314,33],"27527650","RicoElectrico",[316,31,33,273,317,33],"27527906","AndrewStephens",[319,31,33,273,289,33],"27531382",[321,31,33,273,317,33],"27531496",[323,31,33,273,289,33],"27561082",[325,31,33,296,11,38],"27529320",[119,42,38,327,120,33],"project\u002Fcritical\u002Fidea",[329,42,38,330,250,33],"27527543","project\u002Fcritical\u002Fnone",[332,42,33,273,333,33],"27527831","_joel",[335,42,33,273,336,33],"27528528","zajio1am",[338,42,33,273,333,33],"27528673",[340,42,38,330,247,33],"27534461",[342,206,38,327,343,33],"27528109","hprotagonist",[345,206,45,346,347,33],"27528143","project\u002Fpositive\u002Fidea","hexo",[349,206,42,350,351,33],"27528554","project\u002Fneutral\u002Fidea","carlmr",[353,206,33,273,343,33],"27529361",[186,206,33,355,11,38],"maker\u002Fneutral\u002Fidea",[141,45,42,217,135,33],[358,45,42,217,359,33],"27531576","CUViper",[127,361,42,362,128,33],6,"project\u002Fneutral\u002Fexecution",[364,361,33,260,11,38],"27523120",[366,361,33,244,367,33],"27523216","beebeepka",[369,361,33,304,11,38],"27523516",[371,361,33,304,367,33],"27523719",[373,361,33,374,128,33],"27524062","other\u002Fneutral\u002Fperformance",[376,361,33,374,88,33],"27526561",[378,361,33,374,128,33],"27531399",[380,361,33,304,381,33],"27526120","pizza234",[383,361,33,374,384,33],"27527355","celrod",[386,361,33,374,381,33],"27527417",[388,361,33,260,11,38],"27529790",[108,197,38,222,109,33],[79,39,45,391,80,33],"project\u002Fpositive\u002Fperformance",[393,394,33,395,396,33],"27528467",9,"other\u002Fneutral\u002Fidea","tyingq",[174,196,38,327,175,33],[399,400,42,350,401,33],"27528740",11,"michael-ax",[403,404,33,273,294,33],"27529757",12,[406,34,38,222,407,33],"27549455","qwerty456127",[409,410,42,217,411,33],"27526530",14,"bb88",[413,410,33,244,414,33],"27527677","tialaramex",[166,416,42,235,167,33],15,[418,419,33,395,420,33],"27525008",16,"qolop",[422,419,33,395,423,33],"27525254","ViViDboarder",[425,419,33,395,426,33],"27527549","411111111111111",[428,419,33,395,429,33],"27528286","Klasiaster",[],[432,437,442,447,452,456],{"id":433,"name":434,"points":435,"category":436},"27503597","RoutineDB",396,"consumer_social",{"id":438,"name":439,"points":440,"category":441},"27520884","Influence",379,"games",{"id":443,"name":444,"points":445,"category":446},"27481465","LANDrop",250,"infrastructure",{"id":448,"name":449,"points":450,"category":451},"27459664","Datablocks",188,"data",{"id":453,"name":454,"points":455,"category":436},"27507645","HNReader",87,{"id":457,"name":458,"points":459,"category":20},"27509682","Rarchy Visual Sitemap Generator",65,false,"v2",{"id":463,"name":5,"line":6,"ai":460,"cat":20,"year":13,"points":14,"comments":15,"rec":28,"s":464,"top":18,"kw":465,"maker":11,"domain":10,"tier":19,"attempts":33,"strip":466,"url":9,"mix":467,"praise":75,"objection":84,"sub":21},27523014,-0.3,[24,25,26,27],"3311133333133000110000101333300001511330",[31,32,33,34],1794036608026]