[{"data":1,"prerenderedAt":414},["ShallowReactive",2],{"project-34265206":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":29,"sentiment":30,"stance_mix":31,"scored_comments":36,"aspects":37,"strip":46,"post_text":48,"what":49,"said":70,"maker_in_thread":181,"shape":191,"grid":195,"attempts":376,"same_week":377,"maker_shows":18,"maker_hidden":406,"reader_version":407,"card":408},"34265206","ClickHouse-local","a small tool for serverless data analytics","title","Show HN: ClickHouse-local – a small tool for serverless data analytics","https:\u002F\u002Fclickhouse.com\u002Fblog\u002Fextracting-converting-querying-local-files-with-sql-clickhouse-local","clickhouse.com","zX41ZdbW",1672947048,2023,289,70,159,3109,6,"legend","data","embedded sql engine","cli",[24,25,26,27,28],"sql","clickhouse","data analytics","csv","serverless database","mixed",0.076,{"positive":32,"neutral":33,"mixed":34,"critical":35},9,17,3,8,37,{"performance":38,"design":41,"idea":42,"docs":44,"execution":45},{"positive":39,"critical":40},1,0,{"positive":40,"critical":34},{"positive":43,"critical":39},4,{"positive":40,"critical":40},{"positive":40,"critical":34},[34,39,47,47,47,47,34,39,34,34,34,34,34,34,47,34,34,47,47,39,39,34,34,40,47,40,39,34,40,39,40,40,40,40,34,40,40,40,40,40],5,"Me with my friend Vitaly Ludvichenko made an experiment to combine ClickHouse server and client to make a self-contained program running a database engine and processing data without a server: https:\u002F\u002Fgithub.com\u002FClickHouse\u002FClickHouse\u002Fpull\u002F150\n\nDevelopment continued in the past 6 years, and now clickhouse-local becomes a swiss-army knife for data processing. Say \"ffmpeg\" for datasets and more.\n\nIt can resemble textql, octosql, dsq, duckdb, trdsql, q, datafusion-cli, spyql, but has better capabilities and performance.\n\nHere is a tutorial: https:\u002F\u002Fclickhouse.com\u002Fblog\u002Fextracting-converting-queryin",{"problem":50,"audience":56,"pricing":56,"motivation":57,"time_to_build":56,"stack":61,"openness":67,"stage":68,"team_size":69},{"value":51,"source":52,"evidence":53},"Query and convert data files with SQL without running a database server","maker",[54],{"id":55},"post",null,{"value":58,"source":52,"evidence":59},"Experiment to combine ClickHouse server and client into a self-contained program",[60],{"id":55},[62],{"value":63,"source":64,"evidence":65},"C++","inferred",[66],{"id":55},"open_source","launched","company",{"praise":71,"objections":95,"requests":123,"questions":146,"alternatives":162},[72,84],{"value":73,"source":74,"evidence":75},"Fast and easy to use for log and CSV analysis","commenter",[76,80],{"id":77,"by":78,"text":79},"34271425","linsomniac","I recently had a few months of AWS Cloudfront CSV logs I needed to pull some analytics from, and I ended up using clickhouse-local to do the analysis and was just blown away by…",{"id":81,"by":82,"text":83},"34272301","glogla","The memory aspect is pretty interesting. I tried to play with \"serverless ELT\" (the other kind of serverless) where I would define AWS Lambda that turns incoming CSV files into…",{"value":85,"source":74,"evidence":86},"Useful Lambda and CLI data use cases",[87,91],{"id":88,"by":89,"text":90},"34267601","cube2222","Congrats on the Show HN! It's great to see more tools in this area (querying data from various sources in-place) and the Lambda use case is a really cool idea! I've recently done…",{"id":92,"by":93,"text":94},"34270373","devdiary","So cool. Similarly, can we also build clickhouse-remote?",[96,106,116],{"value":97,"source":74,"evidence":98},"No Windows support",[99,103],{"id":100,"by":101,"text":102},"34272855","POPOSYS","Is there some way to use this on Windows \u002F MinGW shells? ``` $ curl https:\u002F\u002Fclickhouse.com\u002F | sh % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload…",{"id":104,"by":101,"text":105},"34273121","Looks like no Windows support for the whole clickhose product family! This is actually great as it gives me one more reason to ask for a real operating system! Now with…",{"value":107,"source":52,"evidence":108},"No library or API bindings, CLI only",[109,112],{"id":110,"by":11,"text":111},"34266721","It is not available as a library, only CLI.",{"id":113,"by":114,"text":115},"34267802","winrid","Ah, okay. Won't work for my use case. Still cool though!",{"value":117,"source":74,"evidence":118},"Joins may exhaust memory",[119],{"id":120,"by":121,"text":122},"34268877","twotwotwo","Huh! I recently used this to play with CH, and had no idea it was so new! Also, I've been impressed with the range of formats\u002Fprotocols\u002Ftypes of storage CH can use, and it's a…",[124,131,139],{"value":125,"source":74,"evidence":126},"Browser or WASM version",[127],{"id":128,"by":129,"text":130},"34267782","varunkmohan","Any plans to be able to directly run this in the browser similar to DuckDB. Nice to see more options in the space.",{"value":132,"source":74,"evidence":133},"npm or Python bindings",[134,138],{"id":135,"by":136,"text":137},"34270192","noduerme","This is really beautiful! I can think of a ton of use cases; so often I'm writing scraps of code or setting up quick DBs just to parse some specific dataset. I have a couple…",{"id":110,"by":11,"text":111},{"value":140,"source":74,"evidence":141},"Easier S3 credential configuration",[142],{"id":143,"by":144,"text":145},"34273499","denvrede","Do you know if it's possible to tell clickhouse-local (without the need of a config file) to use environment variables for AWS authentication? I know there is the…",[147,158],{"value":148,"source":74,"evidence":149},"Can it read files from AWS S3?",[150,154],{"id":151,"by":152,"text":153},"34272325","whinvik","Can Clickhouse local query from files on AWS S3? We recently had some issues querying parquet files on AWS S3. I tried benchmarking various tools and DuckDB seemed the most…",{"id":155,"by":156,"text":157},"34272647","qoega","Sure you can read from S3. You can define table that will look at your files like this https:\u002F\u002Fclickhouse.com\u002Fdocs\u002Fen\u002Fintegrations\u002Fs3\u002Fs3-table-engine\u002F Or just use s3 table…",{"value":159,"source":74,"evidence":160},"How are column types inferred?",[161],{"id":135,"by":136,"text":137},[163,173,177],{"value":164,"source":74,"evidence":165},"DuckDB",[166,170],{"id":167,"by":168,"text":169},"34270354","benjaminwootton","I wonder how hard it would be to put this in process to compete with DuckDB?",{"id":171,"by":11,"text":172},"34266172","Here is one example: https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002Fdcmoura\u002Fspyql\u002Fblob\u002Fmaster\u002Fnotebooks\u002Fjson_benchmark.ipynb Note: DuckDB is not present in this comparison, because it…",{"value":174,"source":74,"evidence":175},"OctoSQL",[176],{"id":88,"by":89,"text":90},{"value":178,"source":52,"evidence":179},"textql, dsq, q, trdsql",[180],{"id":55},{"comments":182,"answered":35,"key":183},10,[184,187,188],{"id":185,"text":186},"34268107","This is not what we want. I admit this problem. Changes will be made. Just for amusement, here is the 2016 website version: https:\u002F\u002Fpresentations.clickhouse.com\u002Foriginal_website\u002F",{"id":110,"text":111},{"id":189,"text":190},"34270576","If you downloaded `clickhouse-local`, you can simply rename it to `clickhouse-server` and run. It will automatically transform itself to ClickHouse server.",{"commenters":192,"max_depth":35,"maker_replies":182,"first_comment_mins":193,"lifespan_hours":194},40,29,75,[196,198,202,205,207,212,216,220,223,227,230,231,234,236,237,241,245,248,250,251,253,257,258,260,263,264,268,271,272,273,276,279,281,283,284,288,290,294,297,299,302,305,307,310,313,315,319,322,325,330,331,334,336,338,339,341,343,345,347,349,351,353,357,360,363,365,367,369,372,374],[88,40,34,197,89,40],"project\u002Fmixed\u002Fperformance",[199,40,34,200,201,40],"34268007","project\u002Fneutral\u002Fperformance","justinsaccount",[203,39,39,204,82,40],"34267955","project\u002Fcritical\u002Fdesign",[185,39,40,206,11,39],"maker\u002Fneutral\u002Fdesign",[208,209,47,210,211,40],"34272856",2,"project\u002Fpositive\u002Fnone","robertlagrant",[213,34,34,214,215,40],"34267510","project\u002Fneutral\u002Fidea","vgt",[217,34,47,218,219,40],"34268340","project\u002Fpositive\u002Fidea","ignoramous",[221,34,47,218,222,40],"34269841","LudwigNagasena",[224,34,40,225,226,40],"34279856","commenter\u002Fneutral\u002Fnone","mhuffman",[228,34,47,218,229,40],"34271399","stingraycharles",[120,43,34,197,121,40],[232,43,34,233,121,40],"34269097","project\u002Fmixed\u002Fdocs",[100,47,39,235,101,40],"project\u002Fcritical\u002Fexecution",[104,47,39,235,101,40],[238,47,34,239,240,40],"34273939","project\u002Fneutral\u002Fexecution","e12e",[242,47,40,243,244,40],"34292151","other\u002Fneutral\u002Fnone","hodgesrm",[246,47,34,239,247,40],"34274808","me_bx",[249,18,34,214,114,40],"34266448",[110,18,40,214,11,39],[113,18,39,252,114,40],"project\u002Fcritical\u002Fidea",[254,255,47,210,256,40],"34272984",7,"dinedal",[135,35,34,239,136,40],[259,32,34,200,226,40],"34277223",[261,32,40,262,89,40],"34279244","other\u002Fneutral\u002Fperformance",[128,182,34,214,129,40],[265,266,47,210,267,40],"34268751",11,"gigatexal",[151,269,34,270,152,40],12,"project\u002Fneutral\u002Fdocs",[155,269,34,270,156,40],[143,269,34,270,144,40],[77,274,47,275,78,40],13,"project\u002Fpositive\u002Fperformance",[277,274,40,225,278,40],"34285168","mr-karan",[167,280,34,214,168,40],14,[92,282,47,218,93,40],15,[189,282,40,239,11,39],[285,286,39,204,287,40],"34269849",16,"orf",[289,286,39,235,240,40],"34274021",[291,33,34,292,293,40],"34266117","project\u002Fneutral\u002Fdesign","notamy",[295,33,40,225,296,40],"34266194","afandian",[298,33,40,225,293,40],"34266274",[300,33,34,292,301,40],"34266423","slt2021",[303,33,40,225,304,40],"34267576","hk__2",[306,33,40,239,11,39],"34266287",[308,33,47,309,293,40],"34266511","maker\u002Fpositive\u002Fnone",[311,33,40,225,312,40],"34267582","CoolCold",[314,33,40,225,11,39],"34268312",[316,33,34,317,318,40],"34269843","maker\u002Fneutral\u002Fnone","cheeaun",[320,33,39,204,321,40],"34266693","goodpoint",[323,33,40,225,324,40],"34272589","mananaysiempre",[326,327,34,328,329,40],"34265896",18,"maker\u002Fneutral\u002Fperformance","lazzlazzlazz",[171,327,40,200,11,39],[332,327,39,333,215,40],"34266885","project\u002Fcritical\u002Fnone",[335,327,40,200,11,39],"34267144",[337,327,40,225,215,40],"34267286",[81,327,40,262,82,40],[340,327,40,262,101,40],"34272802",[342,327,40,200,11,39],"34267192",[344,327,40,225,215,40],"34268000",[346,327,40,225,11,39],"34268192",[348,327,40,225,215,40],"34268429",[350,327,40,210,11,39],"34268971",[352,327,34,317,219,40],"34268254",[354,355,40,243,356,40],"34265633",19,"swyx",[358,355,40,243,359,40],"34265703","maxbond",[361,355,40,243,362,40],"34266338","RyanHamilton",[364,355,40,243,114,40],"34266507",[366,355,40,243,219,40],"34267994",[368,355,40,243,114,40],"34268744",[370,355,40,371,356,40],"34304192","other\u002Fneutral\u002Fidea",[373,355,40,225,356,40],"34272949",[375,355,40,225,219,40],"34294685",[],[378,383,388,393,398,402],{"id":379,"name":380,"points":381,"category":382},"34338629","Kandria",487,"games",{"id":384,"name":385,"points":386,"category":387},"34289581","Futurecoder",371,"learning_science",{"id":389,"name":390,"points":391,"category":392},"34279062","Tablane",352,"productivity",{"id":394,"name":395,"points":396,"category":397},"34343122","Val Town",332,"dev_tools",{"id":399,"name":400,"points":401,"category":397},"34262825","Vim online editor",316,{"id":403,"name":404,"points":405,"category":397},"34320844","Ecode",233,false,"v2",{"id":409,"name":5,"line":6,"ai":406,"cat":20,"year":13,"points":14,"comments":15,"rec":29,"s":410,"top":18,"kw":411,"maker":11,"domain":10,"tier":19,"attempts":40,"strip":412,"url":9,"mix":413,"praise":73,"objection":97,"sub":21},34265206,0.08,[24,25,26,27,28],"3155553133333353355113305013010000300000",[32,33,34,35],1794036629634]