[{"data":1,"prerenderedAt":214},["ShallowReactive",2],{"project-40990641":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":33,"aspects":34,"strip":35,"post_text":38,"what":39,"said":58,"maker_in_thread":125,"shape":134,"grid":139,"attempts":178,"same_week":179,"maker_shows":31,"maker_hidden":207,"reader_version":208,"card":209},"40990641","sqlite-bench","SQLite Transaction Benchmarking Tool","title","Show HN: SQLite Transaction Benchmarking Tool","https:\u002F\u002Fgithub.com\u002Fseddonm1\u002Fsqlite-bench","github.com","seddonm1",1721252682,2024,128,21,667,3515,19,"notable","dev_tools","benchmarking tool","cli",[24,25,26,27],"sqlite","begin concurrent","sqlite benchmark","transactions","quiet",null,{"positive":31,"neutral":31,"mixed":32,"critical":32},2,0,4,{},[36,32,37,32,32,32,36,37,32,32,32,32,32,32,32,32,32,32,32,32,32],5,3,"I wanted to make my own evaluation of what kind of performance I could expect from SQLite on a server and investigate the experimental `BEGIN CONCURRENT` branch vs the inbuilt `DEFERRED` and `IMMEDIATE` behaviors.\n\nExplanatory blog post: https:\u002F\u002Freorchestrate.com\u002Fposts\u002Fsqlite-transactions\u002F",{"problem":40,"audience":29,"pricing":29,"motivation":46,"time_to_build":29,"stack":50,"openness":57,"stage":57,"team_size":57},{"value":41,"source":42,"evidence":43},"Measuring what performance to expect from SQLite on a server under different transaction modes.","maker",[44],{"id":45},"post",{"value":47,"source":42,"evidence":48},"Wanted to evaluate SQLite performance on a server and test the experimental BEGIN CONCURRENT branch against DEFERRED and IMMEDIATE.",[49],{"id":45},[51],{"value":52,"source":42,"evidence":53},"Rust",[54],{"id":55,"by":11,"text":56},"40993559","Thanks. All good and valid questions. 1. I work mostly in Rust so I'll answer there in terms of async. This library [0] uses queues to manage workload. I run a modified version…","unknown",{"praise":59,"objections":76,"requests":86,"questions":87,"alternatives":102},[60],{"value":61,"source":62,"evidence":63},"Commenters found the SQLite concurrency discussion and the maker's answers useful.","commenter",[64,68,72],{"id":65,"by":66,"text":67},"40993447","leononame","Thanks for the interesting article. Lots of things seem to happen in SQLite land at the moment and I appreciate that the SQLite team documents their quirks so openly, it gives…",{"id":69,"by":70,"text":71},"40993730","pdimitar","Valuable info and links, instant bookmarks, thank you! If you don't mind me asking, why did you go with rusqlite + a tokio wrapper for it and not go with sqlx?",{"id":73,"by":74,"text":75},"40995791","matharmin","You get concurrency in SQLite by using multiple connections - and typically a dedicated thread per connection. When using async wrappers, a good solution is connection pooling…",[77],{"value":78,"source":62,"evidence":79},"Synchronous file I\u002FO can block application threads in SQLite async wrappers.",[80,83],{"id":81,"by":66,"text":82},"40994985","It's a made up example to clarify whether I understand potential congestion scenarios and limitations correctly, not my actual situation. If I had a server with 100 cores to…",{"id":84,"by":66,"text":85},"40998037","If synchronous IO is blocking your CPU bound application code, this won't help you. My made up example was not about concurrent writes, and the concurrent reads I mentioned were…",[],[88,97],{"value":89,"source":62,"evidence":90},"How many reader connections or threads are optimal relative to CPU count?",[91,94],{"id":92,"by":66,"text":93},"40993879","Thanks for your answer. For 1, what is a good n? More than NUM_CPU probably does not make sense, right? But would I want to keep it lower? Also, you dispatch transactions in your…",{"id":95,"by":11,"text":96},"40994197","I went through the same mental process as you and also use num_cpus [0] but this is based only on intuition that is likely wrong. More benchmarking is needed as my benchmarks…",{"value":98,"source":62,"evidence":99},"Whether to pool connections or open and close per request, and whether open handles block checkpointing.",[100,101],{"id":65,"by":66,"text":67},{"id":73,"by":74,"text":75},[103,110,117],{"value":104,"source":62,"evidence":105},"libsql (Turso) for Rust and Node async access to SQLite.",[106],{"id":107,"by":108,"text":109},"41001329","jitl","Libsql fork has good rust \u003C-> node async, you could look at them for inspiration. Maintained by Turso.",{"value":111,"source":62,"evidence":112},"Postgres for high-concurrency server workloads.",[113],{"id":114,"by":115,"text":116},"40994701","rmbyrro","If you have a server with 100 cores to serve 100 connections simultaneously - and really need this setup -, you should probably be using Postgres or smth else.",{"value":118,"source":62,"evidence":119},"sqlite_async for Dart and datasette's Python thread-pool wrapper.",[120,121],{"id":73,"by":74,"text":75},{"id":122,"by":123,"text":124},"40995986","simonw","I wrote an async wrapper around SQLite in Python - I'm using a thread pool: https:\u002F\u002Fgithub.com\u002Fsimonw\u002Fdatasette\u002Fblob\u002Fmain\u002Fdatasette\u002Fdatabase.py I have multiple threads for reads…",{"comments":33,"answered":33,"key":126},[127,128,131],{"id":55,"text":56},{"id":129,"text":130},"40994216","Whilst I love the idea of SQLX compile-time checked queries it is not always practical to need a database connection to compile the code in my experience. If it works for you…",{"id":132,"text":133},"41000219","SQLX has an offline mode where it saves the metadata of the SQL database structure but then you run into risk of that being out of sync with the database? Yeah I just drop this…",{"commenters":135,"max_depth":136,"maker_replies":33,"first_comment_mins":137,"lifespan_hours":138},8,7,608,69,[140,142,145,146,147,150,151,153,154,156,158,159,162,164,166,168,169,172,173,175,177],[65,32,36,141,66,32],"maker\u002Fpositive\u002Fnone",[55,32,32,143,11,144],"maker\u002Fneutral\u002Fnone",1,[69,32,37,143,70,32],[129,32,32,143,11,144],[148,32,32,149,70,32],"40994238","commenter\u002Fneutral\u002Fnone",[132,32,32,143,11,144],[152,32,36,141,70,32],"41000283",[92,32,37,143,66,32],[95,32,32,155,11,144],"maker\u002Fneutral\u002Fperformance",[114,32,32,157,115,32],"other\u002Fneutral\u002Fperformance",[81,32,32,157,66,32],[160,32,32,157,161,32],"40995782","Moehassan",[84,32,32,163,66,32],"commenter\u002Fneutral\u002Fperformance",[165,32,32,163,161,32],"41005312",[167,32,32,163,115,32],"41000233",[73,32,32,157,74,32],[170,32,32,171,70,32],"40999923","other\u002Fneutral\u002Fnone",[107,32,32,171,108,32],[174,32,32,171,70,32],"41014938",[176,32,32,171,108,32],"41018493",[122,32,32,171,123,32],[],[180,185,190,195,199,203],{"id":181,"name":182,"points":183,"category":184},"40956087","Resurrecting a dead Dune RTS game",485,"games",{"id":186,"name":187,"points":188,"category":189},"40972122","Horizon",361,"web_saas",{"id":191,"name":192,"points":193,"category":194},"40960508","Buzzinga",284,"consumer_social",{"id":196,"name":197,"points":198,"category":20},"40952509","CSS grid generator",262,{"id":200,"name":201,"points":202,"category":20},"40961101","Kaskade",164,{"id":204,"name":205,"points":206,"category":20},"40980715","Magic-cli",158,false,"v2",{"id":210,"name":5,"line":6,"ai":207,"cat":20,"year":13,"points":14,"comments":15,"rec":28,"s":29,"top":18,"kw":211,"maker":11,"domain":10,"tier":19,"attempts":32,"strip":212,"url":9,"mix":213,"praise":61,"objection":78,"sub":21},40990641,[24,25,26,27],"503000530000000000000",[31,31,32,32],1794036658007]