[{"data":1,"prerenderedAt":460},["ShallowReactive",2],{"project-37114141":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":35,"aspects":36,"strip":45,"post_text":48,"what":49,"said":92,"maker_in_thread":221,"shape":230,"grid":235,"attempts":422,"same_week":423,"maker_shows":43,"maker_hidden":452,"reader_version":453,"card":454},"37114141","learndb-py","A RDBMS (SQLite clone) from scratch in pure Python","title","Show HN: I wrote a RDBMS (SQLite clone) from scratch in pure Python","https:\u002F\u002Fgithub.com\u002Fspandanb\u002Flearndb-py","github.com","spanspan",1691958731,2023,371,77,97,3109,4,"legend","learning_science","database education","library",[24,25,26,27],"sqlite clone python","python database","b-tree","learning database internals","debated",0.159,{"positive":31,"neutral":32,"mixed":33,"critical":34},8,11,0,9,28,{"idea":37,"performance":39,"execution":40,"design":42,"docs":44},{"positive":18,"critical":38},6,{"positive":33,"critical":33},{"positive":41,"critical":33},1,{"positive":41,"critical":43},2,{"positive":43,"critical":41},[46,33,33,33,33,41,33,47,41,33,47,33,46,41,33,41,33,33,33,33,46,46,47,33,46,46,47,47,47,47,41,46,47,41,33,41,41,47,33,33],5,3,"I wrote a relational database management system (RDBMS) (sqlite clone) from scratch in pure Python.",{"problem":50,"audience":60,"pricing":72,"motivation":73,"time_to_build":72,"stack":78,"openness":89,"stage":90,"team_size":91},{"value":51,"source":52,"evidence":53},"Learning how relational databases work internally","maker",[54,57],{"id":55,"by":11,"text":56},"37114755","Most definitely. The b-tree implementation was the first motivation for starting the project. Especially, all the details around node rebalancing and splitting. And the fact that…",{"id":58,"by":11,"text":59},"37116619","I began using python as a way to \"mock\" out the overall design; intending to re-implement it in rust. The main reasoning for using python: was the ability to focus on…",{"value":61,"source":62,"evidence":63},"Programmers learning database internals","inferred",[64,68],{"id":65,"by":66,"text":67},"37114607","sgarland","This is fantastic, and seems like a great way for someone (me) to learn DS&A better. I can explain how a B+tree works, but if you asked me to code it, I’d freeze up. I love…",{"id":69,"by":70,"text":71},"37120146","FrustratedMonky","Guess to get back to the post. He did it for learning, not to be an exact duplicate or competitor to replace other RDBMS. Often when learning, you do not implement every…",null,{"value":74,"source":52,"evidence":75},"Learning how databases work, starting with B-tree and parser",[76,77],{"id":55,"by":11,"text":56},{"id":65,"by":66,"text":67},[79,83],{"value":80,"source":52,"evidence":81},"Python",[82],{"id":58,"by":11,"text":59},{"value":84,"source":52,"evidence":85},"Lark parser library",[86],{"id":87,"by":11,"text":88},"37116006","This string is the grammar that lark uses to parse the user submitted sql into an AST. This parsing is far more complete and robust than what doing this in pure python would…","open_source","prototype","unknown",{"praise":93,"objections":122,"requests":150,"questions":166,"alternatives":194},[94,110],{"value":95,"source":96,"evidence":97},"Readable code and good documentation","commenter",[98,102,106],{"id":99,"by":100,"text":101},"37116127","indymike","First, great project. Second, the code is very readable. Finally thank you for excellent comments.",{"id":103,"by":104,"text":105},"37115002","xjsjxjsjsh","The documentation is excellent.",{"id":107,"by":108,"text":109},"37120443","flakiness","This is very nice! SQLite is very hard to read, but this one is actually quite comprehensible. Especially the VM part:…",{"value":111,"source":96,"evidence":112},"Good learning project for databases and Python",[113,117,118],{"id":114,"by":115,"text":116},"37114709","samsquire","Thank you for sharing My perspective is that writing this kind of system in a language such as Python is actually a great thing because for myself Python is more widely readable…",{"id":65,"by":66,"text":67},{"id":119,"by":120,"text":121},"37115652","quickthrower2","Nice! it is good to see past blind spots e.g. \"you must write this kind of app in this kind of language, pure Python is high level it is only for glue\". I wouldn't have thought…",[123,134,144],{"value":124,"source":96,"evidence":125},"Python unsuitable for serious database work (concurrency, memory layout)",[126,130],{"id":127,"by":128,"text":129},"37119934","crabbone","Not even close. The real problems you will have to solve when working with a database anyone would want to use are, for example: * Memory layout of the buffers used to write \u002F…",{"id":131,"by":132,"text":133},"37120576","pyuser583","I know Python, but little C. I don’t recommend writing a DB in Python. So much of DB development is dealing with race conditions and consistency. Python files require an…",{"value":135,"source":96,"evidence":136},"Calling it a SQLite clone is misleading",[137,141],{"id":138,"by":139,"text":140},"37115698","keithalewis","Perhaps a more accurate claim would be \"SQLite inspired\". Calling it a clone is misleading. Mad props to the author. Many Python programmers never had proper training in computer…",{"id":142,"by":139,"text":143},"37116634","Just trying to encourage clear and accurate communication. I agree with your first sentence. The only thing unfair is the author claiming it is a SQLite clone. It isn't, as we…",{"value":145,"source":52,"evidence":146},"Lacks transactions and full ACID guarantees",[147],{"id":148,"by":11,"text":149},"37116073","Re: ACID guarantees It doesn't have a notion of atomically batching multiple statements, i.e. transaction. But beyond that, it's a single file database, which can only have a…",[151,158],{"value":152,"source":96,"evidence":153},"Benchmarks",[154],{"id":155,"by":156,"text":157},"37114585","KeplerBoy","Kudos, i bet that was a fun and rewarding experience. I know this was never meant to be fast, but could you produce some benchmarks for shits and giggles?",{"value":159,"source":96,"evidence":160},"Run SQLite test suite against it",[161,162],{"id":107,"by":108,"text":109},{"id":163,"by":164,"text":165},"37115441","_tom_","How much of the SQLite test suite will pass?",[167,180,187],{"value":168,"source":96,"evidence":169},"Does it support JOINs?",[170,174,177],{"id":171,"by":172,"text":173},"37119779","rvba","Is this really a relational database if it does not have JOINs?",{"id":175,"by":11,"text":176},"37120552","It does have joins",{"id":178,"by":172,"text":179},"37121653","The splash page says that Learndb supports the following: select, from, where, group by, having, limit, order by I dont see \"JOIN\" there. So either the DB doesnt support it or…",{"value":181,"source":96,"evidence":182},"Does it support ACID guarantees and query optimization?",[183],{"id":184,"by":185,"text":186},"37115601","schemescape","Does it support any ACID guarantees? Query planning\u002Foptimization? I’m not saying this to imply that it can\u002Fshould. I just want to know what all was attempted besides b-trees and…",{"value":188,"source":96,"evidence":189},"How it compares to Python's sqlite3 module",[190],{"id":191,"by":192,"text":193},"37118823","Uptrenda","Nice OP. How would you say this compares to the inbuilt 'sqlite3' module in Python? Is it more or less portable? Different features?",[195,206,214],{"value":196,"source":96,"evidence":197},"H2 (Java embedded RDBMS)",[198,202],{"id":199,"by":200,"text":201},"37116429","gabereiser","H2 is pretty great and fully supports JSONB and the like [1]. Full java-only RDBMS. [1]: https:\u002F\u002Fwww.h2database.com\u002Fhtml\u002Fmain.html",{"id":203,"by":204,"text":205},"37114862","devman0","This is why you see a community of pure Java RDBMSs (Hypersonic, H2, Derby, etc), if you don't need the big iron scale it's easier to ship\u002Fuse the database or even embed it in…",{"value":207,"source":96,"evidence":208},"HSQLDB and Derby",[209,210],{"id":203,"by":204,"text":205},{"id":211,"by":212,"text":213},"37115047","fuzztester","Yes. It's an interesting area, irrespective of implementation language. I had tried out one or two Java-based RDBMSes back in the day, via programs written in Java, for fun. I…",{"value":215,"source":96,"evidence":216},"Gadfly (pure Python SQL database)",[217],{"id":218,"by":219,"text":220},"37116151","kragen","i used to use gadfly, aaron watters's sql database in pure python; the first release was in 01994 i'm not sure current python is capable of running it",{"comments":34,"answered":38,"key":222},[223,226,229],{"id":224,"text":225},"37115183","Cool stuff. I had similar intuitions- Python allow me to focus on the high-level concepts. Albeit, there were times where I wished I had gone with a statically-typed + compiled…",{"id":227,"text":228},"37115216","It would be a fun exercise to implement something like TPC-C for learndb and see how this done.",{"id":87,"text":88},{"commenters":231,"max_depth":232,"maker_replies":34,"first_comment_mins":233,"lifespan_hours":234},53,7,38,154,[236,238,240,241,242,245,247,250,252,255,258,261,265,268,269,270,272,275,277,280,282,285,288,290,291,295,299,302,304,307,309,311,314,317,320,322,324,327,330,333,335,336,337,339,340,341,343,345,349,352,353,356,358,361,364,369,371,373,374,376,379,381,384,387,389,390,393,397,399,400,402,404,407,410,414,416,419],[114,33,46,237,115,33],"project\u002Fpositive\u002Fidea",[203,33,33,239,204,33],"other\u002Fneutral\u002Fnone",[211,33,33,239,212,33],[199,33,33,239,200,33],[243,33,33,239,244,33],"37118530","p2detar",[246,33,33,239,212,33],"37123824",[248,33,33,239,249,33],"37120299","lopatin",[224,33,33,251,11,41],"maker\u002Fneutral\u002Fidea",[253,33,33,239,254,33],"37115272","p4bl0",[256,33,33,239,257,33],"37115501","tmh88j",[259,33,41,260,128,33],"37118839","project\u002Fcritical\u002Fidea",[262,33,33,263,264,33],"37118957","commenter\u002Fneutral\u002Fnone","Eddygandr",[266,33,33,263,267,33],"37118990","dumpsterdiver",[127,33,41,260,128,33],[69,33,46,237,70,33],[271,33,41,260,128,33],"37123271",[273,33,33,263,274,33],"37132705","dkfellows",[276,33,33,263,115,33],"37120804",[278,41,33,239,279,33],"37116134","malkia",[155,43,47,281,156,33],"project\u002Fneutral\u002Fperformance",[283,43,33,239,284,33],"37114850","bryancoxwell",[286,43,33,239,287,33],"37114882","mburns",[289,43,33,239,284,33],"37115450",[227,43,33,281,11,41],[292,47,46,293,294,33],"37115052","project\u002Fpositive\u002Fexecution","simonw",[296,47,41,297,298,33],"37115386","project\u002Fcritical\u002Fdesign","OJFord",[300,47,33,263,301,33],"37115625","tmpX7dMeXU",[303,47,33,263,298,33],"37116095",[305,47,33,239,306,33],"37119851","mejutoco",[308,47,41,297,298,33],"37120461",[310,47,33,263,294,33],"37115743",[312,47,33,239,313,33],"37117536","dleslie",[315,47,33,239,316,33],"37118687","zelphirkalt",[318,47,33,263,319,33],"37115778","semiquaver",[321,47,33,263,298,33],"37116089",[87,47,33,323,11,41],"maker\u002Fneutral\u002Fexecution",[325,47,33,239,326,33],"37196818","erezsh",[328,47,33,239,329,33],"37116328","jarulraj",[331,47,47,251,332,33],"37118938","itissid",[107,18,46,334,108,33],"project\u002Fpositive\u002Fdesign",[65,46,46,237,66,33],[55,46,33,323,11,41],[163,38,47,338,164,33],"project\u002Fneutral\u002Fexecution",[184,232,47,338,185,33],[148,232,33,323,11,41],[342,31,33,239,244,33],"37118543",[99,34,46,344,100,33],"project\u002Fpositive\u002Fdocs",[346,347,33,239,348,33],"37117722",10,"est",[350,347,33,239,351,33],"37118636","networked",[103,32,46,344,104,33],[354,355,47,281,329,33],"37116361",12,[58,355,33,357,11,41],"maker\u002Fneutral\u002Fperformance",[191,359,47,360,192,33],13,"project\u002Fneutral\u002Fidea",[362,359,33,263,363,33],"37118891","ritzaco",[365,366,47,367,368,33],"37188138",14,"project\u002Fneutral\u002Fnone","prmobiledev",[218,370,33,239,219,33],15,[171,372,47,338,172,33],16,[175,372,33,338,11,41],[178,372,41,375,172,33],"project\u002Fcritical\u002Fdocs",[377,372,33,263,378,33],"37127302","DANmode",[119,380,46,237,120,33],17,[382,383,47,360,139,33],"37114941",18,[385,383,47,360,386,33],"37115231","tredre3",[388,383,33,360,11,41],"37115320",[138,383,41,260,139,33],[391,383,33,263,392,33],"37116157","hluska",[394,383,33,395,396,33],"37116566","commenter\u002Fneutral\u002Fidea","esjeon",[398,383,33,263,392,33],"37135092",[142,383,41,260,139,33],[401,383,33,360,11,41],"37117360",[131,403,41,260,132,33],19,[405,403,47,360,406,33],"37120932","lolinder",[408,403,33,263,409,33],"37122885","janc_",[411,412,33,239,413,33],"37114532",20,"kindawinda",[415,412,33,239,413,33],"37115345",[417,412,33,263,418,33],"37115521","YZF",[420,412,33,239,421,33],"37115761","memefrog",[],[424,429,434,439,443,448],{"id":425,"name":426,"points":427,"category":428},"37125118","Outlines",854,"ai",{"id":430,"name":431,"points":432,"category":433},"37119942","Little Rat",555,"security_privacy",{"id":435,"name":436,"points":437,"category":438},"37083309","OFRAK Tetris",472,"games",{"id":440,"name":441,"points":442,"category":428},"37128293","AI-town",429,{"id":444,"name":445,"points":446,"category":447},"37075730","Applite",280,"dev_tools",{"id":449,"name":450,"points":451,"category":428},"37148210","LlamaGPT",225,false,"v2",{"id":455,"name":5,"line":6,"ai":452,"cat":20,"year":13,"points":14,"comments":15,"rec":28,"s":456,"top":18,"kw":457,"maker":11,"domain":10,"tier":19,"attempts":33,"strip":458,"url":9,"mix":459,"praise":95,"objection":124,"sub":21},37114141,0.16,[24,25,26,27],"5000010310305101000055305533331531011300",[31,32,33,34],1794036639994]