[{"data":1,"prerenderedAt":403},["ShallowReactive",2],{"project-14096982":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":34,"aspects":35,"strip":42,"post_text":44,"what":45,"said":80,"maker_in_thread":215,"shape":222,"grid":226,"attempts":366,"same_week":367,"maker_shows":40,"maker_hidden":395,"reader_version":396,"card":397},"14096982","Kim","A Python serialization and marshaling framework","title","Show HN: Kim – A Python serialization and marshaling framework","http:\u002F\u002Fkim.readthedocs.io\u002Fen\u002Flatest\u002F","kim.readthedocs.io","mikeywaites",1492000452,2017,136,57,292,2242,14,"notable","dev_tools","serialization library","library",[24,25,26,27],"python serialization","json marshaling","marshmallow alternative","python data mapping","debated",0.019,{"positive":31,"neutral":32,"mixed":33,"critical":31},7,10,0,24,{"idea":36,"performance":39,"docs":41},{"positive":37,"critical":38},4,5,{"positive":40,"critical":40},1,{"positive":40,"critical":40},[33,33,33,33,33,33,33,33,33,33,33,33,33,33,33,38,40,33,38,43,43,38,40,38,43,43,33,33,38,33,43,33,43,40,40,40,40,38,33,40],3,null,{"problem":46,"audience":57,"pricing":44,"motivation":65,"time_to_build":44,"stack":70,"openness":77,"stage":78,"team_size":79},{"value":47,"source":48,"evidence":49},"Serializing and marshaling Python objects to and from JSON with flexible mapping","maker",[50,53],{"id":51,"by":11,"text":52},"14097727","Thanks for expanding on that mhneu. So our primary focus with Kim has certainly been around serializing\u002Fmarshaling JSON though we've used it for plenty of other uses cases. It's…",{"id":54,"by":55,"text":56},"14097239","jackqu7","(I'm Jack, another developer at OSL.) We started writing Kim around the same time as the Marshmallow project began as we found it wasn't suitable for our needs at that time,…",{"value":58,"source":59,"evidence":60},"Python developers needing serialization beyond simple cases","inferred",[61,62],{"id":54,"by":55,"text":56},{"id":63,"by":11,"text":64},"14097592","One of the things we felt very strongly about when developing Kim was that Simple things should be simple. Complex things should be possible. To that end the Pipeline system…",{"value":66,"source":48,"evidence":67},"Needed serialization that handles unusual or advanced cases simply; started when Marshmallow did not suit their needs",[68,69],{"id":54,"by":55,"text":56},{"id":63,"by":11,"text":64},[71],{"value":72,"source":59,"evidence":73},"Python",[74,76],{"id":75},"post",{"id":54,"by":55,"text":56},"unknown","launched","team",{"praise":81,"objections":94,"requests":132,"questions":164,"alternatives":186},[82,90],{"value":83,"source":84,"evidence":85},"Benchmarks show it faster than some alternatives","commenter",[86],{"id":87,"by":88,"text":89},"14101649","voidfiles","I added Kim to my ongoing set of python serialization framework benchmarks here is how it ranks. ``` Library Many Objects One Object --------------------- --------------…",{"value":91,"source":48,"evidence":92},"Simple things simple, complex things possible design",[93],{"id":63,"by":11,"text":64},[95,103,111,125],{"value":96,"source":84,"evidence":97},"Not suited to pandas or scikit serialization needs",[98,102],{"id":99,"by":100,"text":101},"14097349","sandGorgon","We are really looking for serialization libraries that will work with pandas and scikit. This stuff is really all over the place - PMML, Arrow, Dill, pickle. Some stuff won't…",{"id":51,"by":11,"text":52},{"value":104,"source":84,"evidence":105},"Slower than lighter alternatives; schema validation adds overhead",[106,110],{"id":107,"by":108,"text":109},"14097937","Dowwie","Cool project! In the case of serialization libraries,unless you are validating as part of your (de)serialization, I'd recommend avoiding schema-driven serialization libraries.…",{"id":87,"by":88,"text":89},{"value":112,"source":84,"evidence":113},"Cycles in object graphs are not handled automatically",[114,117,121],{"id":115,"by":55,"text":116},"14097386","Yes, this is possible as long as the second level nested object has a role to stop infinite recursion from occurring. Cycles are not automatically detected. ``` class…",{"id":118,"by":119,"text":120},"14098612","BuuQu9hu","Sorry, I must be harsh. No. This fundamentally doesn't offer much advantage over a .toJSON() instance method and a .fromJSON() class method. Don't say \"security-focused\" if you…",{"id":122,"by":123,"text":124},"14100372","arnarbi","That's not a solution as it will not restore the same object graph, it will just repeat values. One way is to to store a table of objects (as identified by id()) encountered…",{"value":126,"source":84,"evidence":127},"Documentation lacks comparison with related projects",[128],{"id":129,"by":130,"text":131},"14097112","tinnet","Obviously no OS developer owes anybody an explanation, but man would I appreciate if more projects had a \"why you should use this over related projects\" (like e.g. pendulum does…",[133,141,149,157],{"value":134,"source":84,"evidence":135},"Support for pandas and scikit-learn model serialization",[136,137],{"id":99,"by":100,"text":101},{"id":138,"by":139,"text":140},"14097654","mhneu","Python's data infrastructure has a huge problem: serialization and thus saving data results. A good serialization library should serialize: ``` - classes\u002Fobjects (best practice:…",{"value":142,"source":84,"evidence":143},"Better performance",[144,148],{"id":145,"by":146,"text":147},"14097115","siddhant","Cool! Are there any speed comparisons available between this and marshmallow (or other alternatives)?",{"id":107,"by":108,"text":109},{"value":150,"source":84,"evidence":151},"Comparison with alternatives in docs",[152,153],{"id":129,"by":130,"text":131},{"id":154,"by":155,"text":156},"14097091","sakawa","It does look like marshmalllow[1]. How does relate Kim with it? [1]: https:\u002F\u002Fgithub.com\u002Fmarshmallow-code\u002Fmarshmallow\u002F",{"value":158,"source":84,"evidence":159},"Close resolved GitHub issues",[160],{"id":161,"by":162,"text":163},"14098073","yeukhon","Nice, but I recommend closing issues https:\u002F\u002Fgithub.com\u002Fmikeywaites\u002Fkim\u002Fissues which have fixes (some of them show 'merge'). It's one thing I as a user look at choosing whether…",[165,169,179],{"value":166,"source":84,"evidence":167},"How does it relate to marshmallow",[168],{"id":154,"by":155,"text":156},{"value":170,"source":84,"evidence":171},"Can it serialize cyclic structures",[172,176],{"id":173,"by":174,"text":175},"14097098","amelius","Can it serialize cycles?",{"id":177,"by":174,"text":178},"14097283","Roughly speaking, by cycles I mean a structure that refers to itself somehow. For example: ``` A = {} B = {} A[\"ref\"] = B B[\"ref\"] = A ``` So would it be possible to serialize A…",{"value":180,"source":84,"evidence":181},"How does it handle Unicode",[182],{"id":183,"by":184,"text":185},"14100957","ziikutv","So this takes JSON and maps it to namedtuples? Silly question, what happens with Unicode?",[187,195,206,210],{"value":188,"source":84,"evidence":189},"Marshmallow",[190,191],{"id":154,"by":155,"text":156},{"id":192,"by":193,"text":194},"14098881","makmanalp","So stuff like this or marshmallow is more for cases when you have some database \u002F ORM objects and you want to serialize them out to a json object, or you want to process…",{"value":196,"source":84,"evidence":197},"Pickle and dill",[198,202],{"id":199,"by":200,"text":201},"14098662","joshuamorton","To expand on fnord, to my knowledge, pickle handles all of these things. Its still a bad solution, but it does everything you want. ``` pickle.dump(f, anyobject) anyobject =…",{"id":203,"by":204,"text":205},"14102033","ff7c11","why not pickle :) :)",{"value":207,"source":84,"evidence":208},"Asphalt-serialization",[209],{"id":107,"by":108,"text":109},{"value":211,"source":84,"evidence":212},"Arrow, Parquet, HDF5 via PyTables",[213,214],{"id":192,"by":193,"text":194},{"id":99,"by":100,"text":101},{"comments":32,"answered":37,"key":216},[217,218,219],{"id":63,"text":64},{"id":51,"text":52},{"id":220,"text":221},"14098010","Hey Dowwie! That's a great point and an important distinction to make. As I mentioned in some of the other comments, we have certainly been focussed on features over performance…",{"commenters":223,"max_depth":38,"maker_replies":32,"first_comment_mins":224,"lifespan_hours":225},34,15,16,[227,229,230,232,234,238,241,244,248,249,253,256,260,262,264,267,269,271,273,274,278,280,284,288,291,293,296,299,301,305,308,310,312,315,318,319,320,322,326,328,330,333,335,338,340,342,343,345,349,350,351,353,354,356,360,361,363],[99,33,33,228,100,33],"commenter\u002Fneutral\u002Fidea",[63,33,33,228,11,40],[138,33,33,231,139,33],"other\u002Fneutral\u002Fidea",[51,33,33,233,11,40],"maker\u002Fneutral\u002Fidea",[235,33,33,236,237,33],"14097859","commenter\u002Fneutral\u002Fnone","natch",[239,33,33,236,240,33],"14097923","js2",[242,33,33,231,243,33],"14098271","fnord123",[245,33,33,246,247,33],"14103045","other\u002Fneutral\u002Fnone","auxym",[199,33,33,231,200,33],[250,33,33,251,252,33],"14100585","other\u002Fcritical\u002Fperformance","sidlls",[254,33,33,255,200,33],"14100717","commenter\u002Fneutral\u002Fperformance",[257,33,33,258,259,33],"14102833","other\u002Fneutral\u002Fperformance","vosper",[261,33,33,231,100,33],"14099804",[263,33,33,231,243,33],"14098234",[265,33,33,246,266,33],"14099269","dtjon",[268,33,33,231,100,33],"14099378",[270,33,33,231,266,33],"14099410",[272,33,33,231,100,33],"14101504",[192,33,33,231,193,33],[275,33,33,276,277,33],"14103657","other\u002Fneutral\u002Fdocs","digisth",[279,33,33,231,193,33],"14103872",[281,40,38,282,283,33],"14099280","project\u002Fpositive\u002Fidea","limdauto",[285,40,43,286,287,33],"14099543","project\u002Fneutral\u002Fidea","_e",[107,289,40,290,108,33],2,"project\u002Fcritical\u002Fperformance",[220,289,33,292,11,40],"maker\u002Fneutral\u002Fperformance",[294,289,38,295,108,33],"14098025","maker\u002Fpositive\u002Fnone",[297,289,33,298,11,40],"14098049","maker\u002Fneutral\u002Fnone",[87,43,43,300,88,33],"project\u002Fneutral\u002Fperformance",[302,43,43,303,304,33],"14102943","project\u002Fneutral\u002Fdocs","RussianCow",[306,43,38,307,193,33],"14103943","project\u002Fpositive\u002Fperformance",[309,43,33,292,11,40],"14104271",[161,37,40,311,162,33],"project\u002Fcritical\u002Fdocs",[313,37,33,314,11,40],"14098093","maker\u002Fneutral\u002Fdocs",[316,37,38,317,162,33],"14098120","maker\u002Fpositive\u002Fdocs",[154,38,43,286,155,33],[129,38,43,303,130,33],[321,38,33,314,11,40],"14097197",[323,38,33,324,325,33],"14097754","commenter\u002Fneutral\u002Fdocs","pekk",[327,38,33,324,130,33],"14101917",[54,38,38,329,55,33],"maker\u002Fpositive\u002Fidea",[331,38,38,282,332,33],"14098219","bpicolo",[334,38,33,317,11,40],"14100450",[336,38,43,286,337,33],"14097319","sametmax",[145,339,43,300,146,33],6,[341,339,33,292,11,40],"14097154",[173,31,43,286,174,33],[344,31,33,298,11,40],"14097221",[346,31,40,347,348,33],"14097252","project\u002Fcritical\u002Fidea","emidln",[177,31,40,347,174,33],[115,31,43,286,55,33],[352,31,40,347,174,33],"14098311",[122,31,40,347,123,33],[183,355,43,286,184,33],8,[357,358,38,282,359,33],"14097130",9,"rat87",[203,32,33,246,204,33],[118,362,40,347,119,33],11,[364,362,33,236,365,33],"14099137","mafro",[],[368,373,378,382,386,391],{"id":369,"name":370,"points":371,"category":372},"14058815","Resumegen",227,"productivity",{"id":374,"name":375,"points":376,"category":377},"14068525","therecipe\u002Fqt",222,"languages",{"id":379,"name":380,"points":381,"category":372},"14097011","Remote Stash",214,{"id":383,"name":384,"points":14,"category":385},"14076198","LuaGB","games",{"id":387,"name":388,"points":389,"category":390},"14091952","strategic-options.com",96,"finance_crypto",{"id":392,"name":393,"points":394,"category":377},"14084774","Glua",70,false,"v2",{"id":398,"name":5,"line":6,"ai":395,"cat":20,"year":13,"points":14,"comments":15,"rec":28,"s":399,"top":18,"kw":400,"maker":11,"domain":10,"tier":19,"attempts":33,"strip":401,"url":9,"mix":402,"praise":83,"objection":96,"sub":21},14096982,0.02,[24,25,26,27],"0000000000000005105335153300503031111501",[31,32,33,31],1794036554300]