[{"data":1,"prerenderedAt":360},["ShallowReactive",2],{"project-6012556":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":35,"aspects":36,"strip":42,"post_text":43,"what":44,"said":77,"maker_in_thread":203,"shape":211,"grid":214,"attempts":322,"same_week":323,"maker_shows":299,"maker_hidden":352,"reader_version":353,"card":354},"6012556","groupdate.sql","Better SQL group by date","title","Show HN: Better SQL group by date","http:\u002F\u002Fankane.github.io\u002Fgroupdate.sql\u002F","ankane.github.io","akane",1373377023,2013,60,44,424,1739,25,"notable","languages","sql function library","library",[24,25,26,27],"sql group by date","mysql date grouping","postgresql time zone","sql date functions","mixed",-0.185,{"positive":31,"neutral":32,"mixed":33,"critical":34},2,3,1,5,11,{"design":37,"performance":39,"idea":40,"privacy":41},{"positive":38,"critical":31},0,{"positive":38,"critical":33},{"positive":33,"critical":38},{"positive":38,"critical":31},[32,38,38,38,38,38,38,38,38,38,33,38,32,38,38,38,38,32,38,38,33,38,33,38,33,38,38,33,38,38,38,38,38,38,38,34,38,38,32,34],null,{"problem":45,"audience":43,"pricing":43,"motivation":55,"time_to_build":43,"stack":59,"openness":74,"stage":75,"team_size":76},{"value":46,"source":47,"evidence":48},"Grouping SQL results by date, week or time intervals with correct time zone handling is awkward to write by hand","maker",[49,52],{"id":50,"by":11,"text":51},"6012876","Thanks for the feedback. Working correctly with time zones in the database can be tricky. DATE_TRUNC('day', $1 AT TIME ZONE $2) AT TIME ZONE $2; or…",{"id":53,"by":11,"text":54},"6013031","For those concerned about a SQL dependency, it might be useful to create projects in different languages that package these functions. These methods were extracted from…",{"value":56,"source":47,"evidence":57},"Extracted from the groupdate Ruby gem so other databases and languages can use the functions",[58],{"id":53,"by":11,"text":54},[60,66,70],{"value":61,"source":47,"evidence":62},"SQL (MySQL)",[63],{"id":64,"by":11,"text":65},"6012635","curl | sh is a very transparent installer. Here's the source. https:\u002F\u002Fraw.github.com\u002Fankane\u002Fgroupdate.sql\u002Fmaster\u002Fmysql.sql Also, you don't have to use the root user - I'll make…",{"value":67,"source":47,"evidence":68},"SQL (PostgreSQL)",[69],{"id":53,"by":11,"text":54},{"value":71,"source":47,"evidence":72},"Ruby \u002F ActiveRecord (original gem)",[73],{"id":53,"by":11,"text":54},"open_source","launched","unknown",{"praise":78,"objections":102,"requests":138,"questions":166,"alternatives":179},[79,91],{"value":80,"source":81,"evidence":82},"Useful and simple wrapper","commenter",[83,87],{"id":84,"by":85,"text":86},"6012732","pvnick","Pretty cool! I can see this being really useful. Beware, however, that, while there are certain cases that mysql can utilize indexes in a group by clause if values are constant…",{"id":88,"by":89,"text":90},"6012948","richardv","If you know that you need to be grouping by dates... then it's usually best to store the date fragments in a separate column so you can take advantage of indexes. The project is…",{"value":92,"source":81,"evidence":93},"Support for more database engines wanted",[94,98],{"id":95,"by":96,"text":97},"6012578","memset","Very cool. Thank you for sharing! (Now we just need one for Oracle, DB2, and Sybase...)",{"id":99,"by":100,"text":101},"6013127","gtrubetskoy","PostgreSQL and MySQL? Where is the Hive version? :)",[103,115,123],{"value":104,"source":81,"evidence":105},"Storing logic in the database adds dependency and complexity",[106,110,114],{"id":107,"by":108,"text":109},"6012835","pbnjay","I'm not sure why this is necessary. It's a very light wrapper around existing SQL functions that adds additional dependencies and makes it more difficult for developers who…",{"id":111,"by":112,"text":113},"6013149","lowglow","I'm sure I'm a neophyte at this, but can someone explain to me why we would want to store functions in a database? It fragments the logic, adds another layer of complexity when…",{"id":88,"by":89,"text":90},{"value":116,"source":81,"evidence":117},"Functions prevent index use, so the approach is slow on large tables",[118,119],{"id":84,"by":85,"text":86},{"id":120,"by":121,"text":122},"6013741","joevandyk","Postgres can use indexes with these functions. If the functions are marked stable or immutable, then the function bodies will be inlined directly into the query.",{"value":124,"source":81,"evidence":125},"Piping curl to a root shell is a security concern",[126,130,134],{"id":127,"by":128,"text":129},"6012624","Rygu","Looks simple and useful. Only worry is that the installation instruction is a curl request piped to mysql with root user... Thanks for sharing!",{"id":131,"by":132,"text":133},"6013081","hnha","did you access that url with curl to make sure it does no user agent sniffing? how do you know the script is the same on subsequent requests?",{"id":135,"by":136,"text":137},"6013164","bcoates","It's an https request to GitHub, which has about a million more subtle ways to own your machine if they really wanted to. Figuring out how to solve the TOCTOU problem for a small…",[139,143,147,157],{"value":140,"source":81,"evidence":141},"Support for Oracle, DB2, Sybase and other databases",[142],{"id":95,"by":96,"text":97},{"value":144,"source":81,"evidence":145},"Support for Hive",[146],{"id":99,"by":100,"text":101},{"value":148,"source":81,"evidence":149},"Five-minute interval grouping",[150,154],{"id":151,"by":152,"text":153},"6013004","Terretta","A common grouping needed by sysadmins is to group by 5 minute intervals. That query may be more likely to trip up a junior DBA than most of these. Could be useful to add. Example…",{"id":155,"by":11,"text":156},"6013175","Good suggestion!",{"value":158,"source":81,"evidence":159},"Mark PostgreSQL functions IMMUTABLE or STABLE",[160,163],{"id":161,"by":121,"text":162},"6013717","The postgres functions should be marked IMMUTABLE. http:\u002F\u002Fwww.postgresql.org\u002Fdocs\u002F9.2\u002Fstatic\u002Fxfunc-volatility.html This allows the planner to optimize queries that use the…",{"id":164,"by":11,"text":165},"6014312","Awesome, added this to the todo list.",[167,171],{"value":168,"source":81,"evidence":169},"Why store functions in the database at all?",[170],{"id":111,"by":112,"text":113},{"value":172,"source":81,"evidence":173},"Can PostgreSQL use indexes with these functions?",[174,178],{"id":175,"by":176,"text":177},"6012980","bjourne","You don't need that if you use postgresql: ``` create index foo1 on t1 (date(timestamp)) create index foo2 on t2 (extract(week from timestamp)) ``` Postgres can use almost any…",{"id":120,"by":121,"text":122},[180,185,196],{"value":181,"source":81,"evidence":182},"Precomputed date columns or indexes on expressions",[183,184],{"id":88,"by":89,"text":90},{"id":175,"by":176,"text":177},{"value":186,"source":81,"evidence":187},"Calendar\u002Fdate dimension table as used in data warehouses",[188,192],{"id":189,"by":190,"text":191},"6013571","chaz","A common alternative to this for data warehouses and reporting platforms is to construct a date table in advance, with every date enumerated from, say, 1990 through 2050. Every…",{"id":193,"by":194,"text":195},"6013896","pjungwir","This is the approach recommended by Ralph Kimball for data warehouses. I haven't tried it for OLTP systems. I'd be more inclined to go with ad hoc functional indexes (as…",{"value":197,"source":81,"evidence":198},"Store everything in UTC and convert in queries",[199],{"id":200,"by":201,"text":202},"6013384","tudborg","Simply use and store everything in UTC. If you ever need to do grouping by time in any other timezone, it would be trivial to do the conversion. DAYOFYEAR(…",{"comments":204,"answered":31,"key":205},8,[206,209,210],{"id":207,"text":208},"6013320","I would suggest denormalizing only after you find a performance bottleneck. As they say, \"premature optimization is the root of all evil\".",{"id":155,"text":156},{"id":164,"text":165},{"commenters":212,"max_depth":34,"maker_replies":204,"first_comment_mins":34,"lifespan_hours":213},30,19,[215,217,219,223,226,228,230,232,233,235,238,239,241,242,244,247,249,251,253,255,258,261,263,267,270,273,275,277,280,281,283,285,287,288,291,294,297,301,304,305,309,312,314,317],[88,38,32,216,89,38],"project\u002Fmixed\u002Fdesign",[175,38,38,218,176,38],"commenter\u002Fneutral\u002Fperformance",[220,38,38,221,222,38],"6013134","other\u002Fneutral\u002Fnone","kalleboo",[224,38,38,221,225,38],"6014163","orf",[227,38,38,221,222,38],"6018255",[229,38,38,218,136,38],"6013125",[231,38,38,218,136,38],"6014920",[207,38,38,218,11,33],[189,38,38,234,190,38],"other\u002Fneutral\u002Fdesign",[236,38,38,221,237,38],"6017598","jacques_chester",[193,38,38,234,194,38],[84,33,33,240,85,38],"project\u002Fcritical\u002Fperformance",[120,33,38,218,121,38],[151,31,32,243,152,38],"project\u002Fneutral\u002Fidea",[245,31,38,221,246,38],"6013040","umsm",[248,31,38,221,194,38],"6013930",[250,31,38,221,237,38],"6017610",[155,31,38,252,11,33],"commenter\u002Fpositive\u002Fidea",[161,32,32,254,121,38],"project\u002Fneutral\u002Fperformance",[256,32,38,218,257,38],"6013747","wulczer",[259,32,38,218,260,38],"6014138","jeltz",[164,32,38,262,11,33],"maker\u002Fpositive\u002Fperformance",[264,265,33,266,201,38],"6013344",4,"project\u002Fcritical\u002Fdesign",[268,34,38,221,269,38],"6014686","kogir",[127,271,33,272,128,38],6,"project\u002Fcritical\u002Fprivacy",[64,271,38,274,11,33],"maker\u002Fneutral\u002Fprivacy",[131,271,33,276,132,38],"maker\u002Fcritical\u002Fprivacy",[278,271,38,279,11,33],"6013126","commenter\u002Fneutral\u002Fprivacy",[135,271,38,279,136,38],[107,282,33,266,108,38],7,[50,282,38,284,11,33],"maker\u002Fneutral\u002Fdesign",[200,282,38,286,201,38],"commenter\u002Fneutral\u002Fdesign",[111,204,38,221,112,38],[289,204,38,221,290,38],"6013251","Retric",[292,204,38,221,293,38],"6013261","bobwaycott",[295,204,38,221,296,38],"6013813","papsosouid",[298,299,38,221,300,38],"6013621",9,"jotto",[53,302,38,303,11,33],10,"project\u002Fpositive\u002Fidea",[95,35,34,303,96,38],[306,35,38,307,308,38],"6012945","commenter\u002Fneutral\u002Fnone","bratsche",[310,35,38,307,311,38],"6013512","z92",[99,313,32,243,100,38],12,[315,313,38,316,11,33],"6013167","maker\u002Fneutral\u002Fidea",[318,319,34,320,321,38],"6013089",13,"project\u002Fpositive\u002Fnone","jerogarcia",[],[324,329,334,338,343,348],{"id":325,"name":326,"points":327,"category":328},"6002518","vim-airline",95,"dev_tools",{"id":330,"name":331,"points":332,"category":333},"6009428","Lob",80,"web_saas",{"id":335,"name":336,"points":337,"category":328},"6022522","waffle.io",78,{"id":339,"name":340,"points":341,"category":342},"6002009","7min",76,"consumer_social",{"id":344,"name":345,"points":346,"category":347},"5997007","countryaday",73,"learning_science",{"id":349,"name":350,"points":14,"category":351},"5992341","deadchat","security_privacy",false,"v2",{"id":355,"name":5,"line":6,"ai":352,"cat":20,"year":13,"points":14,"comments":15,"rec":28,"s":356,"top":18,"kw":357,"maker":11,"domain":10,"tier":19,"attempts":38,"strip":358,"url":9,"mix":359,"praise":80,"objection":104,"sub":21},6012556,-0.18,[24,25,26,27],"3000000000103000030010101001000000050035",[31,32,33,34],1793990895988]