[{"data":1,"prerenderedAt":236},["ShallowReactive",2],{"project-15945808":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":28,"sentiment":29,"stance_mix":30,"scored_comments":33,"aspects":34,"strip":37,"post_text":41,"what":42,"said":66,"maker_in_thread":138,"shape":145,"grid":150,"attempts":197,"same_week":198,"maker_shows":38,"maker_hidden":228,"reader_version":229,"card":230},"15945808","word2vec-spam-filter","Using word vectors to classify spam messages","title","Show HN: Using word vectors to classify spam messages","https:\u002F\u002Fgithub.com\u002Fdoodyparizada\u002Fword2vec-spam-filter","github.com","doody_parizada",1513520064,2017,104,21,439,2242,20,"notable","ai","spam classification","library",[24,25,26,27],"spam filter","word vectors","glove","text classification","debated",-0.071,{"positive":31,"neutral":31,"mixed":32,"critical":31},2,0,6,{"idea":35,"docs":36},{"positive":31,"critical":31},{"positive":32,"critical":32},[38,39,32,32,40,40,32,32,32,32,32,32,32,32,32,39,38,32,32,32,32],1,5,3,null,{"problem":43,"audience":41,"pricing":41,"motivation":41,"time_to_build":41,"stack":52,"openness":65,"stage":65,"team_size":65},{"value":44,"source":45,"evidence":46},"Classifying spam messages using word vectors","inferred",[47,49],{"id":48},"post",{"id":50,"by":11,"text":51},"15950359","We don't have words like v1agra. As I mentioned in the README, we took vectors pretrained on wikipedia. One of the possible improvements can be to train the vectors on our own…",[53,61],{"value":54,"source":55,"evidence":56},"GloVe word vectors","maker",[57,58],{"id":50,"by":11,"text":51},{"id":59,"by":11,"text":60},"15950375","We started out with word2vec, but discovered it was easier to work with GloVe for our purpose. There are ways to convert glove vectors to word2vec format such as:…",{"value":62,"source":55,"evidence":63},"Wikipedia-pretrained vectors",[64],{"id":50,"by":11,"text":51},"unknown",{"praise":67,"objections":87,"requests":102,"questions":107,"alternatives":119},[68,76],{"value":69,"source":70,"evidence":71},"Interesting, fun project idea","commenter",[72],{"id":73,"by":74,"text":75},"15947635","mci","Sounds like a fun project. However, I doubt if word vectors buy you anything more than, say, old good Nilsimsa from 2001 (https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FNilsimsa_Hash). Side note:…",{"value":77,"source":70,"evidence":78},"Word vectors can catch paraphrased spam",[79,83],{"id":80,"by":81,"text":82},"15948359","laretluval","word2vec has the advantage that you could potentially identify spam messages that are paraphrases rather than exact copies of the ones in the training set.",{"id":84,"by":85,"text":86},"15947275","EmilStenstrom","It seems DCC isn't using word vectors at all? Using word vectors you can know that viagra and v14gr4 is the same word, because it is used in the same way in messages. That in…",[88,95],{"value":89,"source":70,"evidence":90},"Nilsimsa hashing may do as well without word vectors",[91,92],{"id":73,"by":74,"text":75},{"id":93,"by":74,"text":94},"15949831","1. Pedantically: it's GloVe, not word2vec. 2. Nilsimsa or any locality-sensitive hash detect changed messages, too, be the changes synonyms or not. 3. I don't think OP's GloVe…",{"value":96,"source":70,"evidence":97},"Bulk sending is a weak spam signal on its own",[98],{"id":99,"by":100,"text":101},"15947818","massaman_yams","That, and the fact that a message is sent in bulk isn't actually a very strong indicator that the message is spam, at least in the email world. As one input to a filtering…",[103],{"value":104,"source":55,"evidence":105},"Train vectors on own spam dataset",[106],{"id":50,"by":11,"text":51},[108],{"value":109,"source":70,"evidence":110},"Does it use word2vec or GloVe?",[111,115],{"id":112,"by":113,"text":114},"15946746","chrbarrol","I was hoping to learn about word2vec by reading the sourcecode but am I right when I say this has nothing to do with word2vec?",{"id":116,"by":117,"text":118},"15946894","drwl","Looks like it uses GloVe and not word2vec. They're both algorithms for generating word vectors but they are different.",[120,124,131],{"value":121,"source":70,"evidence":122},"Nilsimsa hash",[123],{"id":73,"by":74,"text":75},{"value":125,"source":70,"evidence":126},"DCC",[127],{"id":128,"by":129,"text":130},"15947261","Arnt","This sounds like an early version of DCC: https:\u002F\u002Fwww.rhyolite.com\u002Fdcc\u002F At first glance, I don't see anything that DCC didn't do, what did I miss?",{"value":132,"source":70,"evidence":133},"ConceptNet Numberbatch",[134],{"id":135,"by":136,"text":137},"15948323","physicsyogi","Maybe look into ConceptNet Numberbatch: https:\u002F\u002Fgithub.com\u002Fcommonsense\u002Fconceptnet-numberbatch\u002Fblob\u002Fmaster\u002FREADME.md",{"comments":40,"answered":31,"key":139},[140,141,142],{"id":50,"text":51},{"id":59,"text":60},{"id":143,"text":144},"15950371","One of the things we found out working on this project, is the problem of converting a word vector to a paragraph vector. Apparently there are many ways to do so and each one…",{"commenters":146,"max_depth":147,"maker_replies":40,"first_comment_mins":148,"lifespan_hours":149},17,4,146,28,[151,153,155,157,159,163,164,165,168,169,173,175,176,179,182,184,185,186,189,192,194],[73,32,38,152,74,32],"project\u002Fcritical\u002Fidea",[80,32,39,154,81,32],"project\u002Fpositive\u002Fidea",[93,32,32,156,74,32],"commenter\u002Fneutral\u002Fnone",[50,32,32,158,11,38],"maker\u002Fneutral\u002Fexecution",[160,38,40,161,162,32],"15946399","project\u002Fneutral\u002Fdocs","amelius",[112,38,40,161,113,32],[116,38,32,156,117,32],[166,38,32,156,167,32],"15947001","RHSman2",[59,38,32,158,11,38],[170,31,32,171,172,32],"15948131","other\u002Fneutral\u002Fidea","programmarchy",[135,31,32,174,136,32],"other\u002Fneutral\u002Fnone",[143,31,32,158,11,38],[177,40,32,171,178,32],"15949358","abc-xyz",[180,40,32,174,181,32],"15953885","codegladiator",[128,147,32,183,129,32],"commenter\u002Fneutral\u002Fidea",[84,147,39,154,85,32],[99,147,38,152,100,32],[187,39,32,174,188,32],"15946700","chasing",[190,39,32,156,191,32],"15947396","dang",[193,39,32,156,188,32],"15948198",[195,39,32,174,196,32],"15948260","erik_seaberg",[],[199,204,209,214,219,224],{"id":200,"name":201,"points":202,"category":203},"15945490","dumbcoin",412,"learning_science",{"id":205,"name":206,"points":207,"category":208},"15972402","Termipal",195,"dev_tools",{"id":210,"name":211,"points":212,"category":213},"15925736","Servers.lol",150,"web_saas",{"id":215,"name":216,"points":217,"category":218},"15919646","PyFunctional",129,"languages",{"id":220,"name":221,"points":222,"category":223},"15950325","pm-as-code",122,"productivity",{"id":225,"name":226,"points":227,"category":223},"15945658","Darkwire.io",118,false,"v2",{"id":231,"name":5,"line":6,"ai":228,"cat":20,"year":13,"points":14,"comments":15,"rec":28,"s":232,"top":18,"kw":233,"maker":11,"domain":10,"tier":19,"attempts":32,"strip":234,"url":9,"mix":235,"praise":69,"objection":89,"sub":21},15945808,-0.07,[24,25,26,27],"150033000000000510000",[31,31,32,31],1793990967933]