[{"data":1,"prerenderedAt":602},["ShallowReactive",2],{"project-14440993":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":27,"sentiment":28,"stance_mix":29,"scored_comments":33,"aspects":34,"strip":46,"post_text":48,"what":49,"said":86,"maker_in_thread":222,"shape":233,"grid":238,"attempts":563,"same_week":564,"maker_shows":47,"maker_hidden":594,"reader_version":595,"card":596},"14440993","Pipes","Early-stage Yahoo Pipes spiritual successor","title","Show HN: Early-stage Yahoo Pipes spiritual successor","https:\u002F\u002Fwww.pipes.digital\u002F","pipes.digital","onli",1496086869,2017,280,126,83,2242,4,"legend","dev_tools","visual programming","web_app",[24,25,21,26],"yahoo pipes","rss feeds","feed aggregation","mixed",0.077,{"positive":30,"neutral":31,"mixed":18,"critical":32},15,20,11,50,{"idea":35,"design":38,"docs":39,"privacy":41,"execution":43,"performance":44},{"positive":36,"critical":37},9,2,{"positive":18,"critical":18},{"positive":40,"critical":40},0,{"positive":42,"critical":40},1,{"positive":40,"critical":42},{"positive":40,"critical":45},3,[42,45,45,42,45,47,40,45,47,40,40,40,47,18,40,40,45,45,40,40,45,45,45,40,18,45,45,45,40,45,45,47,40,45,47,47,42,42,42,37],5,null,{"problem":50,"audience":48,"pricing":48,"motivation":60,"time_to_build":48,"stack":68,"openness":83,"stage":84,"team_size":85},{"value":51,"source":52,"evidence":53},"Lets people transform and combine RSS feeds with a visual wiring editor, as Yahoo Pipes did.","maker",[54,57],{"id":55,"by":11,"text":56},"14441371","I'm interested in the answer to this questions myself :) For me, this is about the ease of use of the visual programming interface, and the focus on feeds to enable the pipe…",{"id":58,"by":11,"text":59},"14441296","Every pipe is a recursive list of blocks, like a tree. Basically, a pipe has a starting block that produces the pipe output. To do that it calls the process function of its input…",{"value":61,"source":62,"evidence":63},"Recreate Yahoo Pipes after it was shut down.","inferred",[64,67],{"id":65,"by":11,"text":66},"14441126","I'd be interested to know which blocks and functionality is missing to support the use cases you used Yahoo Pipes for! And of course general feedback is always welcome.",{"id":55,"by":11,"text":56},[69,75,79],{"value":70,"source":52,"evidence":71},"Ruby",[72],{"id":73,"by":11,"text":74},"14549895","Pipes is a Ruby stack, with sinatra at its core. But viewing the HTML is actually all client side, meaning javascript. When the block output view gets more complicated I'll…",{"value":76,"source":52,"evidence":77},"Sinatra",[78],{"id":73,"by":11,"text":74},{"value":80,"source":52,"evidence":81},"JavaScript",[82],{"id":73,"by":11,"text":74},"open_source","beta","unknown",{"praise":87,"objections":104,"requests":137,"questions":166,"alternatives":181},[88],{"value":89,"source":90,"evidence":91},"Nostalgia for Yahoo Pipes and welcome revival of the tool","commenter",[92,96,100],{"id":93,"by":94,"text":95},"14442145","scottmcleod","I really loved Yahoo Pipes - Was ahead of its time if you ask me..Built some very cool automations on top of this. Basically was IFTTT and Zapier well before them.",{"id":97,"by":98,"text":99},"14442489","j45","Pipes was a handy tool, hope this project keeps up!",{"id":101,"by":102,"text":103},"14441600","LoSboccacc","I don't think you remember how the yahoo pipes worked. It was a brilliant mix of graphs for flow control, premade boxes for all common data manipulation and scripting for that…",[105,116,127],{"value":106,"source":90,"evidence":107},"Node-based visual wiring scales poorly to large programs",[108,112],{"id":109,"by":110,"text":111},"14441428","zeptomu","I've some experience in visual programming languages (both professional and in my studies) and I advise against the graph-form where you connect nodes and edges. This does not…",{"id":113,"by":114,"text":115},"14441678","jldugger","IMO, the reason the wiring model doesn't scale up is that most don't provide a way to create submodules (compounds?). I expect that decomposing a problem into a chain of smaller…",{"value":117,"source":90,"evidence":118},"Hard to see the output block and link pipeline to output",[119,123],{"id":120,"by":121,"text":122},"14441166","insomniacity","Nice. It's not very clear that you need to link the pipeline to the right-hand side. (I can't remember if you had to do that on Yahoo, it's been so long!). It's particularly a…",{"id":124,"by":125,"text":126},"14441532","anonfunction","I used to use yahoo pipes for a few twitter bots I had. There was an rss to twitter webapp that I hooked up and it was fully automated for years before it stopped working. I…",{"value":128,"source":90,"evidence":129},"Site was unreachable or slow when first posted",[130,134],{"id":131,"by":132,"text":133},"14441122","rjeli","Dead (could not connect to server)",{"id":135,"by":11,"text":136},"14441138","Please try again. It should be up (works for me and http:\u002F\u002Fdownforeveryoneorjustme.com\u002Fwww.pipes.digital supports that). Though I forgot scaling up the droplet it is running on…",[138,149,159],{"value":139,"source":90,"evidence":140},"Starter example pipes and code examples",[141,145],{"id":142,"by":143,"text":144},"14441512","mgkimsal","would like to suggest a couple of \"starter pipes\" - samples to more easily illustrate some possibilities.",{"id":146,"by":147,"text":148},"14442031","yownie","couple of code examples would be nice",{"value":150,"source":90,"evidence":151},"Support for scraping with CSS selectors or XPath",[152,156],{"id":153,"by":154,"text":155},"14441634","Toast_","Do you plan on supporting scraping content via css selectors\u002Fxpath\u002Fregex?",{"id":157,"by":11,"text":158},"14475808","I added a block to download a page (instead of a feed), another block for extracting content via css selector or xpath, and a feed builder block to later combine those (but the…",{"value":160,"source":90,"evidence":161},"JSON processing like jq",[162],{"id":163,"by":164,"text":165},"14441553","hehheh","Neat. Any chance of adding JSON processing where you can pick a key (ala `jq`[0]) and then combine it with RSS or other JSON or what have you? [0] https:\u002F\u002Fstedolan.github.io\u002Fjq\u002F",[167,174],{"value":168,"source":90,"evidence":169},"How does it compare to Zapier and IFTTT?",[170],{"id":171,"by":172,"text":173},"14441266","michaelbuckbee","This is cool, but I feel like Zapier, IFTTT and their open source equivalents have moved far past Yahoo Pipes. How do you see this stacking up?",{"value":175,"source":90,"evidence":176},"Does it support sub-pipe inputs and modules?",[177,178],{"id":113,"by":114,"text":115},{"id":179,"by":114,"text":180},"14442734","It only had submodules in the sense that you could embed a pipe, but IIRC, they were always sources -- you couldn't pipe data into them. It had a notion of the 'output' block but…",[182,193,200,204,208,215],{"value":183,"source":90,"evidence":184},"Node-RED",[185,189],{"id":186,"by":187,"text":188},"14442278","beardicus","I'm surprised nobody has mentioned Node-RED yet: http:\u002F\u002Fnodered.org\u002F Node.js-based with a Pipes-like visual wiring interface. It's quite popular with the Raspberry Pi and Arduino…",{"id":190,"by":191,"text":192},"14442645","Mister_Snuggles","I'd love something like this that I can run locally. From looking at the comments, it looks like Node-RED and NoFlo are two possibilities. I was intrigued by Yahoo Pipes a while…",{"value":194,"source":90,"evidence":195},"NoFlo",[196],{"id":197,"by":198,"text":199},"14441368","bergie","Interesting, have to take a proper look tomorrow. We've done similar things with NoFlo (https:\u002F\u002Fnoflojs.org)",{"value":201,"source":90,"evidence":202},"Zapier",[203],{"id":171,"by":172,"text":173},{"value":205,"source":90,"evidence":206},"IFTTT",[207],{"id":171,"by":172,"text":173},{"value":209,"source":90,"evidence":210},"Apache NiFi",[211],{"id":212,"by":213,"text":214},"14445331","technologia","Its interesting and I do applaud your work, but I'll probably just stick to Apache Nifi with its flexibility.",{"value":216,"source":90,"evidence":217},"KNIME and RapidMiner",[218],{"id":219,"by":220,"text":221},"14441914","bayonetz","Checkout Rapidminer and Knime for great examples of box'n'arrows applied to machine learning pipelines. I've used Rapidminer a fair bit and they really have a great model…",{"comments":223,"answered":224,"key":225},17,6,[226,229,232],{"id":227,"text":228},"14441771","I agree that it is difficult to tackle bigger problems with this model, and everything that deviates from rss in -> manipulation -> rss out is maybe better served with textual…",{"id":230,"text":231},"14442121","Some screenshots from old Yahoo pipes that show a bit how that worked: https:\u002F\u002Fimgur.com\u002Fa\u002Fp6uf0",{"id":55,"text":56},{"commenters":234,"max_depth":235,"maker_replies":223,"first_comment_mins":236,"lifespan_hours":237},76,8,25,365,[239,241,245,247,251,254,258,259,261,262,266,267,269,272,276,280,282,283,285,288,291,294,297,300,302,304,307,310,313,316,319,322,324,327,330,333,335,338,341,344,348,350,352,355,357,360,363,366,369,372,374,377,378,381,382,383,386,388,391,394,396,398,400,403,406,407,409,411,414,415,418,421,425,428,431,434,436,439,443,445,447,451,453,456,457,459,462,464,467,469,471,474,478,481,483,486,488,490,493,494,497,499,501,502,504,506,508,511,512,515,517,519,522,526,529,534,536,538,540,542,546,551,553,555,557,559],[109,40,42,240,110,40],"project\u002Fcritical\u002Fidea",[242,40,45,243,244,40],"14441448","project\u002Fneutral\u002Fidea","phreeza",[246,40,42,240,110,40],"14441471",[248,40,42,249,250,40],"14442130","project\u002Fcritical\u002Fdesign","fiatjaf",[252,40,47,253,244,40],"14443997","project\u002Fpositive\u002Fidea",[255,40,45,256,257,40],"14441461","project\u002Fmixed\u002Fidea","seanmcdirmid",[113,40,42,249,114,40],[227,40,40,260,11,42],"maker\u002Fneutral\u002Fnone",[219,40,47,253,220,40],[263,40,40,264,265,40],"14445739","other\u002Fneutral\u002Fnone","JPKab",[179,40,42,249,114,40],[268,40,40,264,257,40],"14442491",[270,40,42,249,271,40],"14442963","hyperpallium",[273,40,47,274,275,40],"14445055","project\u002Fpositive\u002Fdesign","dragonwriter",[277,40,40,278,279,40],"14441493","commenter\u002Fneutral\u002Fnone","metachris",[281,40,40,264,110,40],"14441630",[101,40,47,253,102,40],[284,40,47,253,110,40],"14441670",[286,40,40,287,250,40],"14442118","other\u002Fcritical\u002Fnone",[289,40,40,264,290,40],"14442338","robbiemitchell",[292,40,40,264,293,40],"14442453","jameskegel",[295,40,40,264,296,40],"14442584","bryanh",[298,40,40,264,299,40],"14469870","gsvclass",[230,40,40,301,11,42],"maker\u002Fneutral\u002Fdesign",[303,40,45,243,102,40],"14443194",[305,40,40,264,306,40],"14448830","ryanoshea",[308,40,47,274,309,40],"14441608","jacquesm",[311,40,45,243,312,40],"14441954","lobster_johnson",[314,40,40,264,315,40],"14441993","mastax",[317,40,40,264,318,40],"14442024","eriknstr",[320,40,40,264,321,40],"14444347","rubidium",[323,40,40,264,265,40],"14445759",[325,40,40,264,326,40],"14442076","Theodores",[328,40,40,264,329,40],"14442191","kls",[331,40,40,332,326,40],"14449183","other\u002Fpositive\u002Fnone",[334,40,40,264,329,40],"14449743",[336,40,40,264,337,40],"14442157","shiven",[339,40,47,253,340,40],"14442212","supergreg",[342,40,47,274,343,40],"14442672","robinhowlett",[345,40,45,346,347,40],"14442693","project\u002Fneutral\u002Fnone","dyeje",[349,40,45,346,343,40],"14476651",[351,40,47,274,265,40],"14445886",[353,40,45,354,110,40],"14446153","project\u002Fneutral\u002Fdesign",[356,40,45,354,343,40],"14476633",[358,40,40,264,359,40],"14442979","sidegrid",[361,40,40,264,362,40],"14442998","aaron-lebo",[364,40,40,264,365,40],"14443005","flukus",[367,40,40,264,368,40],"14443017","schoen",[370,40,40,264,371,40],"14443043","phillc73",[373,40,40,264,368,40],"14443052",[375,40,40,264,376,40],"14444721","regularfry",[186,42,45,346,187,40],[379,42,40,264,380,40],"14442426","perryprog",[171,37,45,243,172,40],[55,37,40,260,11,42],[384,37,45,243,385,40],"14441439","wocram",[387,37,40,264,296,40],"14442589",[389,37,40,264,390,40],"14444067","Muanh",[392,37,40,332,393,40],"14441582","kilroy123",[395,37,40,287,102,40],"14441612",[397,37,40,278,296,40],"14442599",[399,37,40,278,102,40],"14443189",[401,37,40,278,402,40],"14444050","GordonS",[404,37,40,278,405,40],"14475751","mjn",[65,45,40,260,11,42],[120,18,45,408,121,40],"project\u002Fmixed\u002Fdesign",[410,18,40,260,11,42],"14441380",[412,47,40,264,413,40],"14441985","unityByFreedom",[197,224,45,346,198,40],[142,416,45,417,143,40],7,"project\u002Fneutral\u002Fdocs",[419,235,40,264,420,40],"14444411","houshuang",[422,36,40,423,424,40],"14444021","other\u002Fneutral\u002Fidea","fareesh",[426,36,40,264,427,40],"14444057","bsandert",[429,36,40,264,430,40],"14444062","comboy",[432,36,40,433,11,42],"14444173","maker\u002Fneutral\u002Fidea",[124,435,45,408,125,40],10,[437,32,47,253,438,40],"14442716","Xeoncross",[440,441,45,243,442,40],"14441710",12,"mortadelegle",[444,441,40,253,11,42],"14441822",[190,446,45,346,191,40],13,[448,449,40,264,450,40],"14445820",14,"jonincanada",[452,449,40,260,11,42],"14446000",[454,449,40,278,455,40],"14447283","jwives",[163,30,45,243,164,40],[458,30,40,260,11,42],"14441633",[460,30,45,346,461,40],"14441738","arjie",[463,30,40,264,250,40],"14442165",[465,30,40,264,466,40],"14442913","gingerlime",[468,30,40,264,250,40],"14444190",[470,30,40,264,466,40],"14458433",[472,30,40,264,473,40],"14456546","reubano",[475,476,40,264,477,40],"14442083",16,"infectoid",[479,223,40,264,480,40],"14459084","hehejo",[153,482,45,346,154,40],18,[484,482,45,346,485,40],"14441715","RandomBookmarks",[487,482,40,264,154,40],"14441807",[489,482,40,260,11,42],"14441721",[491,482,47,492,154,40],"14441781","project\u002Fpositive\u002Fprivacy",[157,482,40,260,11,42],[495,482,45,496,154,40],"14548320","project\u002Fmixed\u002Fexecution",[498,482,40,260,11,42],"14549149",[500,482,40,278,154,40],"14549761",[73,482,40,260,11,42],[503,482,40,264,473,40],"14456297",[505,482,40,264,154,40],"14479615",[507,482,40,264,473,40],"14479680",[509,510,45,346,461,40],"14441249",19,[58,510,40,260,11,42],[513,31,47,253,514,40],"14441253","pbiggar",[516,31,40,260,11,42],"14441299",[93,518,47,253,94,40],21,[97,520,47,521,98,40],22,"project\u002Fpositive\u002Fnone",[523,524,40,423,525,40],"14442390",23,"dundercoder",[527,524,40,264,528,40],"14442713","Viper007Bond",[530,531,42,532,533,40],"14447868",24,"project\u002Fcritical\u002Fexecution","timvdalen",[535,531,40,260,11,42],"14448816",[212,236,42,537,213,40],"project\u002Fcritical\u002Fnone",[539,236,40,278,265,40],"14445801",[541,236,40,278,213,40],"14446169",[543,544,40,264,545,40],"14442088",26,"nthcolumn",[547,548,42,549,550,40],"14445522",27,"project\u002Fcritical\u002Fperformance","laretluval",[146,552,45,417,147,40],28,[131,554,42,549,132,40],29,[135,554,40,556,11,42],"maker\u002Fneutral\u002Fperformance",[558,554,42,549,132,40],"14441223",[560,561,47,253,562,40],"14441218",30,"RRRA",[],[565,570,575,579,584,589],{"id":566,"name":567,"points":568,"category":569},"14417340","ORY Editor",369,"web_saas",{"id":571,"name":572,"points":573,"category":574},"14439789","Legion",239,"languages",{"id":576,"name":577,"points":578,"category":574},"14447885","BuildMyVocab",225,{"id":580,"name":581,"points":582,"category":583},"14436381","Stretch",168,"consumer_social",{"id":585,"name":586,"points":587,"category":588},"14415126","µTT",115,"infrastructure",{"id":590,"name":591,"points":592,"category":593},"14444030","Programming with Escher",110,"learning_science",false,"v2",{"id":597,"name":5,"line":6,"ai":594,"cat":20,"year":13,"points":14,"comments":15,"rec":27,"s":598,"top":18,"kw":599,"maker":11,"domain":10,"tier":19,"attempts":40,"strip":600,"url":9,"mix":601,"praise":89,"objection":106,"sub":21},14440993,0.08,[24,25,21,26],"1331350350005400330033304333033503551112",[30,31,18,32],1794036555889]