[{"data":1,"prerenderedAt":416},["ShallowReactive",2],{"project-12136011":3},{"id":4,"name":5,"line":6,"line_source":7,"title":8,"url":9,"domain":10,"maker":11,"posted_at":12,"year":13,"points":14,"comments":15,"rank_in_year":16,"shown_in_year":17,"top":18,"tier":19,"category":20,"subcategory":21,"format":22,"keywords":23,"reception":29,"sentiment":30,"stance_mix":31,"scored_comments":35,"aspects":36,"strip":42,"post_text":44,"what":45,"said":84,"maker_in_thread":210,"shape":219,"grid":223,"attempts":378,"same_week":379,"maker_shows":18,"maker_hidden":408,"reader_version":409,"card":410},"12136011","Riko","A Python stream processing engine modeled after Yahoo! Pipes","title","Show HN: Riko – A Python stream processing engine modeled after Yahoo! Pipes","https:\u002F\u002Fgithub.com\u002Fnerevu\u002Friko","github.com","reubano",1469098656,2016,283,67,74,2264,4,"legend","languages","stream processing library","library",[24,25,26,27,28],"stream processing","python","yahoo pipes","etl","rss","loved",0.274,{"positive":32,"neutral":33,"mixed":34,"critical":34},5,11,0,16,{"design":37,"idea":39,"docs":41},{"positive":38,"critical":34},1,{"positive":40,"critical":34},2,{"positive":38,"critical":34},[34,34,34,34,43,43,34,43,34,34,43,32,32,43,34,43,34,34,34,43,43,34,43,34,34,34,43,32,32,34,34,34,34,34,34,34,34,34,32,43],3,null,{"problem":46,"audience":53,"pricing":44,"motivation":65,"time_to_build":44,"stack":69,"openness":82,"stage":83,"team_size":83},{"value":47,"source":48,"evidence":49},"Processing and analyzing streams of structured data without a complex setup","maker",[50],{"id":51,"by":11,"text":52},"12136031","`riko` is pure python stream processing library for analyzing and processing streams of structured data. It's modeled after Yahoo! Pipes [1] and was originally a fork of pipe2py…",{"value":54,"source":55,"evidence":56},"Python programmers and former Yahoo Pipes users","inferred",[57,61],{"id":58,"by":59,"text":60},"12136100","Fuzzwah","I was a heavy user of pipes and I'm now a heavy user of python. I have built my own dodgy simple replacement for some of the things I used to rely on pipes for. I'm very eager to…",{"id":62,"by":63,"text":64},"12136412","mxuribe","While I didn't use yahoo pipes too often, I loved it. Having this as a python library (I'm trying to get deeper into python), is great! Kudos and good luck!",{"value":66,"source":48,"evidence":67},"Pure Python alternative to Yahoo! Pipes for streaming data",[68],{"id":51,"by":11,"text":52},[70,74,78],{"value":71,"source":48,"evidence":72},"Python",[73],{"id":51,"by":11,"text":52},{"value":75,"source":48,"evidence":76},"Twisted",[77],{"id":51,"by":11,"text":52},{"value":79,"source":48,"evidence":80},"multiprocessing",[81],{"id":51,"by":11,"text":52},"open_source","unknown",{"praise":85,"objections":105,"requests":123,"questions":148,"alternatives":163},[86,94,99],{"value":87,"source":88,"evidence":89},"Readme is well done and could guide other projects","commenter",[90],{"id":91,"by":92,"text":93},"12139873","mkesper","This readme could be a guide for many projects!",{"value":95,"source":88,"evidence":96},"Brings back Yahoo Pipes in a Python library",[97,98],{"id":58,"by":59,"text":60},{"id":62,"by":63,"text":64},{"value":100,"source":48,"evidence":101},"Functional API is succinct compared to class-based alternatives",[102],{"id":103,"by":11,"text":104},"12137163","Interesting project. I hadn't come across this one yet. One difference that riko has is it's based around functions whereas this library (and practically every stream processing…",[106,112],{"value":107,"source":48,"evidence":108},"Pull-based generator design can't fan out without loading data into memory",[109],{"id":110,"by":11,"text":111},"12137787","Cool! I actually starred your project last year. Never really got around to looking under the hood though. How do you handle the \"multiple destinations\" part? In python you can…",{"value":113,"source":88,"evidence":114},"Push model is easier and enables back pressure",[115,119],{"id":116,"by":117,"text":118},"12138972","greenshackle","Python 3.5 introduced the async\u002Fawait syntax, I don't do C# but at a glance it's the same. I've been working on a similar project and I've also found the push model easier.",{"id":120,"by":121,"text":122},"12143698","infinite8s","Another benefit of the push model is that back pressure becomes much easier to implement.",[124,134,141],{"value":125,"source":88,"evidence":126},"Add a GUI",[127,131],{"id":128,"by":129,"text":130},"12136734","tudorw","if someone can spin up a usable gui, charge enough to make a living without compromising on performance, promise some longevity and a way to export of my stuff I would probably…",{"id":132,"by":11,"text":133},"12137052","Have you investigated any of the existing GUIs? [1-3] I'd love to hear your thoughts on their pros\u002Fcons. I do plan to integrate a nice GUI framework if I can find one. [1]…",{"value":135,"source":88,"evidence":136},"Dask integration for distributed processing",[137],{"id":138,"by":139,"text":140},"12137402","tanlermin","Can you consider Dask integration? http:\u002F\u002Fdistributed.readthedocs.io\u002Fen\u002Flatest\u002Fqueues.html https:\u002F\u002Fgithub.com\u002Fdask\u002Fdask It can handle parallel and distributed parts for you.…",{"value":142,"source":88,"evidence":143},"Twitter support",[144],{"id":145,"by":146,"text":147},"12136408","satai","Looks nice. Are there any plans for twitter support?",[149,156],{"value":150,"source":88,"evidence":151},"What kinds of applications people use it for",[152],{"id":153,"by":154,"text":155},"12137779","et2o","Looks interesting. What kind of applications do people use this for?",{"value":157,"source":88,"evidence":158},"Whether it will integrate with Apache Beam",[159],{"id":160,"by":161,"text":162},"12140676","ecesena","This is really interesting. Have you looked at Apache Beam? What I think is interesting about Beam -in this specific context- is that it has a standalone runner (java), that…",[164,171,178,185,192,196,203],{"value":165,"source":88,"evidence":166},"Yahoo Pipes",[167],{"id":168,"by":169,"text":170},"12137618","olviko","Nice project. I wrote something similar in C# long time ago [1]. Mostly to monitor job feeds and craigslist [2] :-) It supports RSS and Atom, async, various filters,…",{"value":172,"source":88,"evidence":173},"Motorway",[174],{"id":175,"by":176,"text":177},"12136859","oellegaard","If you're looking for a stream processing engine more close to Storm, etc. but also simple, check out Motorway: https:\u002F\u002Fgithub.com\u002Fplecto\u002Fmotorway :-)",{"value":179,"source":88,"evidence":180},"Node-RED",[181],{"id":182,"by":183,"text":184},"12139058","xnxn","http:\u002F\u002Fnodered.org\u002F",{"value":186,"source":88,"evidence":187},"Apache NiFi",[188],{"id":189,"by":190,"text":191},"12139507","qw","Apache Nifi looks promising. https:\u002F\u002Fnifi.apache.org",{"value":193,"source":88,"evidence":194},"Apache Beam",[195],{"id":160,"by":161,"text":162},{"value":197,"source":88,"evidence":198},"Plagger",[199],{"id":200,"by":201,"text":202},"12136409","raimue","I am still a user of Plagger [1], but development halted quite some time ago. Maybe this could be a good replacement. [1] https:\u002F\u002Fgithub.com\u002Fmiyagawa\u002Fplagger",{"value":204,"source":88,"evidence":205},"Apache Camel",[206],{"id":207,"by":208,"text":209},"12137802","svieira","Also in this space (and worth looking at for inspiration, especially for other potential sources and sinks of data) - Apache Camel [1]. [1]: http:\u002F\u002Fcamel.apache.org\u002F",{"comments":211,"answered":212,"key":213},24,7,[214,215,216],{"id":51,"text":52},{"id":110,"text":111},{"id":217,"text":218},"12139154","Care to share your code? I'd like to see a proper push implementation.",{"commenters":220,"max_depth":221,"maker_replies":211,"first_comment_mins":32,"lifespan_hours":222},29,6,365,[224,226,228,230,233,234,235,236,238,241,244,246,249,251,255,258,260,263,266,268,270,272,273,275,277,279,281,283,284,286,287,289,291,292,293,296,298,300,303,305,309,311,314,316,319,321,324,326,328,331,334,337,338,340,342,347,349,352,354,357,359,362,364,367,371,374,376],[51,34,34,225,11,38],"project\u002Fneutral\u002Fidea",[168,34,34,227,169,34],"commenter\u002Fpositive\u002Fidea",[110,34,34,229,11,38],"project\u002Fneutral\u002Fdesign",[231,34,34,232,169,34],"12138198","commenter\u002Fneutral\u002Fdesign",[116,34,34,232,117,34],[217,34,34,232,11,38],[120,34,43,229,121,34],[237,34,34,232,11,38],"12145820",[239,34,43,229,240,34],"12138195","rubber_duck",[242,34,34,243,169,34],"12138370","commenter\u002Fcritical\u002Fdesign",[245,34,34,232,240,34],"12138773",[247,34,43,225,248,34],"12138682","alexchamberlain",[250,34,34,225,11,38],"12138825",[252,34,34,253,254,34],"12139092","other\u002Fneutral\u002Fidea","asavinov",[256,34,34,257,11,38],"12139217","project\u002Fneutral\u002Fdocs",[259,34,34,253,254,34],"12139313",[261,34,34,262,248,34],"12139244","commenter\u002Fneutral\u002Fidea",[264,34,43,225,265,34],"12143532","visarga",[58,38,32,267,59,34],"project\u002Fpositive\u002Fidea",[269,38,34,257,11,38],"12136126",[91,38,32,271,92,34],"maker\u002Fpositive\u002Fdocs",[138,40,43,225,139,34],[274,40,34,225,11,38],"12137591",[175,43,34,276,176,34],"other\u002Fneutral\u002Fnone",[103,43,34,278,11,38],"project\u002Fpositive\u002Fdesign",[280,43,43,229,117,34],"12137925",[282,43,34,229,11,38],"12138244",[200,18,34,276,201,34],[285,18,34,225,11,38],"12136487",[160,32,34,253,161,34],[288,32,34,225,11,38],"12147201",[290,32,34,276,161,34],"12233294",[128,221,43,229,129,34],[132,221,34,229,11,38],[294,221,43,229,295,34],"12137172","beardicus",[297,221,34,229,11,38],"12137209",[299,221,43,229,295,34],"12138440",[301,221,34,302,11,38],"12153975","maker\u002Fneutral\u002Fdesign",[304,221,34,232,129,34],"12142207",[306,221,34,307,308,34],"12137808","other\u002Fneutral\u002Fpricing","drdoom",[189,221,34,310,190,34],"other\u002Fpositive\u002Fnone",[312,212,34,276,313,34],"12139946","ewindisch",[315,212,34,276,11,38],"12140039",[153,317,43,318,154,34],8,"project\u002Fneutral\u002Fnone",[320,317,34,225,11,38],"12137946",[322,317,32,323,154,34],"12141274","maker\u002Fpositive\u002Fnone",[62,325,32,267,63,34],9,[207,327,34,276,208,34],10,[329,327,34,330,11,38],"12138430","project\u002Fcritical\u002Fdocs",[332,327,34,333,190,34],"12139586","commenter\u002Fneutral\u002Fdocs",[335,33,34,276,336,34],"12138231","aioprisan",[182,33,34,276,183,34],[339,33,34,310,336,34],"12139682",[341,33,34,276,11,38],"12139714",[343,344,34,345,346,34],"12138924",12,"other\u002Fpositive\u002Fperformance","pastaking",[348,344,34,276,11,38],"12138952",[350,344,34,253,351,34],"12143891","agallego",[353,344,34,276,351,34],"12143895",[355,344,34,356,11,38],"12147279","maker\u002Fneutral\u002Fidea",[358,344,34,345,351,34],"12169238",[360,344,34,262,361,34],"12143905","bimil",[363,344,34,227,11,38],"12147243",[365,344,34,366,361,34],"12169186","commenter\u002Fneutral\u002Fnone",[368,369,32,278,370,34],"12136830",13,"DyslexicAtheist",[372,369,34,373,11,38],"12137063","maker\u002Fneutral\u002Fnone",[145,375,43,225,146,34],14,[377,375,34,225,11,38],"12136434",[],[380,385,390,395,399,404],{"id":381,"name":382,"points":383,"category":384},"12166687","Web Design in 4 minutes",1624,"learning_science",{"id":386,"name":387,"points":388,"category":389},"12159249","Kubernetes Certificate Manager",186,"infrastructure",{"id":391,"name":392,"points":393,"category":394},"12163689","github-widget",176,"dev_tools",{"id":396,"name":397,"points":398,"category":20},"12175830","sha256-simd",156,{"id":400,"name":401,"points":402,"category":403},"12166370","EventQL",90,"data",{"id":405,"name":406,"points":407,"category":394},"12137698","kilo",85,false,"v2",{"id":411,"name":5,"line":6,"ai":408,"cat":20,"year":13,"points":14,"comments":15,"rec":29,"s":412,"top":18,"kw":413,"maker":11,"domain":10,"tier":19,"attempts":34,"strip":414,"url":9,"mix":415,"praise":87,"objection":107,"sub":21},12136011,0.27,[24,25,26,27,28],"0000330300355303000330300035500000000053",[32,33,34,34],1794036546033]