s3-lambda
Dev tools
s3 utility library
SHOW HN2017
How HN took it
3 positive, 9 neutral or asking, 0 mixed, 11 critical, out of 23 comments about the project. Thread score -0.30.
Idea: 3 for, 5 againstDocs: 0 for, 3 againstExecution: 0 for, 1 againstDesign: 0 for, 1 againstPerformance: 0 for, 1 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Composable map/filter chains running in parallel over S3 objects is a strong design
From 1 comment
First impression: this is a brilliant piece of software design. The ability to compose a map/filter chain and execute it in parallel against every object in an S3 bucket that…
simonw on HN
Generic functional interface to S3 is useful
From 1 comment
Really nice to have a generic functional interface to S3. Thanks.
dhpe on HN
Objections
Name 'lambda' is confused with AWS Lambda; the functions run locally
From 3 comments
Might make sense to rename this to avoid confusion with AWS Lambda (I immediately thought it was related). Otherwise, looks like an awesome library!
dschnurr on HN... Except it's not! The "lambda" here isn't AWS Lambda. It's a locally executed function. Now if this scheduled a bunch of real Lambdas to execute the work for each bucket then…
koolba on HNI also came here thinking this is some sort of aws lambda triggered within the context of certain s3 file. I would say anyone who's heard of AWS lambda would think that way.…
cocktailpeanuts on HN
Listing all keys before running is slow for millions of files
From 2 comments
Getting aan index of (millions of) files on s3 is very slow for us, like, days. Is there anything you do to work around this? It seems since this is not an AWS Lambda project the…
kvz on HNThis does not have to do with AWS Lambda, I'm thinking about renaming it to "functional-s3", or something similar. To answer your question, there isn't really a workaround for…
wellsjohnston on HN
Destructive map by default is risky
From 1 comment
I thinking having the default be destructive for mapping is a strange design decision. That is going to bite someone one day soon.
wcdolphin on HN
Requests
Rename the project to avoid confusion with AWS Lambda
From 2 comments
Might make sense to rename this to avoid confusion with AWS Lambda (I immediately thought it was related). Otherwise, looks like an awesome library!
dschnurr on HNI also came here thinking this is some sort of aws lambda triggered within the context of certain s3 file. I would say anyone who's heard of AWS lambda would think that way.…
cocktailpeanuts on HN
Run the functions on real AWS Lambda
From 2 comments
... Except it's not! The "lambda" here isn't AWS Lambda. It's a locally executed function. Now if this scheduled a bunch of real Lambdas to execute the work for each bucket then…
koolba on HNBah. My first impression was totally wrong in that case. Here's hoping someone builds a version of this that executes magically in the lambda cloud.
simonw on HN
Stream keys with a generator instead of listing all first
From 1 comment
This does not have to do with AWS Lambda, I'm thinking about renaming it to "functional-s3", or something similar. To answer your question, there isn't really a workaround for…
wellsjohnston on HN
Questions
Is it subject to S3 eventual consistency
From 1 comment
Is this susceptible to any of S3's eventual consistency constraints?
cle on HN
Where does code run versus where is chaining done
From 1 comment
So... the client-side code iterates S3 objects matching a certain filter, and then schedules a lambda for each one of those objects. Is that right? Or does the iteration…
DenisM on HN
Alternatives named
AWS Athena for querying S3
From 1 comment
see also aws athena https://aws.amazon.com/athena/ ?
hayd on HN
Google BigQuery
From 1 comment
This is a nice project. For real-world use cases, we have good alternatives: 1. Migrate s3 ==> gc and use BigQuery which does support udf 2. Register to databricks (I'm not…
avip on HN
S3 inventory reports
From 1 comment
Have you looked into the inventory functionality? It was just added last November. http://docs.aws.amazon.com/AmazonS3/latest/dev/storage-inventory.html
otterley on HN
Databricks
From 1 comment
This is a nice project. For real-world use cases, we have good alternatives: 1. Migrate s3 ==> gc and use BigQuery which does support udf 2. Register to databricks (I'm not…
avip on HN