many-requests
Languages and libraries
python library
SHOW HN2020
How HN took it
2 positive, 7 neutral or asking, 0 mixed, 6 critical, out of 15 comments about the project. Thread score -0.13.
Idea: 2 for, 2 againstPerformance: 0 for, 3 againstExecution: 0 for, 1 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Useful for high-volume request workloads
From 2 comments
Oh, handy. This would have solved the exact problem I had last month (which I solved with just-good-enough fiddling with async python, which I'm not comfortable in yet)
roywiggins on HNNice job! Is vanilla Python networking is still dog slow, on the order of 1k requests per second per core on mid-range consumer hardware no matter how you wire it all together?
hansvm on HN
Objections
Python is too slow for heavy load; Go or Java preferred
From 1 comment
Really it's better to use a fast language like Go or Java for huge loads like this. Our typical CRUD Spring backends can handle ~15,000 requests a second with no optimization. I…
throwaway189262 on HN
Threads may outperform asyncio for I/O throughput
From 1 comment
Do you have empirical evidence for this? Even theoretically, threads can perform better because pythom does infact release the Gil for i/o bound operations and also, not blocking…
devxpy on HN
Requests
Explain why not use multiprocessing for more cores
From 1 comment
Why not distribute the concurrent work over multiple processes to get the most out of multiple cores? More event loops, better performance, and maybe more text preprocessing.
nurettin on HN
Questions
Is work distributed across cores or single-threaded
From 1 comment
when you say parallel you mean the work is distributed across multiple cores? e.g. multiple python worker processes. Or is all work running single threaded in a single process…
glic3rinu on HN
Why use this over ThreadPoolExecutor
From 1 comment
I'm curious why one would suggest using this over the built-in ThreadPoolExecutor class? Here's a complete, similar example from the Python docs:…
eatonphil on HN
Alternatives named
concurrent.futures ThreadPoolExecutor
From 1 comment
I'm curious why one would suggest using this over the built-in ThreadPoolExecutor class? Here's a complete, similar example from the Python docs:…
eatonphil on HN
Go with colly
From 1 comment
Look into Go, specifically the colly pkg. Learnt Go just so I could use this library after wrestling with highly parralelized Python web scraping. The 1k/requests/sec/core figure…
genidoi on HN