Pipet

Dev tools

web scraping cli

SHOW HN2024

Pipet

CLI tool for scraping and extracting data online, with pipes

github.com

#web scraping#cli#data extraction#unix pipes#curl

49points
7comments
#1,307of 3,515 in 2024
Top 38%of 2024 by points

How HN took it

QuietFewer than 5 comments about the project

2 positive, 0 neutral or asking, 0 mixed, 1 critical, out of 3 comments about the project.

Performance: 0 for, 1 against

The thread, comment by comment

Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.

positive neutral or mixed critical about another commenter or something else the maker

What HN said

What people liked

  • Useful for people who keep one-off scraping scripts

    From 1 comment
    • Thus is great. Thanks for sharing. Sitting on a cache of one-off scripts as well. Looking forward to checking this out further.
      itsbrex on HN
  • Positive reaction to the project

    From 1 comment

Objections

  • Scraping needs advanced bot evasion such as real browsers and GPU-based human emulation

    From 1 comment
    • That's all nonsense. If it's not imitating a real user by launching a real browser on a GPU and moving the mouse cursor with AI like a real human would, you could throw this…
      cynicalsecurity on HN

Requests

  • Example of a site needing AI browser emulation to scrape

    From 1 comment
    • The idea is that you run this from your personal computer, so I don't see any issues with the network really? Regarding the AI emulation browser in GPU - do you have an example…
      yoavm on HN

Questions

  • Whether sites have advanced bot protection that the tool must handle

    From 1 comment
    • Because every website has such an advanced protection?
      moltar on HN

Category, keywords, summaries and labels on this page were made by AI (Thread Reader v2) and not checked by hand. Each point links to the comments it came from.

Add a badge to your READMEShown on HackerShow
[![Shown on HackerShow](https://hackershow.org/badge.svg)](https://hackershow.org/show/41695549)