HoverPy

AI

text classification

SHOW HN2016

HoverPy

Subreddit classifier using scikit-learn and a high-performance Go proxy

ioloop.io

#scikit-learn#text classification#subreddit classifier#http caching proxy#golang

170points
17comments
#182of 2,264 in 2016
Top 9%of 2016 by points

How HN took it

CuriousMostly questions and neutral comments

3 positive, 5 neutral or asking, 0 mixed, 1 critical, out of 9 comments about the project. Thread score +0.13.

Idea: 2 for, 0 againstDocs: 1 for, 1 against

The thread, comment by comment

Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.

positive neutral or mixed critical about another commenter or something else the maker

What HN said

What people liked

  • Clear, useful write-up and resource for learning text classification

    From 3 comments
    • This is really cool. I've been working on a search engine for lectures, and I recently set it up so you can filter conference talks by the programming language.…
      garysieling on HN
    • This is a really great resource. Thank you. I'm just getting started with experimenting in this topic, and it's great to see something that isn't about classifying flower petals…
      Spooky23 on HN
    • Agree, nice article! Afraid other than your posts making it to front page, I don't see how to keep checking: -- can't find an RSS feed for your blog -- ioloop.io has a blank…
      derekja on HN

Objections

  • No classifier accuracy or precision results reported, only 15 test cases

    From 1 comment
    • Wait, you had 15 test cases, and didn't even say how the classifier did? I guess I could run the code myself, and that's nice you provided it on Github, but still, I feel like…
      jackschultz on HN
  • Training data quality limits results; more data needed

    From 1 comment
    • Well sure it's possible, I've done writeups on text classification with Scikit-Learn as well here:…
      jackschultz on HN

Requests

  • Report precision metrics in a follow-up post

    From 2 comments
    • Wait, you had 15 test cases, and didn't even say how the classifier did? I guess I could run the code myself, and that's nice you provided it on Github, but still, I feel like…
      jackschultz on HN
    • Well sure it's possible, I've done writeups on text classification with Scikit-Learn as well here:…
      jackschultz on HN
  • Add RSS feed or a clear place to follow future posts

    From 2 comments
    • Agree, nice article! Afraid other than your posts making it to front page, I don't see how to keep checking: -- can't find an RSS feed for your blog -- ioloop.io has a blank…
      derekja on HN
    • +1
      mcncfie on HN
  • Explain why MultinomialNB was chosen

    From 1 comment
    • This is a great post, and overlaps somewhat with a project I'm working on (scraping and classifying large amounts of text). I know it's not the focus of the post, but was there…
      sandwell on HN

Questions

  • What is the use case for the caching proxy?

    From 2 comments
    • I'm still confused about the use case. Is the purpose of a cache like this to create a unified interface to accessing and storing web pages? So you can write code to scrape pages…
      maksimum on HN
    • It's possible that the proxy caches duplicate requests, e.g. if you're crawling articles linked on reddit.
      garysieling on HN
  • How does the proxy compare with scrapy httpcache middleware?

    From 1 comment
    • This is a great post, and overlaps somewhat with a project I'm working on (scraping and classifying large amounts of text). I know it's not the focus of the post, but was there…
      sandwell on HN

Alternatives named

  • H2O.ai, which exports classifiers as Java code

    From 1 comment
    • Looks very interesting, have you looked at H2O.ai which spits out a classifiers as Java code which can be wrapped in an ultra low latency API without caching.
      zero-x on HN
  • Scrapy with httpcache middleware

    From 1 comment
    • This is a great post, and overlaps somewhat with a project I'm working on (scraping and classifying large amounts of text). I know it's not the focus of the post, but was there…
      sandwell on HN

Category, keywords, summaries and labels on this page were made by AI (Thread Reader v2) and not checked by hand. Each point links to the comments it came from.

Add a badge to your READMEShown on HackerShow
[![Shown on HackerShow](https://hackershow.org/badge.svg)](https://hackershow.org/show/13211132)