RULER
AI
reinforcement learning
SHOW HN2025
81points
11comments
#882of 3,837 in 2025
Top 23%of 2025 by points
How HN took it
CuriousMostly questions and neutral comments
3 positive, 3 neutral or asking, 0 mixed, 0 critical, out of 6 comments about the project. Thread score +0.33.
Idea: 1 for, 0 againstDocs: 1 for, 0 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Interesting, more general than prior RLPR-style approaches
From 2 comments
Excellent, look forward to giving this a go. I was looking at: https://arxiv.org/abs/2506.18254 but your approach is even more general.
sadiq on HNI really like RLPR for when you have a known-good answer to compare to as well!
kcorbitt on HN
Writeups are appreciated
From 1 comment
Love these write ups!
someoneontenet on HN
Objections
Name collides with NVIDIA's RULER long-context benchmark
From 2 comments
Might end up being some confusion with the RULER benchmark from NVIDIA given the (somewhat shared) domain: https://github.com/NVIDIA/RULER EDIT: by shared I only mean the…
spmurrayzzz on HNyeah unforutnately for you this is one of the well known long context benchmarks. too late tho, soldier on.
swyx on HN
Possible ordering bias in LLM judge ranking
From 1 comment
Very cool. Do you do anything to mitigate ordering bias in the evaluation function, or do you just expect it to average out over time?
maxrmk on HN
Requests
Explain why o3 underperforms Qwen on the customer support agent task
From 1 comment
how does o3 on the customer support agent task so dreadfully underperform qwen?
swyx on HN
Questions
Whether ordering bias is mitigated in the judge
From 1 comment
Very cool. Do you do anything to mitigate ordering bias in the evaluation function, or do you just expect it to average out over time?
maxrmk on HN
Alternatives named
RLPR
From 2 comments
Excellent, look forward to giving this a go. I was looking at: https://arxiv.org/abs/2506.18254 but your approach is even more general.
sadiq on HNI really like RLPR for when you have a known-good answer to compare to as well!
kcorbitt on HN