crzy64

Languages and libraries

encoding library

SHOW HN2021

crzy64

"crzy64", base64 mod aimed for fastest decoding

github.com

#base64#base64 decoding#fast decoding#simd#swar

47points
18comments
#890of 2,533 in 2021
Top 36%of 2021 by points

How HN took it

CuriousMostly questions and neutral comments

1 positive, 8 neutral or asking, 0 mixed, 3 critical, out of 12 comments about the project. Thread score -0.08.

Idea: 1 for, 2 againstPerformance: 0 for, 1 against

The thread, comment by comment

Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.

positive neutral or mixed critical about another commenter or something else the maker

What HN said

What people liked

  • Clear algorithm description of the bit reordering and XOR trick

    From 1 comment
    • It would have been better to have an algorithm description somewhere, but it is not hard to follow. So crzy64 is essentially base64 plus two optimizations: - Output bytes…
      lifthrasiir on HN

Objections

  • Thread is sparse on details about the modifications

    From 1 comment
    • I want to say this looks interesting but it's so sparse on details it's hard to say. What modifications were made to base64? Is the only modification replacing "+" with "."? Is…
      cornstalks on HN
  • Memory bandwidth limits gains; may not be faster than memcpy

    From 3 comments
    • Isn't base64 encoding/decoding bound to memory bandwidth? If so, won't all 4/3 memory formats encode/decode at the same speed?
      vortico on HN
    • Memory bandwidth is the limit, so you can't make it faster than memcpy(). However, copying memory is still faster than these algorithms, аnd it's complicated to make the…
      jpegqs on HN
    • Memory bandwidth is a complex beast, one should be able to get 50GB/s for short decodes of single to double digit k on modern hardware. The author measured 11GB/s ish in their…
      sitkack on HN
  • Prior art exists (UU-encode, Ascii85, SSE base64)

    From 3 comments
    • I wonder if they just reinvented UU-(en,de)code: https://en.wikipedia.org/wiki/Uuencoding I guess it'll go... nowhere.
      bellyfullofbac on HN
    • There's been prior art, I was immediately reminded of Ascii85 https://en.wikipedia.org/wiki/Binary-to-text_encoding#Encoding_standards Incidentally every Windows desktop now has…
      Thoreandan on HN
    • There has been work done one optimizing regular base64 using SSE: http://www.alfredklomp.com/programming/sse-base64/
      superjan on HN

Requests

  • Benchmarks for decoding many small inputs

    From 2 comments
    • I want to say this looks interesting but it's so sparse on details it's hard to say. What modifications were made to base64? Is the only modification replacing "+" with "."? Is…
      cornstalks on HN
    • > Is the rest of the table[1] unchanged? The rest is unchanged, but there is a trick in that the table indexes are shuffled, and shuffled differently depending on the neighboring…
      jpegqs on HN
  • Support for URL-safe base64url characters

    From 1 comment
    • I want to say this looks interesting but it's so sparse on details it's hard to say. What modifications were made to base64? Is the only modification replacing "+" with "."? Is…
      cornstalks on HN

Questions

  • What 64 characters does it encode to?

    From 2 comments
    • Can you tell us what set of 64 characters it encodes to?
      rep_movsd on HN
    • Going off the readme, it appears to use "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789./" though not necessarily in that order.
      skykooler on HN
  • How does it compare to optimized standard base64 implementations?

    From 1 comment
    • I want to say this looks interesting but it's so sparse on details it's hard to say. What modifications were made to base64? Is the only modification replacing "+" with "."? Is…
      cornstalks on HN

Alternatives named

  • Standard SSE base64 implementation by Alfred Klomp

    From 1 comment
    • There has been work done one optimizing regular base64 using SSE: http://www.alfredklomp.com/programming/sse-base64/
      superjan on HN
  • UU-encoding

    From 1 comment
    • I wonder if they just reinvented UU-(en,de)code: https://en.wikipedia.org/wiki/Uuencoding I guess it'll go... nowhere.
      bellyfullofbac on HN
  • Ascii85

    From 1 comment
    • There's been prior art, I was immediately reminded of Ascii85 https://en.wikipedia.org/wiki/Binary-to-text_encoding#Encoding_standards Incidentally every Windows desktop now has…
      Thoreandan on HN

Category, keywords, summaries and labels on this page were made by AI (Thread Reader v2) and not checked by hand. Each point links to the comments it came from.

Add a badge to your READMEShown on HackerShow
[![Shown on HackerShow](https://hackershow.org/badge.svg)](https://hackershow.org/show/29637986)