Llama2 Embeddings FastAPI Server
AI
embeddings server
SHOW HN2023
How HN took it
1 positive, 7 neutral or asking, 0 mixed, 8 critical, out of 16 comments about the project. Thread score -0.34.
Idea: 0 for, 6 againstExecution: 1 for, 1 againstPerformance: 0 for, 1 against
The thread, comment by comment
Each row is a top-level comment and its replies, in HN's order. Select a cell to open that comment on HN.
positive neutral or mixed critical about another commenter or something else the maker
What HN said
What people liked
Clean and simple codebase
From 1 comment
Hi, Looks quite clean, congrats. Two questions: 1. Starting from this, what would be the proper way to create embeddings for a complete document (i.e. a long paragraph)? My goal…
lgvld on HN
Useful features like token-level embeddings and similarity measures
From 1 comment
Cool. I was wondering what Tanimoto meant, since I've tried to make myself familiar with all the useful similarity measures, and apparently it's just another name for Jaccard…
eigenvalue on HN
Objections
Raw Llama2 likely underperforms dedicated embedding models like bge, gte and e5
From 2 comments
It seems unlikely that raw llama2 will perform better than purpose made encoder models like bge [1], gte [2], e5 [3] or instructor, despite it's much much larger size (for the…
Palmik on HNThe SGPT model is a very high performing text embeddings model adapted from a decoder. Using the same techniques with Llama-2 might perform better than you expect. I think…
readyplayeremma on HN
7B model is overkill for semantic embeddings
From 1 comment
I think it's still overkill though for semantic embedding, SBERT is on order of ~250mm parameters while smallest llama at 7b parameters.
jerrygenser on HN
Requests
Benchmarks against dedicated embedding models or OpenAI
From 2 comments
Nice work and starred! I am curious how you get the embeddings computed, the # of dimensions for the embeddings and if you have run any benchmarks against OpenAI's offering?…
lhr0909 on HNIt seems unlikely that raw llama2 will perform better than purpose made encoder models like bge [1], gte [2], e5 [3] or instructor, despite it's much much larger size (for the…
Palmik on HN
Word-level highlighting examples for token embeddings
From 1 comment
I’d love to see some examples of how the token-in-context embeddings stack up against sentence-level. What new use-cases are unlocked? Perhaps semantic search with…
clbrmbr on HN
Questions
Does it need a GPU and does inference run on the VPS
From 1 comment
Hi, Looks quite clean, congrats. Two questions: 1. Starting from this, what would be the proper way to create embeddings for a complete document (i.e. a long paragraph)? My goal…
lgvld on HN
How to embed whole documents versus sentences
From 2 comments
Hi, Looks quite clean, congrats. Two questions: 1. Starting from this, what would be the proper way to create embeddings for a complete document (i.e. a long paragraph)? My goal…
lgvld on HNThank you very much! And when is it useful to compute an embedding for each sentence of a document/paragraph then?
lgvld on HN
Alternatives named
bge, gte and e5 embedding models
From 1 comment
It seems unlikely that raw llama2 will perform better than purpose made encoder models like bge [1], gte [2], e5 [3] or instructor, despite it's much much larger size (for the…
Palmik on HN
InstructorEmbedding
From 2 comments
I'd rather stick with InstructEmbedding instead of pandering to flavour of the month LLM. That way I keep my key components insulated from drastic changes
villgax on HNHere's an example of Instructor Embeddings w/FeatureBase: https://gist.github.com/kordless/aae99946e7e2a5afccc83f3c4eeee65a Instructor Embedding rank high on various leaderboards…
kordlessagain on HN
Torchserve
From 1 comment
What's wrong with just using Torchserve[1]? We've been using it to serve embedding models in production. [1] https://pytorch.org/serve/
rvrs on HN