4 Comments
User's avatar
⭠ Return to thread
Param Singh's avatar

so if i had to do my own embedding search kind of thing, what would I use? is fastText the best right now?

Expand full comment
TokenBender's avatar

I'll use the word nobody likes - "Depends" but most likely no. SoTA performing embeddings would be already incorporating a multitude of strategies. You'll have to check best one based on your task.

You can do that via MTEB -

https://huggingface.co/blog/mteb

https://huggingface.co/spaces/mteb/leaderboard

In general, instructor embeddings are the best for the average performance but openAI embeddings are the best if you want minimum length of a document to be of high dimensions.

Expand full comment