llm-gguf 0.2, now with embeddings

from blog Simon Willison's Weblog, | ↗ original
llm-gguf 0.2, now with embeddings This new release of my llm-gguf plugin - which adds support for locally hosted GGUF LLMs - adds a new feature: it now supports embedding models distributed as GGUFs as well. This means you can use models like the bafflingly small (30.8MB in its smallest quantization) mxbai-embed-xsmall-v1 with LLM like this: llm...