This is a REST service implemented in GO (using GIN as a web server) that wraps embeddings of items (that are typically used to generate item recommendation) and serve item similarity requests over HTTP. Although it was mainly created for testing and debugging purposes, it enables you to quickly and efficiently serve REST based similarity requests using embeddings generated by your favorite matrix factorization or ML model. Simply point the server to an embedding file and an item name-id mapping and it can start serving requests as below.
On a reasonable scale, it runs a similarity search on an embedding really fast. It returns an answer for a similarity search over nearly 18K items, each represented by a vector of length 128, in about 30ms.
(Tested using the lastFM dataset)
Example for usage
curl http://127.0.0.1:8080/similar?to=Eminem&topk=10 |jq{
"original": "Eminem",
"similar": [
"Eminem",
"Ludacris",
"Jay-Z",
"Kanye West",
"2Pac",
"50 Cent",
"Juno Reactor vs. Don Davis",
"The Brothers Gutworm",
"Black Eyed Peas",
"Timbaland"
]
Projects/packages that are used here -
- Argsort - for fast matrix sorting - https://github.com/mkmik/argsort/blob/v1.1.0/argsort.go
- Facts - for recommend requests based on colaborative filtering
- gonum - for fast matrix and vector operations
- go learn - for the Cosine distance and Dot functions
- and Gin
The server can be easily compiled as
go build -o bin/server ./server and then run while pointing to the config file
./bin/server ./bin/server.config.yaml Config file should follow this structure
port: 8080
host: localhost
embedding_file: embedding_file.csv
items_file: item_name_id.csv
embedding_size: 129
The supported strcuture of the embedding file should be in the following CSV format:
id, vector_element_1, vector_element_2, vector_element_3 ......
The supported strcuture of the embedding map file should be in the following CSV format:
id, item_name
note that it is case sensitive