openbilibili/vendor/github.com/go-ego/riot

Riot search

Go Open Source, Distributed, Simple and efficient full text search engine.

简体中文

Features

• Efficient indexing and search (1M blog 500M data 28 seconds index finished, 1.65 ms search response time, 19K search QPS）
• Support for logical search
• Support Chinese word segmentation (use gse word segmentation package concurrent word, speed 27MB / s）
• Support the calculation of the keyword in the text close to the distance（token proximity）
• Support calculation BM25 correlation
• Support custom scoring field and scoring rules
• Support add online, delete index
• Support heartbeat
• Support multiple persistent storage
• Support distributed index and search

• Can be achieved distributed index and search

• Look at Word segmentation rules

Riot v0.10.0 was released in Nov 2017, check the Changelog for the full details.

Requirements

Go version >= 1.8

Vendored Dependencies

Riot uses dep to vendor dependencies, but we don't commit the vendored packages themselves to the Riot git repository. Therefore, a simple go get is not supported because the command is not vendor aware.

Please manage it with dep, run dep ensure, to clone dependencies.

Installation/Update

go get -u github.com/go-ego/riot

Build-tools

go get -u github.com/go-ego/re 

re riot

To create a new riot application

$ re riot my-riotapp

re run

To run the application we just created, you can navigate to the application folder and execute:

$ cd my-riotapp && re run

Usage:

Look at an example

package main

import (
	"log"

	"github.com/go-ego/riot"
	"github.com/go-ego/riot/types"
)

var (
	// searcher is coroutine safe
	searcher = riot.Engine{}
)

func main() {
	// Init
	searcher.Init(types.EngineOpts{
		// Using:             4,
		NotUseGse: true,
		})
	defer searcher.Close()

	text := "Google Is Experimenting With Virtual Reality Advertising"
	text1 := `Google accidentally pushed Bluetooth update for Home
	speaker early`
	text2 := `Google is testing another Search results layout with 
	rounded cards, new colors, and the 4 mysterious colored dots again`
	
	// Add the document to the index, docId starts at 1
	searcher.Index(1, types.DocData{Content: text})
	searcher.Index(2, types.DocData{Content: text1}, false)
	searcher.IndexDoc(3, types.DocData{Content: text2}, true)

	// Wait for the index to refresh
	searcher.Flush()
	// engine.FlushIndex()

	// The search output format is found in the types.SearchResp structure
	log.Print(searcher.Search(types.SearchReq{Text:"google testing"}))
}

It is very simple!

Use default engine:

package main

import (
	"log"

	"github.com/go-ego/riot"
	"github.com/go-ego/riot/types"
)

var (
	searcher = riot.New("zh")
)

func main() {
	data := types.DocData{Content: `I wonder how, I wonder why
		, I wonder where they are`}
	data1 := types.DocData{Content: "所以, 你好, 再见"}
	data2 := types.DocData{Content: "没有理由"}
	searcher.Index(1, data)
	searcher.Index(2, data1)
	searcher.Index(3, data2)
	searcher.Flush()

	req := types.SearchReq{Text: "你好"}
	search := searcher.Search(req)
	log.Println("search...", search)
}

Look at more Examples

Look at Store example

Look at Logic search example

Look at Pinyin search example

Look at different dict and language search example

Look at benchmark example

Riot search engine templates, client and dictionaries

Donate

Supporting riot, buy me a coffee.

Paypal

Donate money by paypal to my account vzvway@gmail.com

License

Riot is primarily distributed under the terms of the Apache License (Version 2.0), base on wukong.