gnes.indexer.base module¶
-
class
gnes.indexer.base.
BaseChunkIndexer
(helper_indexer=None, *args, **kwargs)[source]¶ Bases:
gnes.indexer.base.BaseIndexer
Storing chunks and their vector representations
-
add
(keys, vectors, weights, *args, **kwargs)[source]¶ adding new chunks and their vector representations
Parameters: - keys (
List
[Tuple
[int
,int
]]) – list of (doc_id, offset) tuple - vectors (
ndarray
) – vector representations - weights (
List
[float
]) – weight of the chunks
- keys (
-
num_chunks
¶
-
num_docs
¶
-
train
(*args, **kwargs)¶ Train the model, need to be overrided
-
-
class
gnes.indexer.base.
BaseChunkIndexerHelper
(helper_indexer=None, *args, **kwargs)[source]¶ Bases:
gnes.indexer.base.BaseChunkIndexer
A helper class for storing chunk info, doc mapping, weights. This is especially useful when ChunkIndexer can not store these information by itself
-
add
(keys, weights, *args, **kwargs)[source]¶ adding new chunks and their vector representations
Parameters: - keys (
List
[Tuple
[int
,int
]]) – list of (doc_id, offset) tuple - vectors – vector representations
- weights (
List
[float
]) – weight of the chunks
Return type: int
- keys (
-
train
(*args, **kwargs)¶ Train the model, need to be overrided
-
-
class
gnes.indexer.base.
BaseDocIndexer
(normalize_fn=None, score_fn=None, is_big_score_similar=False, *args, **kwargs)[source]¶ Bases:
gnes.indexer.base.BaseIndexer
Storing documents and contents
Base indexer, a valid indexer must implement
add()
andquery()
methods-
add
(keys, docs, *args, **kwargs)[source]¶ adding new docs and their protobuf representation
Parameters: - keys (
List
[int
]) – list of doc_id - docs (
List
[Document
]) – list of protobuf Document objects
- keys (
-
train
(*args, **kwargs)¶ Train the model, need to be overrided
-
-
class
gnes.indexer.base.
BaseIndexer
(normalize_fn=None, score_fn=None, is_big_score_similar=False, *args, **kwargs)[source]¶ Bases:
gnes.base.TrainableBase
Base indexer, a valid indexer must implement
add()
andquery()
methods-
num_chunks
¶
-
num_docs
¶
-
train
(*args, **kwargs)¶ Train the model, need to be overrided
-
-
class
gnes.indexer.base.
JointIndexer
(*args, **kwargs)[source]¶ Bases:
gnes.base.CompositionalTrainableBase
-
components
¶
-
train
(*args, **kwargs)¶ Train the model, need to be overrided
-