Indexing

From FindAndLearn::Cookbook
Jump to: navigation, search

Disambiguation[edit source]

Entities keyworded Indexing[edit source]


Indexing Terms Mentioned In

Indexing Domains:


Indexing involves 2 steps:

  1. Feeding resources from a ResourceSilo into a Datastore
  2. Applying an Indexer to resources in a Datastore

Feeding resources from a ResourceSilo into a Datastore[edit | edit source]

See SystemComponent "datastores"

AAAA-Ingestion "From E-Mail account to Datastore by"

Data fed?
Configuration
Subsequent pipeline
Tools

AAAA-Ingestion "From File system to Datastore by https://github.com/dataspects/dsdocument-cli"

Data fed?
Configuration

Configure the datastore:

<Datastore ID>        automatic (e.g. 12)
<Datastore Label>     Shown in information sources list on https://ui.dataspects.com/search
<Datastore API Key>   automatic (e.g. c8b89bc3-0139-11wa-8ef3-8c164563716b)
<Datastore Doc Regex> Only file names matching this regex will be fed to the datastore (Regex Tester - Golang)

Configure and run the feeder to index matching files in and below the current folder:

[email protected]:/yourfolder$ ./dsdocument-cli \
                              --id  <Datastore ID>      # From https://ui.dataspects.com/datastores/files \
                              --url https://datastore.dataspects.com \
                              --key <Datastore API Key> # From https://ui.dataspects.com/datastores/files
Subsequent pipeline
Tools

AAAA-Ingestion "From Repository to dataspectsd by"

Data fed?
Configuration

Configure the datastore:

<ID>        automatic
<Label>     Shown in information sources list on https://ui.dataspects.com/search
<ClientKey> automatic
<Regex>     Only file names matching this regex will be fed to the datastore (Regex Tester - Golang)

Configure the feeder:

[email protected]:/yourrepo$ ./dataspecter \
                              --id  <ID>      # From https://ui.dataspects.com/datastores/repositories/code\
                              --url https://datastore.dataspects.com \
                              --key <API Key> # From https://ui.dataspects.com/datastores/repositories/code
Subsequent pipeline
Tools


Applying an Indexer to resources in a Datastore[edit | edit source]