From File system to Datastore by https://github.com/dataspects/dsdocument-cli

From FindAndLearn::Cookbook
C0992221051
Jump to: navigation, search

Warning: Display title "From File system to Datastore by https://github.com/dataspects/dsdocument-cli" overrides earlier display title "AAAA-Ingestion "From file system by dataspects/dsdocument-cli"".






Configuration[edit | edit source]

Configure the datastore:

<Datastore ID>        automatic (e.g. 12)
<Datastore Label>     Shown in information sources list on https://ui.dataspects.com/search
<Datastore API Key>   automatic (e.g. c8b89bc3-0139-11wa-8ef3-8c164563716b)
<Datastore Doc Regex> Only file names matching this regex will be fed to the datastore (Regex Tester - Golang)

Configure and run the feeder to index matching files in and below the current folder:

[email protected]:/yourfolder$ ./dsdocument-cli \
                              --id  <Datastore ID>      # From https://ui.dataspects.com/datastores/files \
                              --url https://datastore.dataspects.com \
                              --key <Datastore API Key> # From https://ui.dataspects.com/datastores/files

Subsequent pipeline[edit | edit source]