Difference between revisions of "Dataspects"

From FindAndLearn::Cookbook
Jump to: navigation, search
(Architecture)
(Index)
 
(11 intermediate revisions by the same user not shown)
Line 7: Line 7:
 
{{#mermaid:
 
{{#mermaid:
 
graph TD
 
graph TD
Dgraph["Dgraph"]
+
Neo4j["Neo4j"]
 
MWBE["SQL/ARC2 Store"]
 
MWBE["SQL/ARC2 Store"]
 
subgraph MediaWiki
 
subgraph MediaWiki
Line 24: Line 24:
 
ES["Elasticsearch"]
 
ES["Elasticsearch"]
 
CORE-->|CRUD|MWBE
 
CORE-->|CRUD|MWBE
VCF-->|CRUD|Dgraph
+
VCF-->|CRUD|Neo4j
 
DR-->|CRUD|DD
 
DR-->|CRUD|DD
DD-->|CRUD|Dgraph
+
DD-->|CRUD|Neo4j
 
DD-->|CRUD|ES
 
DD-->|CRUD|ES
 
DSUIX-.-DD
 
DSUIX-.-DD
 
}}
 
}}
 +
 +
== Default operation ==
 +
 +
=== Run system components ===
 +
 +
dataspectsSystems$ docker-compose up                      --> localhost:5601
 +
dataspectsd$      go run main.go --c config.yml --p 3001
 +
dataspects-ui$    nodemon                                --> localhost:3000
 +
~$                docker run ... neo4j                  --> localhost:7687
 +
 +
=== Configure a datastore ===
 +
 +
localhost:3000/datastores/mediawikis
 +
 +
You might want to empty an existing Elasticsearch index:
 +
 +
POST <INDEX_NAME>/_delete_by_query
 +
{"query": {"match_all": {}}}
 +
 +
You might want to empty an existing Neo4j database:
 +
 +
MATCH (n)
 +
DETACH DELETE n
 +
 +
==== SIGINT Config ====
 +
 +
http://localhost:3000/search/facet/general/ProvidesAction shows all available actions.
 +
 +
tasks:
 +
  - name: <TASK_NAME>
 +
    actions:
 +
      # See '''type Actions struct''' in a ResourceSilo's '''action.go''' about what actions are available.
 +
 +
Check dataspectsd's log for indexing details including tasks and actions.
 +
 +
=== Index ===
 +
 +
dataspecter$ go run main.go mw-ask \
 +
                --mw-url https://cookbook.findandlearn.net/w/api.php \
 +
                --mw-conditions Category:Entity \
 +
                --id 4 \
 +
                --url http://localhost:3001 \
 +
                --key 45b966ce-3545-4a1d-96de-98bb1935997e
 +
 +
dataspecter$ go run main.go mw-namespace \
 +
                --mw-namespace main \
 +
                --mw-url https://cookbook.findandlearn.net/w/api.php \
 +
                --id 4 \
 +
                --url http://localhost:3001 \
 +
                --key 45b966ce-3545-4a1d-96de-98bb1935997e
 +
 +
dataspecter$ go run main.go code-repo \
 +
                --id 3 \
 +
                --url http://localhost:3001 \
 +
                --key a7afa361-e13e-422f-97cc-3e975b3a9d4e
 +
 +
=== Test ===
 +
 +
https://blog.alexellis.io/golang-writing-unit-tests/
 +
 +
dataspectsd/test$ clear; go test -v -count=1

Latest revision as of 12:31, 16 January 2020

Disambiguation[edit source]

RubyGem "dataspects"

Entities keyworded Dataspects[edit source]

SystemComponent "Semantic MediaWiki"

Dataspects, Wiki


Dataspects Terms Mentioned In

Dataspects Domains:



Architecture[edit | edit source]

https://mermaidjs.github.io/mermaid-live-editor

Default operation[edit | edit source]

Run system components[edit | edit source]

dataspectsSystems$ docker-compose up                      --> localhost:5601
dataspectsd$       go run main.go --c config.yml --p 3001
dataspects-ui$     nodemon                                --> localhost:3000
~$                 docker run ... neo4j                   --> localhost:7687

Configure a datastore[edit | edit source]

localhost:3000/datastores/mediawikis

You might want to empty an existing Elasticsearch index:

POST <INDEX_NAME>/_delete_by_query
{"query": {"match_all": {}}}

You might want to empty an existing Neo4j database:

MATCH (n)
DETACH DELETE n

SIGINT Config[edit | edit source]

http://localhost:3000/search/facet/general/ProvidesAction shows all available actions.

tasks:
  - name: <TASK_NAME>
    actions:
      # See type Actions struct in a ResourceSilo's action.go about what actions are available.

Check dataspectsd's log for indexing details including tasks and actions.

Index[edit | edit source]

dataspecter$ go run main.go mw-ask \
               --mw-url https://cookbook.findandlearn.net/w/api.php \
               --mw-conditions Category:Entity \
               --id 4 \
               --url http://localhost:3001 \
               --key 45b966ce-3545-4a1d-96de-98bb1935997e
dataspecter$ go run main.go mw-namespace \
               --mw-namespace main \
               --mw-url https://cookbook.findandlearn.net/w/api.php \
               --id 4 \
               --url http://localhost:3001 \
               --key 45b966ce-3545-4a1d-96de-98bb1935997e
dataspecter$ go run main.go code-repo \
               --id 3 \
               --url http://localhost:3001 \
               --key a7afa361-e13e-422f-97cc-3e975b3a9d4e

Test[edit | edit source]

https://blog.alexellis.io/golang-writing-unit-tests/

dataspectsd/test$ clear; go test -v -count=1