barak/cayley
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity

Rename Provenance -> Label

Browse source 
's/Provenance/Label/g' 's/provenance/label/g' with human vetting.
This commit is contained in:
kortschak 2014-07-29 08:44:39 +09:30
parent 410202f3e0
commit 274d9ef57e
20 changed files with 416 additions and 416 deletions
Download patch file Download diff file Expand all files Collapse all files

16
TODO.md
View file

@ -26,11 +26,11 @@ Usually something that should be taken care of.
### Bootstraps ### Bootstraps
Start discussing bootstrap triples, things that make the database self-describing, if they exist (though they need not). Talk about sameAs and indexing and type systems and whatnot. Start discussing bootstrap triples, things that make the database self-describing, if they exist (though they need not). Talk about sameAs and indexing and type systems and whatnot.
### Better surfacing of Provenance ### Better surfacing of Label
It exists, it's indexed, but it's basically useless right now It exists, it's indexed, but it's basically useless right now
### Optimize HasA Iterator ### Optimize HasA Iterator
There are some simple optimizations that can be done there. And was the first one to get right, this is the next one. There are some simple optimizations that can be done there. And was the first one to get right, this is the next one.
A simple example is just to convert the HasA to a fixed (next them out) if the subiterator size is guessable and small. A simple example is just to convert the HasA to a fixed (next them out) if the subiterator size is guessable and small.
### Gremlin features ### Gremlin features
@ -39,7 +39,7 @@ A simple example is just to convert the HasA to a fixed (next them out) if the s
A way to limit the number of subresults at a point, without even running the query. Essentially, much as GetLimit() does for the end, be able to do the same in between A way to limit the number of subresults at a point, without even running the query. Essentially, much as GetLimit() does for the end, be able to do the same in between
#### "Up" and "Down" traversals #### "Up" and "Down" traversals
Getting to the predicates from a node, or the nodes from a predicate, or some odd combinations thereof. Ditto for provenance. Getting to the predicates from a node, or the nodes from a predicate, or some odd combinations thereof. Ditto for label.
#### Value comparison #### Value comparison
Expose the value-comparison iterator in the language Expose the value-comparison iterator in the language
@ -66,7 +66,7 @@ The necessary component to make mid-query limit work. Acts as a limit on Next(),
Hopefully easy now that the AppEngine shim exists. Questionably fast. Hopefully easy now that the AppEngine shim exists. Questionably fast.
### Postgres Backend ### Postgres Backend
It'd be nice to run on SQL as well. It's a big why not? It'd be nice to run on SQL as well. It's a big why not?
#### Generalist layout #### Generalist layout
Notionally, this is a simple triple table with a number of indicies. Iterators and iterator optimization (ie, rewriting SQL queries) is the 'fun' part Notionally, this is a simple triple table with a number of indicies. Iterators and iterator optimization (ie, rewriting SQL queries) is the 'fun' part
#### "Short Schema" Layout? #### "Short Schema" Layout?
@ -75,7 +75,7 @@ The necessary component to make mid-query limit work. Acts as a limit on Next(),
### New Iterators ### New Iterators
#### Predicate Iterator #### Predicate Iterator
Really, this is just the generalized value comparison iterator, across strings and dates and such. Really, this is just the generalized value comparison iterator, across strings and dates and such.
## Longer Term (and fuzzy) ## Longer Term (and fuzzy)
@ -83,7 +83,7 @@ The necessary component to make mid-query limit work. Acts as a limit on Next(),
There's a whole body of work there, and a lot of interested researchers. They're the choir who already know the sermon of graph stores. Once ease-of-use gets people in the door, supporting extensions that make everyone happy seems like a win. And because we're query-language agnostic, it's a cleaner win. See also bootstrapping, which is the first goal toward this (eg, let's talk about sameAs, and index it appropriately.) There's a whole body of work there, and a lot of interested researchers. They're the choir who already know the sermon of graph stores. Once ease-of-use gets people in the door, supporting extensions that make everyone happy seems like a win. And because we're query-language agnostic, it's a cleaner win. See also bootstrapping, which is the first goal toward this (eg, let's talk about sameAs, and index it appropriately.)
### Replication ### Replication
Technically it works now if you piggyback on someone else's replication, but that's cheating. We speak HTTP, we can send triple sets over the wire to some other instance. Bonus points for a way to apply morphisms first -- massive graph on the backend, important graph on the frontend. Technically it works now if you piggyback on someone else's replication, but that's cheating. We speak HTTP, we can send triple sets over the wire to some other instance. Bonus points for a way to apply morphisms first -- massive graph on the backend, important graph on the frontend.
### Related services ### Related services
Eg, topic service, recon service -- whether in Cayley itself or as part of the greater project. Eg, topic service, recon service -- whether in Cayley itself or as part of the greater project.
@ -102,6 +102,6 @@ The necessary component to make mid-query limit work. Acts as a limit on Next(),
### All sorts of backends: ### All sorts of backends:
#### Git? #### Git?
Can we access git in a meaningful fashion, giving a history and rollbacks to memory/flat files? Can we access git in a meaningful fashion, giving a history and rollbacks to memory/flat files?
#### ElasticSearch #### ElasticSearch
#### Cassandra #### Cassandra
#### Redis #### Redis

4
docs/HTTP.md
View file

@ -93,7 +93,7 @@ POST Body: JSON triples
"subject": "Subject Node", "subject": "Subject Node",
"predicate": "Predicate Node", "predicate": "Predicate Node",
"object": "Object node", "object": "Object node",
"provenance": "Provenance node" // Optional "label": "Label node" // Optional
}] // More than one triple allowed. }] // More than one triple allowed.
``` ```
@ -121,7 +121,7 @@ POST Body: JSON triples
"subject": "Subject Node", "subject": "Subject Node",
"predicate": "Predicate Node", "predicate": "Predicate Node",
"object": "Object node", "object": "Object node",
"provenance": "Provenance node" // Optional "label": "Label node" // Optional
}] // More than one triple allowed. }] // More than one triple allowed.
``` ```

2
graph/iterator/value_comparison_iterator.go
View file

@ -17,7 +17,7 @@ package iterator
// "Value Comparison" is a unary operator -- a filter across the values in the // "Value Comparison" is a unary operator -- a filter across the values in the
// relevant subiterator. // relevant subiterator.
// //
// This is hugely useful for things like provenance, but value ranges in general // This is hugely useful for things like label, but value ranges in general
// come up from time to time. At *worst* we're as big as our underlying iterator. // come up from time to time. At *worst* we're as big as our underlying iterator.
// At best, we're the null iterator. // At best, we're the null iterator.
// //

8
graph/leveldb/iterator.go
View file

@ -124,7 +124,7 @@ func PositionOf(prefix []byte, d quad.Direction, qs *TripleStore) int {
return qs.hasher.Size() + 2 return qs.hasher.Size() + 2
case quad.Object: case quad.Object:
return 2*qs.hasher.Size() + 2 return 2*qs.hasher.Size() + 2
case quad.Provenance: case quad.Label:
return -1 return -1
} }
} }
@ -136,7 +136,7 @@ func PositionOf(prefix []byte, d quad.Direction, qs *TripleStore) int {
return 2 return 2
case quad.Object: case quad.Object:
return qs.hasher.Size() + 2 return qs.hasher.Size() + 2
case quad.Provenance: case quad.Label:
return -1 return -1
} }
} }
@ -148,7 +148,7 @@ func PositionOf(prefix []byte, d quad.Direction, qs *TripleStore) int {
return 2*qs.hasher.Size() + 2 return 2*qs.hasher.Size() + 2
case quad.Object: case quad.Object:
return 2 return 2
case quad.Provenance: case quad.Label:
return -1 return -1
} }
} }
@ -160,7 +160,7 @@ func PositionOf(prefix []byte, d quad.Direction, qs *TripleStore) int {
return qs.hasher.Size() + 2 return qs.hasher.Size() + 2
case quad.Object: case quad.Object:
return 3*qs.hasher.Size() + 2 return 3*qs.hasher.Size() + 2
case quad.Provenance: case quad.Label:
return 2 return 2
} }
} }

6
graph/leveldb/leveldb_test.go
View file

@ -73,7 +73,7 @@ func (o ordered) Less(i, j int) bool {
o[i].Subject == o[j].Subject && o[i].Subject == o[j].Subject &&
o[i].Predicate == o[j].Predicate && o[i].Predicate == o[j].Predicate &&
o[i].Object == o[j].Object && o[i].Object == o[j].Object &&
o[i].Provenance < o[j].Provenance: o[i].Label < o[j].Label:
return true return true
@ -359,8 +359,8 @@ func TestSetIterator(t *testing.T) {
t.Errorf("Failed to get expected results from predicate iterator, got:%v expect:%v", got, expect) t.Errorf("Failed to get expected results from predicate iterator, got:%v expect:%v", got, expect)
} }
// Provenance iterator. // Label iterator.
it = qs.TripleIterator(quad.Provenance, qs.ValueOf("status_graph")) it = qs.TripleIterator(quad.Label, qs.ValueOf("status_graph"))
expect = []*quad.Quad{ expect = []*quad.Quad{
{"B", "status", "cool", "status_graph"}, {"B", "status", "cool", "status_graph"},

20
graph/leveldb/triplestore.go
View file

@ -125,8 +125,8 @@ func (qs *TripleStore) createKeyFor(d [3]quad.Direction, triple *quad.Quad) []by
func (qs *TripleStore) createProvKeyFor(d [3]quad.Direction, triple *quad.Quad) []byte { func (qs *TripleStore) createProvKeyFor(d [3]quad.Direction, triple *quad.Quad) []byte {
key := make([]byte, 0, 2+(qs.hasher.Size()*4)) key := make([]byte, 0, 2+(qs.hasher.Size()*4))
// TODO(kortschak) Remove dependence on String() method. // TODO(kortschak) Remove dependence on String() method.
key = append(key, []byte{quad.Provenance.Prefix(), d[0].Prefix()}...) key = append(key, []byte{quad.Label.Prefix(), d[0].Prefix()}...)
key = append(key, qs.convertStringToByteHash(triple.Get(quad.Provenance))...) key = append(key, qs.convertStringToByteHash(triple.Get(quad.Label))...)
key = append(key, qs.convertStringToByteHash(triple.Get(d[0]))...) key = append(key, qs.convertStringToByteHash(triple.Get(d[0]))...)
key = append(key, qs.convertStringToByteHash(triple.Get(d[1]))...) key = append(key, qs.convertStringToByteHash(triple.Get(d[1]))...)
key = append(key, qs.convertStringToByteHash(triple.Get(d[2]))...) key = append(key, qs.convertStringToByteHash(triple.Get(d[2]))...)
@ -176,9 +176,9 @@ func (qs *TripleStore) RemoveTriple(t *quad.Quad) {
qs.UpdateValueKeyBy(t.Get(quad.Subject), -1, batch) qs.UpdateValueKeyBy(t.Get(quad.Subject), -1, batch)
qs.UpdateValueKeyBy(t.Get(quad.Predicate), -1, batch) qs.UpdateValueKeyBy(t.Get(quad.Predicate), -1, batch)
qs.UpdateValueKeyBy(t.Get(quad.Object), -1, batch) qs.UpdateValueKeyBy(t.Get(quad.Object), -1, batch)
if t.Get(quad.Provenance) != "" { if t.Get(quad.Label) != "" {
batch.Delete(qs.createProvKeyFor(pso, t)) batch.Delete(qs.createProvKeyFor(pso, t))
qs.UpdateValueKeyBy(t.Get(quad.Provenance), -1, batch) qs.UpdateValueKeyBy(t.Get(quad.Label), -1, batch)
} }
err = qs.db.Write(batch, nil) err = qs.db.Write(batch, nil)
if err != nil { if err != nil {
@ -197,7 +197,7 @@ func (qs *TripleStore) buildTripleWrite(batch *leveldb.Batch, t *quad.Quad) {
batch.Put(qs.createKeyFor(spo, t), bytes) batch.Put(qs.createKeyFor(spo, t), bytes)
batch.Put(qs.createKeyFor(osp, t), bytes) batch.Put(qs.createKeyFor(osp, t), bytes)
batch.Put(qs.createKeyFor(pos, t), bytes) batch.Put(qs.createKeyFor(pos, t), bytes)
if t.Get(quad.Provenance) != "" { if t.Get(quad.Label) != "" {
batch.Put(qs.createProvKeyFor(pso, t), bytes) batch.Put(qs.createProvKeyFor(pso, t), bytes)
} }
} }
@ -207,8 +207,8 @@ func (qs *TripleStore) buildWrite(batch *leveldb.Batch, t *quad.Quad) {
qs.UpdateValueKeyBy(t.Get(quad.Subject), 1, nil) qs.UpdateValueKeyBy(t.Get(quad.Subject), 1, nil)
qs.UpdateValueKeyBy(t.Get(quad.Predicate), 1, nil) qs.UpdateValueKeyBy(t.Get(quad.Predicate), 1, nil)
qs.UpdateValueKeyBy(t.Get(quad.Object), 1, nil) qs.UpdateValueKeyBy(t.Get(quad.Object), 1, nil)
if t.Get(quad.Provenance) != "" { if t.Get(quad.Label) != "" {
qs.UpdateValueKeyBy(t.Get(quad.Provenance), 1, nil) qs.UpdateValueKeyBy(t.Get(quad.Label), 1, nil)
} }
} }
@ -272,8 +272,8 @@ func (qs *TripleStore) AddTripleSet(t_s []*quad.Quad) {
resizeMap[t.Subject]++ resizeMap[t.Subject]++
resizeMap[t.Predicate]++ resizeMap[t.Predicate]++
resizeMap[t.Object]++ resizeMap[t.Object]++
if t.Provenance != "" { if t.Label != "" {
resizeMap[t.Provenance]++ resizeMap[t.Label]++
} }
} }
for k, v := range resizeMap { for k, v := range resizeMap {
@ -411,7 +411,7 @@ func (qs *TripleStore) TripleIterator(d quad.Direction, val graph.Value) graph.I
prefix = "po" prefix = "po"
case quad.Object: case quad.Object:
prefix = "os" prefix = "os"
case quad.Provenance: case quad.Label:
prefix = "cp" prefix = "cp"
default: default:
panic("unreachable " + d.String()) panic("unreachable " + d.String())

38
graph/memstore/triplestore.go
View file

@ -26,10 +26,10 @@ import (
) )
type TripleDirectionIndex struct { type TripleDirectionIndex struct {
subject map[int64]*llrb.LLRB subject map[int64]*llrb.LLRB
predicate map[int64]*llrb.LLRB predicate map[int64]*llrb.LLRB
object map[int64]*llrb.LLRB object map[int64]*llrb.LLRB
provenance map[int64]*llrb.LLRB label map[int64]*llrb.LLRB
} }
func NewTripleDirectionIndex() *TripleDirectionIndex { func NewTripleDirectionIndex() *TripleDirectionIndex {
@ -37,7 +37,7 @@ func NewTripleDirectionIndex() *TripleDirectionIndex {
tdi.subject = make(map[int64]*llrb.LLRB) tdi.subject = make(map[int64]*llrb.LLRB)
tdi.predicate = make(map[int64]*llrb.LLRB) tdi.predicate = make(map[int64]*llrb.LLRB)
tdi.object = make(map[int64]*llrb.LLRB) tdi.object = make(map[int64]*llrb.LLRB)
tdi.provenance = make(map[int64]*llrb.LLRB) tdi.label = make(map[int64]*llrb.LLRB)
return &tdi return &tdi
} }
@ -49,8 +49,8 @@ func (tdi *TripleDirectionIndex) GetForDir(d quad.Direction) map[int64]*llrb.LLR
return tdi.object return tdi.object
case quad.Predicate: case quad.Predicate:
return tdi.predicate return tdi.predicate
case quad.Provenance: case quad.Label:
return tdi.provenance return tdi.label
} }
panic("illegal direction") panic("illegal direction")
} }
@ -104,9 +104,9 @@ func (ts *TripleStore) AddTripleSet(triples []*quad.Quad) {
func (ts *TripleStore) tripleExists(t *quad.Quad) (bool, int64) { func (ts *TripleStore) tripleExists(t *quad.Quad) (bool, int64) {
smallest := -1 smallest := -1
var smallest_tree *llrb.LLRB var smallest_tree *llrb.LLRB
for d := quad.Subject; d <= quad.Provenance; d++ { for d := quad.Subject; d <= quad.Label; d++ {
sid := t.Get(d) sid := t.Get(d)
if d == quad.Provenance && sid == "" { if d == quad.Label && sid == "" {
continue continue
} }
id, ok := ts.idMap[sid] id, ok := ts.idMap[sid]
@ -148,9 +148,9 @@ func (ts *TripleStore) AddTriple(t *quad.Quad) {
ts.size++ ts.size++
ts.tripleIdCounter++ ts.tripleIdCounter++
for d := quad.Subject; d <= quad.Provenance; d++ { for d := quad.Subject; d <= quad.Label; d++ {
sid := t.Get(d) sid := t.Get(d)
if d == quad.Provenance && sid == "" { if d == quad.Label && sid == "" {
continue continue
} }
if _, ok := ts.idMap[sid]; !ok { if _, ok := ts.idMap[sid]; !ok {
@ -160,8 +160,8 @@ func (ts *TripleStore) AddTriple(t *quad.Quad) {
} }
} }
for d := quad.Subject; d <= quad.Provenance; d++ { for d := quad.Subject; d <= quad.Label; d++ {
if d == quad.Provenance && t.Get(d) == "" { if d == quad.Label && t.Get(d) == "" {
continue continue
} }
id := ts.idMap[t.Get(d)] id := ts.idMap[t.Get(d)]
@ -183,8 +183,8 @@ func (ts *TripleStore) RemoveTriple(t *quad.Quad) {
ts.triples[tripleID] = quad.Quad{} ts.triples[tripleID] = quad.Quad{}
ts.size-- ts.size--
for d := quad.Subject; d <= quad.Provenance; d++ { for d := quad.Subject; d <= quad.Label; d++ {
if d == quad.Provenance && t.Get(d) == "" { if d == quad.Label && t.Get(d) == "" {
continue continue
} }
id := ts.idMap[t.Get(d)] id := ts.idMap[t.Get(d)]
@ -192,8 +192,8 @@ func (ts *TripleStore) RemoveTriple(t *quad.Quad) {
tree.Delete(Int64(tripleID)) tree.Delete(Int64(tripleID))
} }
for d := quad.Subject; d <= quad.Provenance; d++ { for d := quad.Subject; d <= quad.Label; d++ {
if d == quad.Provenance && t.Get(d) == "" { if d == quad.Label && t.Get(d) == "" {
continue continue
} }
id, ok := ts.idMap[t.Get(d)] id, ok := ts.idMap[t.Get(d)]
@ -201,8 +201,8 @@ func (ts *TripleStore) RemoveTriple(t *quad.Quad) {
continue continue
} }
stillExists := false stillExists := false
for d := quad.Subject; d <= quad.Provenance; d++ { for d := quad.Subject; d <= quad.Label; d++ {
if d == quad.Provenance && t.Get(d) == "" { if d == quad.Label && t.Get(d) == "" {
continue continue
} }
nodeTree := ts.index.GetOrCreate(d, id) nodeTree := ts.index.GetOrCreate(d, id)

2
graph/memstore/triplestore_test.go
View file

@ -58,7 +58,7 @@ func makeTestStore(data []*quad.Quad) (*TripleStore, []pair) {
ind []pair ind []pair
) )
for _, t := range data { for _, t := range data {
for _, qp := range []string{t.Subject, t.Predicate, t.Object, t.Provenance} { for _, qp := range []string{t.Subject, t.Predicate, t.Object, t.Label} {
if _, ok := seen[qp]; !ok && qp != "" { if _, ok := seen[qp]; !ok && qp != "" {
val++ val++
ind = append(ind, pair{qp, val}) ind = append(ind, pair{qp, val})

6
graph/mongo/iterator.go
View file

@ -53,8 +53,8 @@ func NewIterator(qs *TripleStore, collection string, d quad.Direction, val graph
m.constraint = bson.M{"Predicate": m.name} m.constraint = bson.M{"Predicate": m.name}
case quad.Object: case quad.Object:
m.constraint = bson.M{"Object": m.name} m.constraint = bson.M{"Object": m.name}
case quad.Provenance: case quad.Label:
m.constraint = bson.M{"Provenance": m.name} m.constraint = bson.M{"Label": m.name}
} }
m.qs = qs m.qs = qs
@ -143,7 +143,7 @@ func (it *Iterator) Check(v graph.Value) bool {
offset = (it.qs.hasher.Size() * 2) offset = (it.qs.hasher.Size() * 2)
case quad.Object: case quad.Object:
offset = (it.qs.hasher.Size() * 2) * 2 offset = (it.qs.hasher.Size() * 2) * 2
case quad.Provenance: case quad.Label:
offset = (it.qs.hasher.Size() * 2) * 3 offset = (it.qs.hasher.Size() * 2) * 3
} }
val := v.(string)[offset : it.qs.hasher.Size()*2+offset] val := v.(string)[offset : it.qs.hasher.Size()*2+offset]

34
graph/mongo/triplestore.go
View file

@ -69,7 +69,7 @@ func createNewMongoGraph(addr string, options graph.Options) error {
db.C("triples").EnsureIndex(indexOpts) db.C("triples").EnsureIndex(indexOpts)
indexOpts.Key = []string{"Obj"} indexOpts.Key = []string{"Obj"}
db.C("triples").EnsureIndex(indexOpts) db.C("triples").EnsureIndex(indexOpts)
indexOpts.Key = []string{"Provenance"} indexOpts.Key = []string{"Label"}
db.C("triples").EnsureIndex(indexOpts) db.C("triples").EnsureIndex(indexOpts)
return nil return nil
} }
@ -96,7 +96,7 @@ func (qs *TripleStore) getIdForTriple(t *quad.Quad) string {
id := qs.ConvertStringToByteHash(t.Subject) id := qs.ConvertStringToByteHash(t.Subject)
id += qs.ConvertStringToByteHash(t.Predicate) id += qs.ConvertStringToByteHash(t.Predicate)
id += qs.ConvertStringToByteHash(t.Object) id += qs.ConvertStringToByteHash(t.Object)
id += qs.ConvertStringToByteHash(t.Provenance) id += qs.ConvertStringToByteHash(t.Label)
return id return id
} }
@ -153,11 +153,11 @@ func (qs *TripleStore) updateNodeBy(node_name string, inc int) {
func (qs *TripleStore) writeTriple(t *quad.Quad) bool { func (qs *TripleStore) writeTriple(t *quad.Quad) bool {
tripledoc := bson.M{ tripledoc := bson.M{
"_id": qs.getIdForTriple(t), "_id": qs.getIdForTriple(t),
"Subject": t.Subject, "Subject": t.Subject,
"Predicate": t.Predicate, "Predicate": t.Predicate,
"Object": t.Object, "Object": t.Object,
"Provenance": t.Provenance, "Label": t.Label,
} }
err := qs.db.C("triples").Insert(tripledoc) err := qs.db.C("triples").Insert(tripledoc)
if err != nil { if err != nil {
@ -176,8 +176,8 @@ func (qs *TripleStore) AddTriple(t *quad.Quad) {
qs.updateNodeBy(t.Subject, 1) qs.updateNodeBy(t.Subject, 1)
qs.updateNodeBy(t.Predicate, 1) qs.updateNodeBy(t.Predicate, 1)
qs.updateNodeBy(t.Object, 1) qs.updateNodeBy(t.Object, 1)
if t.Provenance != "" { if t.Label != "" {
qs.updateNodeBy(t.Provenance, 1) qs.updateNodeBy(t.Label, 1)
} }
} }
@ -190,8 +190,8 @@ func (qs *TripleStore) AddTripleSet(in []*quad.Quad) {
ids[t.Subject]++ ids[t.Subject]++
ids[t.Object]++ ids[t.Object]++
ids[t.Predicate]++ ids[t.Predicate]++
if t.Provenance != "" { if t.Label != "" {
ids[t.Provenance]++ ids[t.Label]++
} }
} }
} }
@ -212,8 +212,8 @@ func (qs *TripleStore) RemoveTriple(t *quad.Quad) {
qs.updateNodeBy(t.Subject, -1) qs.updateNodeBy(t.Subject, -1)
qs.updateNodeBy(t.Predicate, -1) qs.updateNodeBy(t.Predicate, -1)
qs.updateNodeBy(t.Object, -1) qs.updateNodeBy(t.Object, -1)
if t.Provenance != "" { if t.Label != "" {
qs.updateNodeBy(t.Provenance, -1) qs.updateNodeBy(t.Label, -1)
} }
} }
@ -227,7 +227,7 @@ func (qs *TripleStore) Quad(val graph.Value) *quad.Quad {
bsonDoc["Subject"].(string), bsonDoc["Subject"].(string),
bsonDoc["Predicate"].(string), bsonDoc["Predicate"].(string),
bsonDoc["Object"].(string), bsonDoc["Object"].(string),
bsonDoc["Provenance"].(string), bsonDoc["Label"].(string),
} }
} }
@ -292,7 +292,7 @@ func (qs *TripleStore) TripleDirection(in graph.Value, d quad.Direction) graph.V
offset = (qs.hasher.Size() * 2) offset = (qs.hasher.Size() * 2)
case quad.Object: case quad.Object:
offset = (qs.hasher.Size() * 2) * 2 offset = (qs.hasher.Size() * 2) * 2
case quad.Provenance: case quad.Label:
offset = (qs.hasher.Size() * 2) * 3 offset = (qs.hasher.Size() * 2) * 3
} }
val := in.(string)[offset : qs.hasher.Size()*2+offset] val := in.(string)[offset : qs.hasher.Size()*2+offset]
@ -328,8 +328,8 @@ func (qs *TripleStore) BulkLoad(dec quad.Unmarshaler) error {
emit(s_key, {"_id": s_key, "Name" : this.Subject, "Size" : 1}) emit(s_key, {"_id": s_key, "Name" : this.Subject, "Size" : 1})
emit(p_key, {"_id": p_key, "Name" : this.Predicate, "Size" : 1}) emit(p_key, {"_id": p_key, "Name" : this.Predicate, "Size" : 1})
emit(o_key, {"_id": o_key, "Name" : this.Object, "Size" : 1}) emit(o_key, {"_id": o_key, "Name" : this.Object, "Size" : 1})
if (this.Provenance != "") { if (this.Label != "") {
emit(c_key, {"_id": c_key, "Name" : this.Provenance, "Size" : 1}) emit(c_key, {"_id": c_key, "Name" : this.Label, "Size" : 1})
} }
} }
`, `,

2
http/http_test.go
View file

@ -32,7 +32,7 @@ var parseTests = []struct {
message: "parse correct JSON", message: "parse correct JSON",
input: `[ input: `[
{"subject": "foo", "predicate": "bar", "object": "baz"}, {"subject": "foo", "predicate": "bar", "object": "baz"},
{"subject": "foo", "predicate": "bar", "object": "baz", "provenance": "graph"} {"subject": "foo", "predicate": "bar", "object": "baz", "label": "graph"}
]`, ]`,
expect: []*quad.Quad{ expect: []*quad.Quad{
{"foo", "bar", "baz", ""}, {"foo", "bar", "baz", ""},

2
quad/cquads/actions.rl
View file

@ -70,7 +70,7 @@
if label < 0 { if label < 0 {
panic("unexpected parser state: label start not set") panic("unexpected parser state: label start not set")
} }
q.Provenance = unEscape(data[label:p], isQuoted, isEscaped) q.Label = unEscape(data[label:p], isQuoted, isEscaped)
isEscaped = false isEscaped = false
isQuoted = false isQuoted = false
} }

360
quad/cquads/cquads_test.go
View file

File diff suppressed because it is too large Load diff

28
quad/cquads/parse.go
View file

@ -1006,7 +1006,7 @@ tr37:
if label < 0 { if label < 0 {
panic("unexpected parser state: label start not set") panic("unexpected parser state: label start not set")
} }
q.Provenance = unEscape(data[label:p], isQuoted, isEscaped) q.Label = unEscape(data[label:p], isQuoted, isEscaped)
isEscaped = false isEscaped = false
isQuoted = false isQuoted = false
@ -1023,7 +1023,7 @@ tr47:
if label < 0 { if label < 0 {
panic("unexpected parser state: label start not set") panic("unexpected parser state: label start not set")
} }
q.Provenance = unEscape(data[label:p], isQuoted, isEscaped) q.Label = unEscape(data[label:p], isQuoted, isEscaped)
isEscaped = false isEscaped = false
isQuoted = false isQuoted = false
@ -1040,7 +1040,7 @@ tr60:
if label < 0 { if label < 0 {
panic("unexpected parser state: label start not set") panic("unexpected parser state: label start not set")
} }
q.Provenance = unEscape(data[label:p], isQuoted, isEscaped) q.Label = unEscape(data[label:p], isQuoted, isEscaped)
isEscaped = false isEscaped = false
isQuoted = false isQuoted = false
@ -1079,7 +1079,7 @@ tr66:
if label < 0 { if label < 0 {
panic("unexpected parser state: label start not set") panic("unexpected parser state: label start not set")
} }
q.Provenance = unEscape(data[label:p], isQuoted, isEscaped) q.Label = unEscape(data[label:p], isQuoted, isEscaped)
isEscaped = false isEscaped = false
isQuoted = false isQuoted = false
@ -1096,7 +1096,7 @@ tr61:
if label < 0 { if label < 0 {
panic("unexpected parser state: label start not set") panic("unexpected parser state: label start not set")
} }
q.Provenance = unEscape(data[label:p], isQuoted, isEscaped) q.Label = unEscape(data[label:p], isQuoted, isEscaped)
isEscaped = false isEscaped = false
isQuoted = false isQuoted = false
@ -1160,7 +1160,7 @@ tr39:
if label < 0 { if label < 0 {
panic("unexpected parser state: label start not set") panic("unexpected parser state: label start not set")
} }
q.Provenance = unEscape(data[label:p], isQuoted, isEscaped) q.Label = unEscape(data[label:p], isQuoted, isEscaped)
isEscaped = false isEscaped = false
isQuoted = false isQuoted = false
@ -1177,7 +1177,7 @@ tr49:
if label < 0 { if label < 0 {
panic("unexpected parser state: label start not set") panic("unexpected parser state: label start not set")
} }
q.Provenance = unEscape(data[label:p], isQuoted, isEscaped) q.Label = unEscape(data[label:p], isQuoted, isEscaped)
isEscaped = false isEscaped = false
isQuoted = false isQuoted = false
@ -1265,7 +1265,7 @@ tr253:
if label < 0 { if label < 0 {
panic("unexpected parser state: label start not set") panic("unexpected parser state: label start not set")
} }
q.Provenance = unEscape(data[label:p], isQuoted, isEscaped) q.Label = unEscape(data[label:p], isQuoted, isEscaped)
isEscaped = false isEscaped = false
isQuoted = false isQuoted = false
@ -1282,7 +1282,7 @@ tr263:
if label < 0 { if label < 0 {
panic("unexpected parser state: label start not set") panic("unexpected parser state: label start not set")
} }
q.Provenance = unEscape(data[label:p], isQuoted, isEscaped) q.Label = unEscape(data[label:p], isQuoted, isEscaped)
isEscaped = false isEscaped = false
isQuoted = false isQuoted = false
@ -1299,7 +1299,7 @@ tr287:
if label < 0 { if label < 0 {
panic("unexpected parser state: label start not set") panic("unexpected parser state: label start not set")
} }
q.Provenance = unEscape(data[label:p], isQuoted, isEscaped) q.Label = unEscape(data[label:p], isQuoted, isEscaped)
isEscaped = false isEscaped = false
isQuoted = false isQuoted = false
@ -1326,7 +1326,7 @@ tr293:
if label < 0 { if label < 0 {
panic("unexpected parser state: label start not set") panic("unexpected parser state: label start not set")
} }
q.Provenance = unEscape(data[label:p], isQuoted, isEscaped) q.Label = unEscape(data[label:p], isQuoted, isEscaped)
isEscaped = false isEscaped = false
isQuoted = false isQuoted = false
@ -1343,7 +1343,7 @@ tr288:
if label < 0 { if label < 0 {
panic("unexpected parser state: label start not set") panic("unexpected parser state: label start not set")
} }
q.Provenance = unEscape(data[label:p], isQuoted, isEscaped) q.Label = unEscape(data[label:p], isQuoted, isEscaped)
isEscaped = false isEscaped = false
isQuoted = false isQuoted = false
@ -1377,7 +1377,7 @@ tr255:
if label < 0 { if label < 0 {
panic("unexpected parser state: label start not set") panic("unexpected parser state: label start not set")
} }
q.Provenance = unEscape(data[label:p], isQuoted, isEscaped) q.Label = unEscape(data[label:p], isQuoted, isEscaped)
isEscaped = false isEscaped = false
isQuoted = false isQuoted = false
@ -1394,7 +1394,7 @@ tr265:
if label < 0 { if label < 0 {
panic("unexpected parser state: label start not set") panic("unexpected parser state: label start not set")
} }
q.Provenance = unEscape(data[label:p], isQuoted, isEscaped) q.Label = unEscape(data[label:p], isQuoted, isEscaped)
isEscaped = false isEscaped = false
isQuoted = false isQuoted = false

2
quad/nquads/actions.rl
View file

@ -63,7 +63,7 @@
if label < 0 { if label < 0 {
panic("unexpected parser state: label start not set") panic("unexpected parser state: label start not set")
} }
q.Provenance = unEscape(data[label:p], isEscaped) q.Label = unEscape(data[label:p], isEscaped)
isEscaped = false isEscaped = false
} }

256
quad/nquads/nquads_test.go
View file

@ -41,10 +41,10 @@ var testNTriples = []struct {
message: "parse triple with commment", message: "parse triple with commment",
input: `_:100000 </film/performance/actor> </en/larry_fine_1902> . # example from 30movies`, input: `_:100000 </film/performance/actor> </en/larry_fine_1902> . # example from 30movies`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "_:100000", Subject: "_:100000",
Predicate: "</film/performance/actor>", Predicate: "</film/performance/actor>",
Object: "</en/larry_fine_1902>", Object: "</en/larry_fine_1902>",
Provenance: "", Label: "",
}, },
err: nil, err: nil,
}, },
@ -53,10 +53,10 @@ var testNTriples = []struct {
message: "parse triple with commment", message: "parse triple with commment",
input: `_:10011 </film/performance/character> "Tomás de Torquemada" . # example from 30movies with unicode`, input: `_:10011 </film/performance/character> "Tomás de Torquemada" . # example from 30movies with unicode`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "_:10011", Subject: "_:10011",
Predicate: "</film/performance/character>", Predicate: "</film/performance/character>",
Object: `"Tomás de Torquemada"`, Object: `"Tomás de Torquemada"`,
Provenance: "", Label: "",
}, },
err: nil, err: nil,
}, },
@ -66,10 +66,10 @@ var testNTriples = []struct {
message: "parse triple with commment", message: "parse triple with commment",
input: `<http://one.example/subject1> <http://one.example/predicate1> <http://one.example/object1> . # comments here`, input: `<http://one.example/subject1> <http://one.example/predicate1> <http://one.example/object1> . # comments here`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "<http://one.example/subject1>", Subject: "<http://one.example/subject1>",
Predicate: "<http://one.example/predicate1>", Predicate: "<http://one.example/predicate1>",
Object: "<http://one.example/object1>", Object: "<http://one.example/object1>",
Provenance: "", Label: "",
}, },
err: nil, err: nil,
}, },
@ -77,10 +77,10 @@ var testNTriples = []struct {
message: "parse triple with blank subject node, literal object and no comment (1)", message: "parse triple with blank subject node, literal object and no comment (1)",
input: `_:subject1 <http://an.example/predicate1> "object1" .`, input: `_:subject1 <http://an.example/predicate1> "object1" .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "_:subject1", Subject: "_:subject1",
Predicate: "<http://an.example/predicate1>", Predicate: "<http://an.example/predicate1>",
Object: `"object1"`, Object: `"object1"`,
Provenance: "", Label: "",
}, },
err: nil, err: nil,
}, },
@ -88,10 +88,10 @@ var testNTriples = []struct {
message: "parse triple with blank subject node, literal object and no comment (2)", message: "parse triple with blank subject node, literal object and no comment (2)",
input: `_:subject2 <http://an.example/predicate2> "object2" .`, input: `_:subject2 <http://an.example/predicate2> "object2" .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "_:subject2", Subject: "_:subject2",
Predicate: "<http://an.example/predicate2>", Predicate: "<http://an.example/predicate2>",
Object: `"object2"`, Object: `"object2"`,
Provenance: "", Label: "",
}, },
err: nil, err: nil,
}, },
@ -101,10 +101,10 @@ var testNTriples = []struct {
message: "parse triple with three IRIREFs", message: "parse triple with three IRIREFs",
input: `<http://example.org/#spiderman> <http://www.perceive.net/schemas/relationship/enemyOf> <http://example.org/#green-goblin> .`, input: `<http://example.org/#spiderman> <http://www.perceive.net/schemas/relationship/enemyOf> <http://example.org/#green-goblin> .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "<http://example.org/#spiderman>", Subject: "<http://example.org/#spiderman>",
Predicate: "<http://www.perceive.net/schemas/relationship/enemyOf>", Predicate: "<http://www.perceive.net/schemas/relationship/enemyOf>",
Object: "<http://example.org/#green-goblin>", Object: "<http://example.org/#green-goblin>",
Provenance: "", Label: "",
}, },
err: nil, err: nil,
}, },
@ -114,10 +114,10 @@ var testNTriples = []struct {
message: "parse triple with blank node labelled subject and object and IRIREF predicate (1)", message: "parse triple with blank node labelled subject and object and IRIREF predicate (1)",
input: `_:alice <http://xmlns.com/foaf/0.1/knows> _:bob .`, input: `_:alice <http://xmlns.com/foaf/0.1/knows> _:bob .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "_:alice", Subject: "_:alice",
Predicate: "<http://xmlns.com/foaf/0.1/knows>", Predicate: "<http://xmlns.com/foaf/0.1/knows>",
Object: "_:bob", Object: "_:bob",
Provenance: "", Label: "",
}, },
err: nil, err: nil,
}, },
@ -125,10 +125,10 @@ var testNTriples = []struct {
message: "parse triple with blank node labelled subject and object and IRIREF predicate (2)", message: "parse triple with blank node labelled subject and object and IRIREF predicate (2)",
input: `_:bob <http://xmlns.com/foaf/0.1/knows> _:alice .`, input: `_:bob <http://xmlns.com/foaf/0.1/knows> _:alice .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "_:bob", Subject: "_:bob",
Predicate: "<http://xmlns.com/foaf/0.1/knows>", Predicate: "<http://xmlns.com/foaf/0.1/knows>",
Object: "_:alice", Object: "_:alice",
Provenance: "", Label: "",
}, },
err: nil, err: nil,
}, },
@ -138,10 +138,10 @@ var testNTriples = []struct {
message: "parse quad with commment", message: "parse quad with commment",
input: `<http://one.example/subject1> <http://one.example/predicate1> <http://one.example/object1> <http://example.org/graph3> . # comments here`, input: `<http://one.example/subject1> <http://one.example/predicate1> <http://one.example/object1> <http://example.org/graph3> . # comments here`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "<http://one.example/subject1>", Subject: "<http://one.example/subject1>",
Predicate: "<http://one.example/predicate1>", Predicate: "<http://one.example/predicate1>",
Object: "<http://one.example/object1>", Object: "<http://one.example/object1>",
Provenance: "<http://example.org/graph3>", Label: "<http://example.org/graph3>",
}, },
err: nil, err: nil,
}, },
@ -149,10 +149,10 @@ var testNTriples = []struct {
message: "parse quad with blank subject node, literal object, IRIREF predicate and label, and no comment (1)", message: "parse quad with blank subject node, literal object, IRIREF predicate and label, and no comment (1)",
input: `_:subject1 <http://an.example/predicate1> "object1" <http://example.org/graph1> .`, input: `_:subject1 <http://an.example/predicate1> "object1" <http://example.org/graph1> .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "_:subject1", Subject: "_:subject1",
Predicate: "<http://an.example/predicate1>", Predicate: "<http://an.example/predicate1>",
Object: `"object1"`, Object: `"object1"`,
Provenance: "<http://example.org/graph1>", Label: "<http://example.org/graph1>",
}, },
err: nil, err: nil,
}, },
@ -160,10 +160,10 @@ var testNTriples = []struct {
message: "parse quad with blank subject node, literal object, IRIREF predicate and label, and no comment (2)", message: "parse quad with blank subject node, literal object, IRIREF predicate and label, and no comment (2)",
input: `_:subject2 <http://an.example/predicate2> "object2" <http://example.org/graph5> .`, input: `_:subject2 <http://an.example/predicate2> "object2" <http://example.org/graph5> .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "_:subject2", Subject: "_:subject2",
Predicate: "<http://an.example/predicate2>", Predicate: "<http://an.example/predicate2>",
Object: `"object2"`, Object: `"object2"`,
Provenance: "<http://example.org/graph5>", Label: "<http://example.org/graph5>",
}, },
err: nil, err: nil,
}, },
@ -173,10 +173,10 @@ var testNTriples = []struct {
message: "parse quad with all IRIREF parts", message: "parse quad with all IRIREF parts",
input: `<http://example.org/#spiderman> <http://www.perceive.net/schemas/relationship/enemyOf> <http://example.org/#green-goblin> <http://example.org/graphs/spiderman> .`, input: `<http://example.org/#spiderman> <http://www.perceive.net/schemas/relationship/enemyOf> <http://example.org/#green-goblin> <http://example.org/graphs/spiderman> .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "<http://example.org/#spiderman>", Subject: "<http://example.org/#spiderman>",
Predicate: "<http://www.perceive.net/schemas/relationship/enemyOf>", Predicate: "<http://www.perceive.net/schemas/relationship/enemyOf>",
Object: "<http://example.org/#green-goblin>", Object: "<http://example.org/#green-goblin>",
Provenance: "<http://example.org/graphs/spiderman>", Label: "<http://example.org/graphs/spiderman>",
}, },
err: nil, err: nil,
}, },
@ -186,10 +186,10 @@ var testNTriples = []struct {
message: "parse quad with blank node labelled subject and object and IRIREF predicate and label (1)", message: "parse quad with blank node labelled subject and object and IRIREF predicate and label (1)",
input: `_:alice <http://xmlns.com/foaf/0.1/knows> _:bob <http://example.org/graphs/john> .`, input: `_:alice <http://xmlns.com/foaf/0.1/knows> _:bob <http://example.org/graphs/john> .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "_:alice", Subject: "_:alice",
Predicate: "<http://xmlns.com/foaf/0.1/knows>", Predicate: "<http://xmlns.com/foaf/0.1/knows>",
Object: "_:bob", Object: "_:bob",
Provenance: "<http://example.org/graphs/john>", Label: "<http://example.org/graphs/john>",
}, },
err: nil, err: nil,
}, },
@ -197,10 +197,10 @@ var testNTriples = []struct {
message: "parse quad with blank node labelled subject and object and IRIREF predicate and label (2)", message: "parse quad with blank node labelled subject and object and IRIREF predicate and label (2)",
input: `_:bob <http://xmlns.com/foaf/0.1/knows> _:alice <http://example.org/graphs/james> .`, input: `_:bob <http://xmlns.com/foaf/0.1/knows> _:alice <http://example.org/graphs/james> .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "_:bob", Subject: "_:bob",
Predicate: "<http://xmlns.com/foaf/0.1/knows>", Predicate: "<http://xmlns.com/foaf/0.1/knows>",
Object: "_:alice", Object: "_:alice",
Provenance: "<http://example.org/graphs/james>", Label: "<http://example.org/graphs/james>",
}, },
err: nil, err: nil,
}, },
@ -210,10 +210,10 @@ var testNTriples = []struct {
message: "parse triple with all IRIREF parts", message: "parse triple with all IRIREF parts",
input: `<http://example.org/bob#me> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://xmlns.com/foaf/0.1/Person> .`, input: `<http://example.org/bob#me> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://xmlns.com/foaf/0.1/Person> .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "<http://example.org/bob#me>", Subject: "<http://example.org/bob#me>",
Predicate: "<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>", Predicate: "<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>",
Object: "<http://xmlns.com/foaf/0.1/Person>", Object: "<http://xmlns.com/foaf/0.1/Person>",
Provenance: "", Label: "",
}, },
err: nil, err: nil,
}, },
@ -221,10 +221,10 @@ var testNTriples = []struct {
message: "parse triple with all IRIREF parts", message: "parse triple with all IRIREF parts",
input: `<http://example.org/bob#me> <http://xmlns.com/foaf/0.1/knows> <http://example.org/alice#me> .`, input: `<http://example.org/bob#me> <http://xmlns.com/foaf/0.1/knows> <http://example.org/alice#me> .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "<http://example.org/bob#me>", Subject: "<http://example.org/bob#me>",
Predicate: "<http://xmlns.com/foaf/0.1/knows>", Predicate: "<http://xmlns.com/foaf/0.1/knows>",
Object: "<http://example.org/alice#me>", Object: "<http://example.org/alice#me>",
Provenance: "", Label: "",
}, },
err: nil, err: nil,
}, },
@ -232,10 +232,10 @@ var testNTriples = []struct {
message: "parse triple with IRIREF schema on literal object", message: "parse triple with IRIREF schema on literal object",
input: `<http://example.org/bob#me> <http://schema.org/birthDate> "1990-07-04"^^<http://www.w3.org/2001/XMLSchema#date> .`, input: `<http://example.org/bob#me> <http://schema.org/birthDate> "1990-07-04"^^<http://www.w3.org/2001/XMLSchema#date> .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "<http://example.org/bob#me>", Subject: "<http://example.org/bob#me>",
Predicate: "<http://schema.org/birthDate>", Predicate: "<http://schema.org/birthDate>",
Object: `"1990-07-04"^^<http://www.w3.org/2001/XMLSchema#date>`, Object: `"1990-07-04"^^<http://www.w3.org/2001/XMLSchema#date>`,
Provenance: "", Label: "",
}, },
err: nil, err: nil,
}, },
@ -243,10 +243,10 @@ var testNTriples = []struct {
message: "parse commented IRIREF in triple", message: "parse commented IRIREF in triple",
input: `<http://example.org/bob#me> <http://xmlns.com/foaf/0.1/topic_interest> <http://www.wikidata.org/entity/Q12418> .`, input: `<http://example.org/bob#me> <http://xmlns.com/foaf/0.1/topic_interest> <http://www.wikidata.org/entity/Q12418> .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "<http://example.org/bob#me>", Subject: "<http://example.org/bob#me>",
Predicate: "<http://xmlns.com/foaf/0.1/topic_interest>", Predicate: "<http://xmlns.com/foaf/0.1/topic_interest>",
Object: "<http://www.wikidata.org/entity/Q12418>", Object: "<http://www.wikidata.org/entity/Q12418>",
Provenance: "", Label: "",
}, },
err: nil, err: nil,
}, },
@ -254,10 +254,10 @@ var testNTriples = []struct {
message: "parse triple with literal subject", message: "parse triple with literal subject",
input: `<http://www.wikidata.org/entity/Q12418> <http://purl.org/dc/terms/title> "Mona Lisa" .`, input: `<http://www.wikidata.org/entity/Q12418> <http://purl.org/dc/terms/title> "Mona Lisa" .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "<http://www.wikidata.org/entity/Q12418>", Subject: "<http://www.wikidata.org/entity/Q12418>",
Predicate: "<http://purl.org/dc/terms/title>", Predicate: "<http://purl.org/dc/terms/title>",
Object: `"Mona Lisa"`, Object: `"Mona Lisa"`,
Provenance: "", Label: "",
}, },
err: nil, err: nil,
}, },
@ -265,10 +265,10 @@ var testNTriples = []struct {
message: "parse triple with all IRIREF parts (1)", message: "parse triple with all IRIREF parts (1)",
input: `<http://www.wikidata.org/entity/Q12418> <http://purl.org/dc/terms/creator> <http://dbpedia.org/resource/Leonardo_da_Vinci> .`, input: `<http://www.wikidata.org/entity/Q12418> <http://purl.org/dc/terms/creator> <http://dbpedia.org/resource/Leonardo_da_Vinci> .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "<http://www.wikidata.org/entity/Q12418>", Subject: "<http://www.wikidata.org/entity/Q12418>",
Predicate: "<http://purl.org/dc/terms/creator>", Predicate: "<http://purl.org/dc/terms/creator>",
Object: "<http://dbpedia.org/resource/Leonardo_da_Vinci>", Object: "<http://dbpedia.org/resource/Leonardo_da_Vinci>",
Provenance: "", Label: "",
}, },
err: nil, err: nil,
}, },
@ -276,10 +276,10 @@ var testNTriples = []struct {
message: "parse triple with all IRIREF parts (2)", message: "parse triple with all IRIREF parts (2)",
input: `<http://data.europeana.eu/item/04802/243FA8618938F4117025F17A8B813C5F9AA4D619> <http://purl.org/dc/terms/subject> <http://www.wikidata.org/entity/Q12418> .`, input: `<http://data.europeana.eu/item/04802/243FA8618938F4117025F17A8B813C5F9AA4D619> <http://purl.org/dc/terms/subject> <http://www.wikidata.org/entity/Q12418> .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "<http://data.europeana.eu/item/04802/243FA8618938F4117025F17A8B813C5F9AA4D619>", Subject: "<http://data.europeana.eu/item/04802/243FA8618938F4117025F17A8B813C5F9AA4D619>",
Predicate: "<http://purl.org/dc/terms/subject>", Predicate: "<http://purl.org/dc/terms/subject>",
Object: "<http://www.wikidata.org/entity/Q12418>", Object: "<http://www.wikidata.org/entity/Q12418>",
Provenance: "", Label: "",
}, },
err: nil, err: nil,
}, },
@ -289,10 +289,10 @@ var testNTriples = []struct {
message: "parse commented IRIREF in quad (1)", message: "parse commented IRIREF in quad (1)",
input: `<http://example.org/bob#me> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://xmlns.com/foaf/0.1/Person> <http://example.org/bob> .`, input: `<http://example.org/bob#me> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> <http://xmlns.com/foaf/0.1/Person> <http://example.org/bob> .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "<http://example.org/bob#me>", Subject: "<http://example.org/bob#me>",
Predicate: "<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>", Predicate: "<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>",
Object: "<http://xmlns.com/foaf/0.1/Person>", Object: "<http://xmlns.com/foaf/0.1/Person>",
Provenance: "<http://example.org/bob>", Label: "<http://example.org/bob>",
}, },
err: nil, err: nil,
}, },
@ -300,10 +300,10 @@ var testNTriples = []struct {
message: "parse quad with all IRIREF parts", message: "parse quad with all IRIREF parts",
input: `<http://example.org/bob#me> <http://xmlns.com/foaf/0.1/knows> <http://example.org/alice#me> <http://example.org/bob> .`, input: `<http://example.org/bob#me> <http://xmlns.com/foaf/0.1/knows> <http://example.org/alice#me> <http://example.org/bob> .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "<http://example.org/bob#me>", Subject: "<http://example.org/bob#me>",
Predicate: "<http://xmlns.com/foaf/0.1/knows>", Predicate: "<http://xmlns.com/foaf/0.1/knows>",
Object: "<http://example.org/alice#me>", Object: "<http://example.org/alice#me>",
Provenance: "<http://example.org/bob>", Label: "<http://example.org/bob>",
}, },
err: nil, err: nil,
}, },
@ -311,10 +311,10 @@ var testNTriples = []struct {
message: "parse quad with IRIREF schema on literal object", message: "parse quad with IRIREF schema on literal object",
input: `<http://example.org/bob#me> <http://schema.org/birthDate> "1990-07-04"^^<http://www.w3.org/2001/XMLSchema#date> <http://example.org/bob> .`, input: `<http://example.org/bob#me> <http://schema.org/birthDate> "1990-07-04"^^<http://www.w3.org/2001/XMLSchema#date> <http://example.org/bob> .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "<http://example.org/bob#me>", Subject: "<http://example.org/bob#me>",
Predicate: "<http://schema.org/birthDate>", Predicate: "<http://schema.org/birthDate>",
Object: `"1990-07-04"^^<http://www.w3.org/2001/XMLSchema#date>`, Object: `"1990-07-04"^^<http://www.w3.org/2001/XMLSchema#date>`,
Provenance: "<http://example.org/bob>", Label: "<http://example.org/bob>",
}, },
err: nil, err: nil,
}, },
@ -322,10 +322,10 @@ var testNTriples = []struct {
message: "parse commented IRIREF in quad (2)", message: "parse commented IRIREF in quad (2)",
input: `<http://example.org/bob#me> <http://xmlns.com/foaf/0.1/topic_interest> <http://www.wikidata.org/entity/Q12418> <http://example.org/bob> .`, input: `<http://example.org/bob#me> <http://xmlns.com/foaf/0.1/topic_interest> <http://www.wikidata.org/entity/Q12418> <http://example.org/bob> .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "<http://example.org/bob#me>", Subject: "<http://example.org/bob#me>",
Predicate: "<http://xmlns.com/foaf/0.1/topic_interest>", Predicate: "<http://xmlns.com/foaf/0.1/topic_interest>",
Object: "<http://www.wikidata.org/entity/Q12418>", Object: "<http://www.wikidata.org/entity/Q12418>",
Provenance: "<http://example.org/bob>", Label: "<http://example.org/bob>",
}, },
err: nil, err: nil,
}, },
@ -333,10 +333,10 @@ var testNTriples = []struct {
message: "parse literal object and colon qualified label in quad", message: "parse literal object and colon qualified label in quad",
input: `<http://www.wikidata.org/entity/Q12418> <http://purl.org/dc/terms/title> "Mona Lisa" <https://www.wikidata.org/wiki/Special:EntityData/Q12418> .`, input: `<http://www.wikidata.org/entity/Q12418> <http://purl.org/dc/terms/title> "Mona Lisa" <https://www.wikidata.org/wiki/Special:EntityData/Q12418> .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "<http://www.wikidata.org/entity/Q12418>", Subject: "<http://www.wikidata.org/entity/Q12418>",
Predicate: "<http://purl.org/dc/terms/title>", Predicate: "<http://purl.org/dc/terms/title>",
Object: `"Mona Lisa"`, Object: `"Mona Lisa"`,
Provenance: "<https://www.wikidata.org/wiki/Special:EntityData/Q12418>", Label: "<https://www.wikidata.org/wiki/Special:EntityData/Q12418>",
}, },
err: nil, err: nil,
}, },
@ -344,10 +344,10 @@ var testNTriples = []struct {
message: "parse all IRIREF parts with colon qualified label in quad (1)", message: "parse all IRIREF parts with colon qualified label in quad (1)",
input: `<http://www.wikidata.org/entity/Q12418> <http://purl.org/dc/terms/creator> <http://dbpedia.org/resource/Leonardo_da_Vinci> <https://www.wikidata.org/wiki/Special:EntityData/Q12418> .`, input: `<http://www.wikidata.org/entity/Q12418> <http://purl.org/dc/terms/creator> <http://dbpedia.org/resource/Leonardo_da_Vinci> <https://www.wikidata.org/wiki/Special:EntityData/Q12418> .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "<http://www.wikidata.org/entity/Q12418>", Subject: "<http://www.wikidata.org/entity/Q12418>",
Predicate: "<http://purl.org/dc/terms/creator>", Predicate: "<http://purl.org/dc/terms/creator>",
Object: "<http://dbpedia.org/resource/Leonardo_da_Vinci>", Object: "<http://dbpedia.org/resource/Leonardo_da_Vinci>",
Provenance: "<https://www.wikidata.org/wiki/Special:EntityData/Q12418>", Label: "<https://www.wikidata.org/wiki/Special:EntityData/Q12418>",
}, },
err: nil, err: nil,
}, },
@ -355,10 +355,10 @@ var testNTriples = []struct {
message: "parse all IRIREF parts with colon qualified label in quad (2)", message: "parse all IRIREF parts with colon qualified label in quad (2)",
input: `<http://data.europeana.eu/item/04802/243FA8618938F4117025F17A8B813C5F9AA4D619> <http://purl.org/dc/terms/subject> <http://www.wikidata.org/entity/Q12418> <https://www.wikidata.org/wiki/Special:EntityData/Q12418> .`, input: `<http://data.europeana.eu/item/04802/243FA8618938F4117025F17A8B813C5F9AA4D619> <http://purl.org/dc/terms/subject> <http://www.wikidata.org/entity/Q12418> <https://www.wikidata.org/wiki/Special:EntityData/Q12418> .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "<http://data.europeana.eu/item/04802/243FA8618938F4117025F17A8B813C5F9AA4D619>", Subject: "<http://data.europeana.eu/item/04802/243FA8618938F4117025F17A8B813C5F9AA4D619>",
Predicate: "<http://purl.org/dc/terms/subject>", Predicate: "<http://purl.org/dc/terms/subject>",
Object: "<http://www.wikidata.org/entity/Q12418>", Object: "<http://www.wikidata.org/entity/Q12418>",
Provenance: "<https://www.wikidata.org/wiki/Special:EntityData/Q12418>", Label: "<https://www.wikidata.org/wiki/Special:EntityData/Q12418>",
}, },
err: nil, err: nil,
}, },
@ -366,10 +366,10 @@ var testNTriples = []struct {
message: "parse all IRIREF parts (quad section - 1)", message: "parse all IRIREF parts (quad section - 1)",
input: `<http://example.org/bob> <http://purl.org/dc/terms/publisher> <http://example.org> .`, input: `<http://example.org/bob> <http://purl.org/dc/terms/publisher> <http://example.org> .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "<http://example.org/bob>", Subject: "<http://example.org/bob>",
Predicate: "<http://purl.org/dc/terms/publisher>", Predicate: "<http://purl.org/dc/terms/publisher>",
Object: "<http://example.org>", Object: "<http://example.org>",
Provenance: "", Label: "",
}, },
err: nil, err: nil,
}, },
@ -377,10 +377,10 @@ var testNTriples = []struct {
message: "parse all IRIREF parts (quad section - 2)", message: "parse all IRIREF parts (quad section - 2)",
input: `<http://example.org/bob> <http://purl.org/dc/terms/rights> <http://creativecommons.org/licenses/by/3.0/> .`, input: `<http://example.org/bob> <http://purl.org/dc/terms/rights> <http://creativecommons.org/licenses/by/3.0/> .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "<http://example.org/bob>", Subject: "<http://example.org/bob>",
Predicate: "<http://purl.org/dc/terms/rights>", Predicate: "<http://purl.org/dc/terms/rights>",
Object: "<http://creativecommons.org/licenses/by/3.0/>", Object: "<http://creativecommons.org/licenses/by/3.0/>",
Provenance: "", Label: "",
}, },
err: nil, err: nil,
}, },
@ -402,10 +402,10 @@ var testNTriples = []struct {
message: "parse incomplete quad", message: "parse incomplete quad",
input: `<http://example.org/bob#me> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> .`, input: `<http://example.org/bob#me> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "<http://example.org/bob#me>", Subject: "<http://example.org/bob#me>",
Predicate: "<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>", Predicate: "<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>",
Object: "", Object: "",
Provenance: "", Label: "",
}, },
err: fmt.Errorf("%v: unexpected rune '.' at 78", quad.ErrInvalid), err: fmt.Errorf("%v: unexpected rune '.' at 78", quad.ErrInvalid),
}, },
@ -413,10 +413,10 @@ var testNTriples = []struct {
message: "parse incomplete quad", message: "parse incomplete quad",
input: `<http://example.org/bob#me> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> .`, input: `<http://example.org/bob#me> <http://www.w3.org/1999/02/22-rdf-syntax-ns#type> .`,
expect: &quad.Quad{ expect: &quad.Quad{
Subject: "<http://example.org/bob#me>", Subject: "<http://example.org/bob#me>",
Predicate: "<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>", Predicate: "<http://www.w3.org/1999/02/22-rdf-syntax-ns#type>",
Object: "", Object: "",
Provenance: "", Label: "",
}, },
err: fmt.Errorf("%v: unexpected rune '.' at 78", quad.ErrInvalid), err: fmt.Errorf("%v: unexpected rune '.' at 78", quad.ErrInvalid),
}, },

10
quad/nquads/parse.go
View file

@ -619,7 +619,7 @@ tr96:
if label < 0 { if label < 0 {
panic("unexpected parser state: label start not set") panic("unexpected parser state: label start not set")
} }
q.Provenance = unEscape(data[label:p], isEscaped) q.Label = unEscape(data[label:p], isEscaped)
isEscaped = false isEscaped = false
goto st10 goto st10
@ -660,7 +660,7 @@ tr39:
if label < 0 { if label < 0 {
panic("unexpected parser state: label start not set") panic("unexpected parser state: label start not set")
} }
q.Provenance = unEscape(data[label:p], isEscaped) q.Label = unEscape(data[label:p], isEscaped)
isEscaped = false isEscaped = false
goto st88 goto st88
@ -785,7 +785,7 @@ tr38:
if label < 0 { if label < 0 {
panic("unexpected parser state: label start not set") panic("unexpected parser state: label start not set")
} }
q.Provenance = unEscape(data[label:p], isEscaped) q.Label = unEscape(data[label:p], isEscaped)
isEscaped = false isEscaped = false
goto st13 goto st13
@ -1202,7 +1202,7 @@ tr55:
if label < 0 { if label < 0 {
panic("unexpected parser state: label start not set") panic("unexpected parser state: label start not set")
} }
q.Provenance = unEscape(data[label:p], isEscaped) q.Label = unEscape(data[label:p], isEscaped)
isEscaped = false isEscaped = false
goto st90 goto st90
@ -2682,7 +2682,7 @@ tr97:
if label < 0 { if label < 0 {
panic("unexpected parser state: label start not set") panic("unexpected parser state: label start not set")
} }
q.Provenance = unEscape(data[label:p], isEscaped) q.Label = unEscape(data[label:p], isEscaped)
isEscaped = false isEscaped = false
goto st92 goto st92

28
quad/quad.go
View file

@ -26,8 +26,8 @@ package quad
// list of triples. The rest is just indexing for speed. // list of triples. The rest is just indexing for speed.
// //
// Adding fields to the triple is not to be taken lightly. You'll see I mention // Adding fields to the triple is not to be taken lightly. You'll see I mention
// provenance, but don'q as yet use it in any backing store. In general, there // label, but don't as yet use it in any backing store. In general, there
// can be features that can be turned on or off for any store, but I haven'q // can be features that can be turned on or off for any store, but I haven't
// decided how to allow/disallow them yet. Another such example would be to add // decided how to allow/disallow them yet. Another such example would be to add
// a forward and reverse index field -- forward being "order the list of // a forward and reverse index field -- forward being "order the list of
// objects pointed at by this subject with this predicate" such as first and // objects pointed at by this subject with this predicate" such as first and
@ -48,10 +48,10 @@ var (
// Our triple struct, used throughout. // Our triple struct, used throughout.
type Quad struct { type Quad struct {
Subject string `json:"subject"` Subject string `json:"subject"`
Predicate string `json:"predicate"` Predicate string `json:"predicate"`
Object string `json:"object"` Object string `json:"object"`
Provenance string `json:"provenance,omitempty"` Label string `json:"label,omitempty"`
} }
// Direction specifies an edge's type. // Direction specifies an edge's type.
@ -63,7 +63,7 @@ const (
Subject Subject
Predicate Predicate
Object Object
Provenance Label
) )
func (d Direction) Prefix() byte { func (d Direction) Prefix() byte {
@ -74,7 +74,7 @@ func (d Direction) Prefix() byte {
return 's' return 's'
case Predicate: case Predicate:
return 'p' return 'p'
case Provenance: case Label:
return 'c' return 'c'
case Object: case Object:
return 'o' return 'o'
@ -91,8 +91,8 @@ func (d Direction) String() string {
return "subject" return "subject"
case Predicate: case Predicate:
return "predicate" return "predicate"
case Provenance: case Label:
return "provenance" return "label"
case Object: case Object:
return "object" return "object"
default: default:
@ -110,8 +110,8 @@ func (q *Quad) Get(d Direction) string {
return q.Subject return q.Subject
case Predicate: case Predicate:
return q.Predicate return q.Predicate
case Provenance: case Label:
return q.Provenance return q.Label
case Object: case Object:
return q.Object return q.Object
default: default:
@ -139,11 +139,11 @@ func (q *Quad) IsValid() bool {
// Prints a triple in N-Quad format. // Prints a triple in N-Quad format.
func (q *Quad) NTriple() string { func (q *Quad) NTriple() string {
if q.Provenance == "" { if q.Label == "" {
//TODO(barakmich): Proper escaping. //TODO(barakmich): Proper escaping.
return fmt.Sprintf("%s %s %s .", q.Subject, q.Predicate, q.Object) return fmt.Sprintf("%s %s %s .", q.Subject, q.Predicate, q.Object)
} else { } else {
return fmt.Sprintf("%s %s %s %s .", q.Subject, q.Predicate, q.Object, q.Provenance) return fmt.Sprintf("%s %s %s %s .", q.Subject, q.Predicate, q.Object, q.Label)
} }
} }

4
static/js/cayley_write.js
View file

@ -48,7 +48,7 @@ $(function() {
subject: $("#subject").val(), subject: $("#subject").val(),
predicate: $("#predicate").val(), predicate: $("#predicate").val(),
object: $("#object").val(), object: $("#object").val(),
provenance: $("#provenance").val() label: $("#label").val()
} }
if (!checkTriple(triple)) { if (!checkTriple(triple)) {
return return
@ -68,7 +68,7 @@ $(function() {
subject: $("#rsubject").val(), subject: $("#rsubject").val(),
predicate: $("#rpredicate").val(), predicate: $("#rpredicate").val(),
object: $("#robject").val(), object: $("#robject").val(),
provenance: $("#rprovenance").val() label: $("#rlabel").val()
} }
if (!checkTriple(triple)) { if (!checkTriple(triple)) {
return return

4
templates/write.html
View file

@ -45,7 +45,7 @@
<input id="subject" type="text" placeholder="Subject"></input> <input id="subject" type="text" placeholder="Subject"></input>
<input id="predicate" type="text" placeholder="Predicate"></input> <input id="predicate" type="text" placeholder="Predicate"></input>
<input id="object" type="text" placeholder="Object"></input> <input id="object" type="text" placeholder="Object"></input>
<input id="provenance" type="text" placeholder="Provenance"></input> <input id="label" type="text" placeholder="Label"></input>
</div> </div>
</div> </div>
<div class="row button-row"> <div class="row button-row">
@ -59,7 +59,7 @@
<input id="rsubject" type="text" placeholder="Subject"></input> <input id="rsubject" type="text" placeholder="Subject"></input>
<input id="rpredicate" type="text" placeholder="Predicate"></input> <input id="rpredicate" type="text" placeholder="Predicate"></input>
<input id="robject" type="text" placeholder="Object"></input> <input id="robject" type="text" placeholder="Object"></input>
<input id="rprovenance" type="text" placeholder="Provenance"></input> <input id="rlabel" type="text" placeholder="Label"></input>
</div> </div>
</div><!-- /.col-xs-12 main --> </div><!-- /.col-xs-12 main -->
<div class="row button-row"> <div class="row button-row">