barak/cayley
1
0
Fork 0
Code Issues Pull requests Releases Wiki Activity

Merge pull request #148 from kortschak/renaming

Browse source 
Renaming triple-related entities to the quads they actually are (eg, TripleStore -> QuadStore)
This commit is contained in:
Barak Michener 2014-08-27 16:36:06 -04:00
commit 6614466d23
72 changed files with 2152 additions and 2378 deletions
Download patch file Download diff file Expand all files Collapse all files

6
README.md
View file

@ -27,15 +27,15 @@ Its goal is to be a part of the developer's toolbox where [Linked Data](http://l
* JavaScript, with a [Gremlin](http://gremlindocs.com/)-inspired\* graph object. * JavaScript, with a [Gremlin](http://gremlindocs.com/)-inspired\* graph object.
* (simplified) [MQL](https://developers.google.com/freebase/v1/mql-overview), for Freebase fans * (simplified) [MQL](https://developers.google.com/freebase/v1/mql-overview), for Freebase fans
* Plays well with multiple backend stores: * Plays well with multiple backend stores:
* [LevelDB](http://code.google.com/p/leveldb/) * [LevelDB](http://code.google.com/p/leveldb/)
* [Bolt](http://github.com/boltdb/bolt) * [Bolt](http://github.com/boltdb/bolt)
* [MongoDB](http://mongodb.org) for distributed stores * [MongoDB](http://mongodb.org) for distributed stores
* In-memory, ephemeral * In-memory, ephemeral
* Modular design; easy to extend with new languages and backends * Modular design; easy to extend with new languages and backends
* Good test coverage * Good test coverage
* Speed, where possible. * Speed, where possible.
Rough performance testing shows that, on consumer hardware and an average disk, 134m triples in LevelDB is no problem and a multi-hop intersection query -- films starring X and Y -- takes ~150ms. Rough performance testing shows that, on consumer hardware and an average disk, 134m quads in LevelDB is no problem and a multi-hop intersection query -- films starring X and Y -- takes ~150ms.
\* Note that while it's not exactly Gremlin, it certainly takes inspiration from that API. For this flavor, [see the documentation](docs/GremlinAPI.md). \* Note that while it's not exactly Gremlin, it certainly takes inspiration from that API. For this flavor, [see the documentation](docs/GremlinAPI.md).

6
TODO.md
View file

@ -24,7 +24,7 @@ Also always good.
Usually something that should be taken care of. Usually something that should be taken care of.
### Bootstraps ### Bootstraps
Start discussing bootstrap triples, things that make the database self-describing, if they exist (though they need not). Talk about sameAs and indexing and type systems and whatnot. Start discussing bootstrap quads, things that make the database self-describing, if they exist (though they need not). Talk about sameAs and indexing and type systems and whatnot.
### Better surfacing of Label ### Better surfacing of Label
It exists, it's indexed, but it's basically useless right now It exists, it's indexed, but it's basically useless right now
@ -68,7 +68,7 @@ The necessary component to make mid-query limit work. Acts as a limit on Next(),
### Postgres Backend ### Postgres Backend
It'd be nice to run on SQL as well. It's a big why not? It'd be nice to run on SQL as well. It's a big why not?
#### Generalist layout #### Generalist layout
Notionally, this is a simple triple table with a number of indicies. Iterators and iterator optimization (ie, rewriting SQL queries) is the 'fun' part Notionally, this is a simple quad table with a number of indicies. Iterators and iterator optimization (ie, rewriting SQL queries) is the 'fun' part
#### "Short Schema" Layout? #### "Short Schema" Layout?
This one is the crazy one. Suppose a world where we actually use the table schema for predicates, and update the table schema as we go along. Yes, it sucks when you add a new predicate (and the cell values are unclear) but for small worlds (or, "short schemas") it may (or may not) be interesting. This one is the crazy one. Suppose a world where we actually use the table schema for predicates, and update the table schema as we go along. Yes, it sucks when you add a new predicate (and the cell values are unclear) but for small worlds (or, "short schemas") it may (or may not) be interesting.
@ -83,7 +83,7 @@ The necessary component to make mid-query limit work. Acts as a limit on Next(),
There's a whole body of work there, and a lot of interested researchers. They're the choir who already know the sermon of graph stores. Once ease-of-use gets people in the door, supporting extensions that make everyone happy seems like a win. And because we're query-language agnostic, it's a cleaner win. See also bootstrapping, which is the first goal toward this (eg, let's talk about sameAs, and index it appropriately.) There's a whole body of work there, and a lot of interested researchers. They're the choir who already know the sermon of graph stores. Once ease-of-use gets people in the door, supporting extensions that make everyone happy seems like a win. And because we're query-language agnostic, it's a cleaner win. See also bootstrapping, which is the first goal toward this (eg, let's talk about sameAs, and index it appropriately.)
### Replication ### Replication
Technically it works now if you piggyback on someone else's replication, but that's cheating. We speak HTTP, we can send triple sets over the wire to some other instance. Bonus points for a way to apply morphisms first -- massive graph on the backend, important graph on the frontend. Technically it works now if you piggyback on someone else's replication, but that's cheating. We speak HTTP, we can send quad sets over the wire to some other instance. Bonus points for a way to apply morphisms first -- massive graph on the backend, important graph on the frontend.
### Related services ### Related services
Eg, topic service, recon service -- whether in Cayley itself or as part of the greater project. Eg, topic service, recon service -- whether in Cayley itself or as part of the greater project.

6
appengine.go
View file

@ -30,8 +30,8 @@ import (
func init() { func init() {
glog.SetToStderr(true) glog.SetToStderr(true)
cfg := config.ParseConfigFromFile("cayley_appengine.cfg") cfg := config.ParseConfigFromFile("cayley_appengine.cfg")
ts, _ := graph.NewTripleStore("memstore", "", nil) qs, _ := graph.NewQuadStore("memstore", "", nil)
glog.Errorln(cfg) glog.Errorln(cfg)
db.Load(ts, cfg, cfg.DatabasePath) db.Load(qs, cfg, cfg.DatabasePath)
http.SetupRoutes(ts, cfg) http.SetupRoutes(qs, cfg)
} }

18
cayley.go
View file

@ -52,8 +52,8 @@ import (
) )
var ( var (
tripleFile = flag.String("triples", "", "Triple File to load before going to REPL.") quadFile = flag.String("quads", "", "Quad file to load before going to REPL.")
tripleType = flag.String("format", "cquad", `Triple format to use for loading ("cquad" or "nquad").`) quadType = flag.String("format", "cquad", `Quad format to use for loading ("cquad" or "nquad").`)
cpuprofile = flag.String("prof", "", "Output profiling file.") cpuprofile = flag.String("prof", "", "Output profiling file.")
queryLanguage = flag.String("query_lang", "gremlin", "Use this parser as the query language.") queryLanguage = flag.String("query_lang", "gremlin", "Use this parser as the query language.")
configFile = flag.String("config", "", "Path to an explicit configuration file.") configFile = flag.String("config", "", "Path to an explicit configuration file.")
@ -61,7 +61,7 @@ var (
databaseBackend = flag.String("db", "memstore", "Database Backend.") databaseBackend = flag.String("db", "memstore", "Database Backend.")
replicationBackend = flag.String("replication", "single", "Replication method.") replicationBackend = flag.String("replication", "single", "Replication method.")
host = flag.String("host", "127.0.0.1", "Host to listen on (defaults to all).") host = flag.String("host", "127.0.0.1", "Host to listen on (defaults to all).")
loadSize = flag.Int("load_size", 10000, "Size of triplesets to load") loadSize = flag.Int("load_size", 10000, "Size of quadsets to load")
port = flag.String("port", "64210", "Port to listen on.") port = flag.String("port", "64210", "Port to listen on.")
readOnly = flag.Bool("read_only", false, "Disable writing via HTTP.") readOnly = flag.Bool("read_only", false, "Disable writing via HTTP.")
timeout = flag.Duration("timeout", 30*time.Second, "Elapsed time until an individual query times out.") timeout = flag.Duration("timeout", 30*time.Second, "Elapsed time until an individual query times out.")
@ -80,7 +80,7 @@ Usage:
Commands: Commands:
init Create an empty database. init Create an empty database.
load Bulk-load a triple file into the database. load Bulk-load a quad file into the database.
http Serve an HTTP endpoint on the given host and port. http Serve an HTTP endpoint on the given host and port.
repl Drop into a REPL of the given query language. repl Drop into a REPL of the given query language.
version Version information. version Version information.
@ -190,12 +190,12 @@ func main() {
if err != nil { if err != nil {
break break
} }
if *tripleFile != "" { if *quadFile != "" {
handle, err = db.Open(cfg) handle, err = db.Open(cfg)
if err != nil { if err != nil {
break break
} }
err = load(handle.QuadWriter, cfg, *tripleFile, *tripleType) err = load(handle.QuadWriter, cfg, *quadFile, *quadType)
if err != nil { if err != nil {
break break
} }
@ -207,7 +207,7 @@ func main() {
if err != nil { if err != nil {
break break
} }
err = load(handle.QuadWriter, cfg, *tripleFile, *tripleType) err = load(handle.QuadWriter, cfg, *quadFile, *quadType)
if err != nil { if err != nil {
break break
} }
@ -220,7 +220,7 @@ func main() {
break break
} }
if !graph.IsPersistent(cfg.DatabaseType) { if !graph.IsPersistent(cfg.DatabaseType) {
err = load(handle.QuadWriter, cfg, "", *tripleType) err = load(handle.QuadWriter, cfg, "", *quadType)
if err != nil { if err != nil {
break break
} }
@ -236,7 +236,7 @@ func main() {
break break
} }
if !graph.IsPersistent(cfg.DatabaseType) { if !graph.IsPersistent(cfg.DatabaseType) {
err = load(handle.QuadWriter, cfg, "", *tripleType) err = load(handle.QuadWriter, cfg, "", *quadType)
if err != nil { if err != nil {
break break
} }

16
cayley_test.go
View file

@ -20,7 +20,6 @@ import (
"compress/gzip" "compress/gzip"
"flag" "flag"
"fmt" "fmt"
"github.com/google/cayley/quad"
"io" "io"
"os" "os"
"reflect" "reflect"
@ -33,6 +32,7 @@ import (
"github.com/google/cayley/config" "github.com/google/cayley/config"
"github.com/google/cayley/db" "github.com/google/cayley/db"
"github.com/google/cayley/graph" "github.com/google/cayley/graph"
"github.com/google/cayley/quad"
"github.com/google/cayley/query/gremlin" "github.com/google/cayley/query/gremlin"
) )
@ -373,8 +373,6 @@ var (
create sync.Once create sync.Once
deleteAndRecreate sync.Once deleteAndRecreate sync.Once
cfg = &config.Config{ cfg = &config.Config{
DatabasePath: "30kmoviedata.nq.gz",
DatabaseType: "memstore",
ReplicationType: "single", ReplicationType: "single",
Timeout: 300 * time.Second, Timeout: 300 * time.Second,
} }
@ -383,17 +381,17 @@ var (
) )
func prepare(t testing.TB) { func prepare(t testing.TB) {
cfg.DatabaseType = *backend
switch *backend { switch *backend {
case "memstore": case "memstore":
break cfg.DatabasePath = "30kmoviedata.nq.gz"
case "leveldb": case "leveldb", "bolt":
fallthrough cfg.DatabasePath = "/tmp/cayley_test_" + *backend
case "bolt":
cfg.DatabaseType = *backend
cfg.DatabasePath = fmt.Sprint("/tmp/cayley_test_", *backend)
cfg.DatabaseOptions = map[string]interface{}{ cfg.DatabaseOptions = map[string]interface{}{
"nosync": true, // It's a test. If we need to load, do it fast. "nosync": true, // It's a test. If we need to load, do it fast.
} }
case "mongo":
cfg.DatabasePath = "localhost:27017"
default: default:
t.Fatalf("Untestable backend store %s", *backend) t.Fatalf("Untestable backend store %s", *backend)
} }

10
db/db.go
View file

@ -33,7 +33,7 @@ func Init(cfg *config.Config) error {
return fmt.Errorf("ignoring unproductive database initialization request: %v", ErrNotPersistent) return fmt.Errorf("ignoring unproductive database initialization request: %v", ErrNotPersistent)
} }
return graph.InitTripleStore(cfg.DatabaseType, cfg.DatabasePath, cfg.DatabaseOptions) return graph.InitQuadStore(cfg.DatabaseType, cfg.DatabasePath, cfg.DatabaseOptions)
} }
func Open(cfg *config.Config) (*graph.Handle, error) { func Open(cfg *config.Config) (*graph.Handle, error) {
@ -48,17 +48,17 @@ func Open(cfg *config.Config) (*graph.Handle, error) {
return &graph.Handle{QuadStore: qs, QuadWriter: qw}, nil return &graph.Handle{QuadStore: qs, QuadWriter: qw}, nil
} }
func OpenQuadStore(cfg *config.Config) (graph.TripleStore, error) { func OpenQuadStore(cfg *config.Config) (graph.QuadStore, error) {
glog.Infof("Opening quad store %q at %s", cfg.DatabaseType, cfg.DatabasePath) glog.Infof("Opening quad store %q at %s", cfg.DatabaseType, cfg.DatabasePath)
ts, err := graph.NewTripleStore(cfg.DatabaseType, cfg.DatabasePath, cfg.DatabaseOptions) qs, err := graph.NewQuadStore(cfg.DatabaseType, cfg.DatabasePath, cfg.DatabaseOptions)
if err != nil { if err != nil {
return nil, err return nil, err
} }
return ts, nil return qs, nil
} }
func OpenQuadWriter(qs graph.TripleStore, cfg *config.Config) (graph.QuadWriter, error) { func OpenQuadWriter(qs graph.QuadStore, cfg *config.Config) (graph.QuadWriter, error) {
glog.Infof("Opening replication method %q", cfg.ReplicationType) glog.Infof("Opening replication method %q", cfg.ReplicationType)
w, err := graph.NewQuadWriter(cfg.ReplicationType, qs, cfg.ReplicationOptions) w, err := graph.NewQuadWriter(cfg.ReplicationType, qs, cfg.ReplicationOptions)
if err != nil { if err != nil {

8
docs/Configuration.md
View file

@ -23,7 +23,7 @@ All command line flags take precedence over the configuration file.
* `mem`: An in-memory store, based on an initial N-Quads file. Loses all changes when the process exits. * `mem`: An in-memory store, based on an initial N-Quads file. Loses all changes when the process exits.
* `leveldb`: A persistent on-disk store backed by [LevelDB](http://code.google.com/p/leveldb/). * `leveldb`: A persistent on-disk store backed by [LevelDB](http://code.google.com/p/leveldb/).
* `bolt`: Stores the graph data on-disk in a [Bolt](http://github.com/boltdb/bolt) file. Uses more disk space and memory than LevelDB for smaller stores, but is often faster to write to and comparable for large ones, with faster average query times. * `bolt`: Stores the graph data on-disk in a [Bolt](http://github.com/boltdb/bolt) file. Uses more disk space and memory than LevelDB for smaller stores, but is often faster to write to and comparable for large ones, with faster average query times.
* `mongo`: Stores the graph data and indices in a [MongoDB](http://mongodb.org) instance. Slower, as it incurs network traffic, but multiple Cayley instances can disappear and reconnect at will, across a potentially horizontally-scaled store. * `mongo`: Stores the graph data and indices in a [MongoDB](http://mongodb.org) instance. Slower, as it incurs network traffic, but multiple Cayley instances can disappear and reconnect at will, across a potentially horizontally-scaled store.
#### **`db_path`** #### **`db_path`**
@ -33,7 +33,7 @@ All command line flags take precedence over the configuration file.
Where does the database actually live? Dependent on the type of database. For each datastore: Where does the database actually live? Dependent on the type of database. For each datastore:
* `mem`: Path to a triple file to automatically load. * `mem`: Path to a quad file to automatically load.
* `leveldb`: Directory to hold the LevelDB database files. * `leveldb`: Directory to hold the LevelDB database files.
* `bolt`: Path to the persistent single Bolt database file. * `bolt`: Path to the persistent single Bolt database file.
* `mongo`: "hostname:port" of the desired MongoDB server. * `mongo`: "hostname:port" of the desired MongoDB server.
@ -64,7 +64,7 @@ All command line flags take precedence over the configuration file.
* Type: Integer * Type: Integer
* Default: 10000 * Default: 10000
The number of triples to buffer from a loaded file before writing a block of triples to the database. Larger numbers are good for larger loads. The number of quads to buffer from a loaded file before writing a block of quads to the database. Larger numbers are good for larger loads.
#### **`db_options`** #### **`db_options`**
@ -103,7 +103,7 @@ The size in MiB of the LevelDB write cache. Increasing this number allows for mo
* Type: Integer * Type: Integer
* Default: 2 * Default: 2
The size in MiB of the LevelDB block cache. Increasing this number uses more memory to maintain a bigger cache of triple blocks for better performance. The size in MiB of the LevelDB block cache. Increasing this number uses more memory to maintain a bigger cache of quad blocks for better performance.
### Bolt ### Bolt

6
docs/GremlinAPI.md
View file

@ -85,7 +85,7 @@ Arguments:
* a string: A single tag to add the predicate used to the output set. * a string: A single tag to add the predicate used to the output set.
* a list of strings: Multiple tags to use as keys to save the predicate used to the output set. * a list of strings: Multiple tags to use as keys to save the predicate used to the output set.
Out is the work-a-day way to get between nodes, in the forward direction. Starting with the nodes in `path` on the subject, follow the triples with predicates defined by `predicatePath` to their objects. Out is the work-a-day way to get between nodes, in the forward direction. Starting with the nodes in `path` on the subject, follow the quads with predicates defined by `predicatePath` to their objects.
Example: Example:
```javascript ```javascript
@ -117,7 +117,7 @@ Arguments:
* a string: A single tag to add the predicate used to the output set. * a string: A single tag to add the predicate used to the output set.
* a list of strings: Multiple tags to use as keys to save the predicate used to the output set. * a list of strings: Multiple tags to use as keys to save the predicate used to the output set.
Same as Out, but in the other direction. Starting with the nodes in `path` on the object, follow the triples with predicates defined by `predicatePath` to their subjects. Same as Out, but in the other direction. Starting with the nodes in `path` on the object, follow the quads with predicates defined by `predicatePath` to their subjects.
Example: Example:
```javascript ```javascript
@ -235,7 +235,7 @@ Arguments:
* `predicate`: A string for a predicate node. * `predicate`: A string for a predicate node.
* `tag`: A string for a tag key to store the object node. * `tag`: A string for a tag key to store the object node.
From the current node as the subject, save the object of all triples with `predicate` into `tag`, without traversal. From the current node as the subject, save the object of all quads with `predicate` into `tag`, without traversal.
Example: Example:
```javascript ```javascript

8
docs/HTTP.md
View file

@ -86,7 +86,7 @@ Responses come in the form
#### `/api/v1/write` #### `/api/v1/write`
POST Body: JSON triples POST Body: JSON quads
```json ```json
[{ [{
@ -94,7 +94,7 @@ POST Body: JSON triples
"predicate": "Predicate Node", "predicate": "Predicate Node",
"object": "Object node", "object": "Object node",
"label": "Label node" // Optional "label": "Label node" // Optional
}] // More than one triple allowed. }] // More than one quad allowed.
``` ```
Response: JSON response message Response: JSON response message
@ -114,7 +114,7 @@ curl http://localhost:64210/api/v1/write/file/nquad -F NQuadFile=@30k.n3
#### `/api/v1/delete` #### `/api/v1/delete`
POST Body: JSON triples POST Body: JSON quads
```json ```json
[{ [{
@ -122,7 +122,7 @@ POST Body: JSON triples
"predicate": "Predicate Node", "predicate": "Predicate Node",
"object": "Object node", "object": "Object node",
"label": "Label node" // Optional "label": "Label node" // Optional
}] // More than one triple allowed. }] // More than one quad allowed.
``` ```
Response: JSON response message. Response: JSON response message.

4
docs/MQL.md
View file

@ -40,7 +40,7 @@ Predicates always assume a forward direction. That is,
}] }]
``` ```
will only match if the triple will only match if the quad
``` ```
A some_predicate B . A some_predicate B .
``` ```
@ -54,7 +54,7 @@ exists. In order to reverse the directions, "!predicates" are used. So that:
}] }]
``` ```
will only match if the triple will only match if the quad
``` ```
B some_predicate A . B some_predicate A .
``` ```

6
docs/Overview.md
View file

@ -29,13 +29,13 @@ You can repeat the `--db` and `--dbpath` flags from here forward instead of the
First we load the data. First we load the data.
```bash ```bash
./cayley load --config=cayley.cfg.overview --triples=30kmoviedata.nq.gz ./cayley load --config=cayley.cfg.overview --quads=30kmoviedata.nq.gz
``` ```
And wait. It will load. If you'd like to watch it load, you can run And wait. It will load. If you'd like to watch it load, you can run
```bash ```bash
./cayley load --config=cayley.cfg.overview --triples=30kmoviedata.nq.gz --alsologtostderr ./cayley load --config=cayley.cfg.overview --quads=30kmoviedata.nq.gz --alsologtostderr
``` ```
And watch the log output go by. And watch the log output go by.
@ -82,7 +82,7 @@ Along the side are the various actions or views you can take. From the top, thes
* Query (a request/response editor for the query language) * Query (a request/response editor for the query language)
* Query Shape (a visualization of the shape of the final query. Does not execute the query.) * Query Shape (a visualization of the shape of the final query. Does not execute the query.)
* Visualize (runs a query and, if tagged correctly, gives a sigmajs view of the results) * Visualize (runs a query and, if tagged correctly, gives a sigmajs view of the results)
* Write (an interface to write or remove individual triples or triple files) * Write (an interface to write or remove individual quads or quad files)
---- ----

2
graph/bolt/iterator.go
View file

@ -273,7 +273,7 @@ func (it *Iterator) Contains(v graph.Value) bool {
} }
offset := PositionOf(val, it.dir, it.qs) offset := PositionOf(val, it.dir, it.qs)
if bytes.HasPrefix(val.key[offset:], it.checkId) { if bytes.HasPrefix(val.key[offset:], it.checkId) {
// You may ask, why don't we check to see if it's a valid (not deleted) triple // You may ask, why don't we check to see if it's a valid (not deleted) quad
// again? // again?
// //
// We've already done that -- in order to get the graph.Value token in the // We've already done that -- in order to get the graph.Value token in the

48
graph/bolt/quadstore.go
View file

@ -32,7 +32,7 @@ import (
) )
func init() { func init() {
graph.RegisterTripleStore("bolt", true, newQuadStore, createNewBolt) graph.RegisterQuadStore("bolt", true, newQuadStore, createNewBolt)
} }
var ( var (
@ -77,7 +77,7 @@ func createNewBolt(path string, _ graph.Options) error {
return nil return nil
} }
func newQuadStore(path string, options graph.Options) (graph.TripleStore, error) { func newQuadStore(path string, options graph.Options) (graph.QuadStore, error) {
var qs QuadStore var qs QuadStore
var err error var err error
db, err := bolt.Open(path, 0600, nil) db, err := bolt.Open(path, 0600, nil)
@ -136,18 +136,28 @@ func bucketFor(d [4]quad.Direction) []byte {
return []byte{d[0].Prefix(), d[1].Prefix(), d[2].Prefix(), d[3].Prefix()} return []byte{d[0].Prefix(), d[1].Prefix(), d[2].Prefix(), d[3].Prefix()}
} }
func (qs *QuadStore) createKeyFor(d [4]quad.Direction, triple quad.Quad) []byte { func hashOf(s string) []byte {
h := hashPool.Get().(hash.Hash)
h.Reset()
defer hashPool.Put(h)
key := make([]byte, 0, hashSize)
h.Write([]byte(s))
key = h.Sum(key)
return key
}
func (qs *QuadStore) createKeyFor(d [4]quad.Direction, q quad.Quad) []byte {
key := make([]byte, 0, (hashSize * 4)) key := make([]byte, 0, (hashSize * 4))
key = append(key, qs.convertStringToByteHash(triple.Get(d[0]))...) key = append(key, hashOf(q.Get(d[0]))...)
key = append(key, qs.convertStringToByteHash(triple.Get(d[1]))...) key = append(key, hashOf(q.Get(d[1]))...)
key = append(key, qs.convertStringToByteHash(triple.Get(d[2]))...) key = append(key, hashOf(q.Get(d[2]))...)
key = append(key, qs.convertStringToByteHash(triple.Get(d[3]))...) key = append(key, hashOf(q.Get(d[3]))...)
return key return key
} }
func (qs *QuadStore) createValueKeyFor(s string) []byte { func (qs *QuadStore) createValueKeyFor(s string) []byte {
key := make([]byte, 0, hashSize) key := make([]byte, 0, hashSize)
key = append(key, qs.convertStringToByteHash(s)...) key = append(key, hashOf(s)...)
return key return key
} }
@ -243,11 +253,11 @@ func (qs *QuadStore) buildQuadWrite(tx *bolt.Tx, q quad.Quad, id int64, isAdd bo
} }
if isAdd && len(entry.History)%2 == 1 { if isAdd && len(entry.History)%2 == 1 {
glog.Error("Adding a valid triple ", entry) glog.Error("Adding a valid quad ", entry)
return graph.ErrQuadExists return graph.ErrQuadExists
} }
if !isAdd && len(entry.History)%2 == 0 { if !isAdd && len(entry.History)%2 == 0 {
glog.Error("Deleting an invalid triple ", entry) glog.Error("Deleting an invalid quad ", entry)
return graph.ErrQuadNotExist return graph.ErrQuadNotExist
} }
@ -373,22 +383,12 @@ func (qs *QuadStore) Quad(k graph.Value) quad.Quad {
return json.Unmarshal(data, &q) return json.Unmarshal(data, &q)
}) })
if err != nil { if err != nil {
glog.Error("Error getting triple: ", err) glog.Error("Error getting quad: ", err)
return quad.Quad{} return quad.Quad{}
} }
return q return q
} }
func (qs *QuadStore) convertStringToByteHash(s string) []byte {
h := hashPool.Get().(hash.Hash)
h.Reset()
defer hashPool.Put(h)
key := make([]byte, 0, hashSize)
h.Write([]byte(s))
key = h.Sum(key)
return key
}
func (qs *QuadStore) ValueOf(s string) graph.Value { func (qs *QuadStore) ValueOf(s string) graph.Value {
return &Token{ return &Token{
bucket: nodeBucket, bucket: nodeBucket,
@ -459,7 +459,7 @@ func (qs *QuadStore) getMetadata() error {
return err return err
} }
func (qs *QuadStore) TripleIterator(d quad.Direction, val graph.Value) graph.Iterator { func (qs *QuadStore) QuadIterator(d quad.Direction, val graph.Value) graph.Iterator {
var bucket []byte var bucket []byte
switch d { switch d {
case quad.Subject: case quad.Subject:
@ -480,11 +480,11 @@ func (qs *QuadStore) NodesAllIterator() graph.Iterator {
return NewAllIterator(nodeBucket, quad.Any, qs) return NewAllIterator(nodeBucket, quad.Any, qs)
} }
func (qs *QuadStore) TriplesAllIterator() graph.Iterator { func (qs *QuadStore) QuadsAllIterator() graph.Iterator {
return NewAllIterator(posBucket, quad.Predicate, qs) return NewAllIterator(posBucket, quad.Predicate, qs)
} }
func (qs *QuadStore) TripleDirection(val graph.Value, d quad.Direction) graph.Value { func (qs *QuadStore) QuadDirection(val graph.Value, d quad.Direction) graph.Value {
v := val.(*Token) v := val.(*Token)
offset := PositionOf(v, d, qs) offset := PositionOf(v, d, qs)
if offset != -1 { if offset != -1 {

8
graph/bolt/quadstore_iterator_optimize.go
View file

@ -19,16 +19,16 @@ import (
"github.com/google/cayley/graph/iterator" "github.com/google/cayley/graph/iterator"
) )
func (ts *QuadStore) OptimizeIterator(it graph.Iterator) (graph.Iterator, bool) { func (qs *QuadStore) OptimizeIterator(it graph.Iterator) (graph.Iterator, bool) {
switch it.Type() { switch it.Type() {
case graph.LinksTo: case graph.LinksTo:
return ts.optimizeLinksTo(it.(*iterator.LinksTo)) return qs.optimizeLinksTo(it.(*iterator.LinksTo))
} }
return it, false return it, false
} }
func (ts *QuadStore) optimizeLinksTo(it *iterator.LinksTo) (graph.Iterator, bool) { func (qs *QuadStore) optimizeLinksTo(it *iterator.LinksTo) (graph.Iterator, bool) {
subs := it.SubIterators() subs := it.SubIterators()
if len(subs) != 1 { if len(subs) != 1 {
return it, false return it, false
@ -41,7 +41,7 @@ func (ts *QuadStore) optimizeLinksTo(it *iterator.LinksTo) (graph.Iterator, bool
panic("unexpected size during optimize") panic("unexpected size during optimize")
} }
val := primary.Result() val := primary.Result()
newIt := ts.TripleIterator(it.Direction(), val) newIt := qs.QuadIterator(it.Direction(), val)
nt := newIt.Tagger() nt := newIt.Tagger()
nt.CopyFrom(it) nt.CopyFrom(it)
for _, tag := range primary.Tagger().Tags() { for _, tag := range primary.Tagger().Tags() {

4
graph/iterator/all_iterator.go
View file

@ -17,9 +17,9 @@ package iterator
// Defines one of the base iterators, the All iterator. Which, logically // Defines one of the base iterators, the All iterator. Which, logically
// enough, represents all nodes or all links in the graph. // enough, represents all nodes or all links in the graph.
// //
// This particular file is actually vestigial. It's up to the TripleStore to give // This particular file is actually vestigial. It's up to the QuadStore to give
// us an All iterator that represents all things in the graph. So this is // us an All iterator that represents all things in the graph. So this is
// really the All iterator for the MemTripleStore. That said, it *is* one of // really the All iterator for the memstore.QuadStore. That said, it *is* one of
// the base iterators, and it helps just to see it here. // the base iterators, and it helps just to see it here.
import ( import (

2
graph/iterator/and_iterator.go
View file

@ -1,5 +1,5 @@
// Defines the And iterator, one of the base iterators. And requires no // Defines the And iterator, one of the base iterators. And requires no
// knowledge of the constituent TripleStore; its sole purpose is to act as an // knowledge of the constituent QuadStore; its sole purpose is to act as an
// intersection operator across the subiterators it is given. If one iterator // intersection operator across the subiterators it is given. If one iterator
// contains [1,3,5] and another [2,3,4] -- then And is an iterator that // contains [1,3,5] and another [2,3,4] -- then And is an iterator that
// 'contains' [3] // 'contains' [3]

2
graph/iterator/fixed_iterator.go
View file

@ -18,7 +18,7 @@ package iterator
// contains an explicit fixed array of values. // contains an explicit fixed array of values.
// //
// A fixed iterator requires an Equality function to be passed to it, by reason that graph.Value, the // A fixed iterator requires an Equality function to be passed to it, by reason that graph.Value, the
// opaque Triple store value, may not answer to ==. // opaque Quad store value, may not answer to ==.
import ( import (
"fmt" "fmt"

42
graph/iterator/hasa_iterator.go
View file

@ -21,7 +21,7 @@ package iterator
// //
// HasA is weird in that it may return the same value twice if on the Next() // HasA is weird in that it may return the same value twice if on the Next()
// path. That's okay -- in reality, it can be viewed as returning the value for // path. That's okay -- in reality, it can be viewed as returning the value for
// a new triple, but to make logic much simpler, here we have the HasA. // a new quad, but to make logic much simpler, here we have the HasA.
// //
// Likewise, it's important to think about Contains()ing a HasA. When given a // Likewise, it's important to think about Contains()ing a HasA. When given a
// value to check, it means "Check all predicates that have this value for your // value to check, it means "Check all predicates that have this value for your
@ -43,13 +43,13 @@ import (
"github.com/google/cayley/quad" "github.com/google/cayley/quad"
) )
// A HasA consists of a reference back to the graph.TripleStore that it references, // A HasA consists of a reference back to the graph.QuadStore that it references,
// a primary subiterator, a direction in which the triples for that subiterator point, // a primary subiterator, a direction in which the quads for that subiterator point,
// and a temporary holder for the iterator generated on Contains(). // and a temporary holder for the iterator generated on Contains().
type HasA struct { type HasA struct {
uid uint64 uid uint64
tags graph.Tagger tags graph.Tagger
ts graph.TripleStore qs graph.QuadStore
primaryIt graph.Iterator primaryIt graph.Iterator
dir quad.Direction dir quad.Direction
resultIt graph.Iterator resultIt graph.Iterator
@ -57,12 +57,12 @@ type HasA struct {
runstats graph.IteratorStats runstats graph.IteratorStats
} }
// Construct a new HasA iterator, given the triple subiterator, and the triple // Construct a new HasA iterator, given the quad subiterator, and the quad
// direction for which it stands. // direction for which it stands.
func NewHasA(ts graph.TripleStore, subIt graph.Iterator, d quad.Direction) *HasA { func NewHasA(qs graph.QuadStore, subIt graph.Iterator, d quad.Direction) *HasA {
return &HasA{ return &HasA{
uid: NextUID(), uid: NextUID(),
ts: ts, qs: qs,
primaryIt: subIt, primaryIt: subIt,
dir: d, dir: d,
} }
@ -89,7 +89,7 @@ func (it *HasA) Tagger() *graph.Tagger {
} }
func (it *HasA) Clone() graph.Iterator { func (it *HasA) Clone() graph.Iterator {
out := NewHasA(it.ts, it.primaryIt.Clone(), it.dir) out := NewHasA(it.qs, it.primaryIt.Clone(), it.dir)
out.tags.CopyFrom(it) out.tags.CopyFrom(it)
return out return out
} }
@ -98,7 +98,7 @@ func (it *HasA) Clone() graph.Iterator {
func (it *HasA) Direction() quad.Direction { return it.dir } func (it *HasA) Direction() quad.Direction { return it.dir }
// Pass the Optimize() call along to the subiterator. If it becomes Null, // Pass the Optimize() call along to the subiterator. If it becomes Null,
// then the HasA becomes Null (there are no triples that have any directions). // then the HasA becomes Null (there are no quads that have any directions).
func (it *HasA) Optimize() (graph.Iterator, bool) { func (it *HasA) Optimize() (graph.Iterator, bool) {
newPrimary, changed := it.primaryIt.Optimize() newPrimary, changed := it.primaryIt.Optimize()
if changed { if changed {
@ -140,34 +140,34 @@ func (it *HasA) DebugString(indent int) string {
} }
// Check a value against our internal iterator. In order to do this, we must first open a new // Check a value against our internal iterator. In order to do this, we must first open a new
// iterator of "triples that have `val` in our direction", given to us by the triple store, // iterator of "quads that have `val` in our direction", given to us by the quad store,
// and then Next() values out of that iterator and Contains() them against our subiterator. // and then Next() values out of that iterator and Contains() them against our subiterator.
func (it *HasA) Contains(val graph.Value) bool { func (it *HasA) Contains(val graph.Value) bool {
graph.ContainsLogIn(it, val) graph.ContainsLogIn(it, val)
it.runstats.Contains += 1 it.runstats.Contains += 1
if glog.V(4) { if glog.V(4) {
glog.V(4).Infoln("Id is", it.ts.NameOf(val)) glog.V(4).Infoln("Id is", it.qs.NameOf(val))
} }
// TODO(barakmich): Optimize this // TODO(barakmich): Optimize this
if it.resultIt != nil { if it.resultIt != nil {
it.resultIt.Close() it.resultIt.Close()
} }
it.resultIt = it.ts.TripleIterator(it.dir, val) it.resultIt = it.qs.QuadIterator(it.dir, val)
return graph.ContainsLogOut(it, val, it.NextContains()) return graph.ContainsLogOut(it, val, it.NextContains())
} }
// NextContains() is shared code between Contains() and GetNextResult() -- calls next on the // NextContains() is shared code between Contains() and GetNextResult() -- calls next on the
// result iterator (a triple iterator based on the last checked value) and returns true if // result iterator (a quad iterator based on the last checked value) and returns true if
// another match is made. // another match is made.
func (it *HasA) NextContains() bool { func (it *HasA) NextContains() bool {
for graph.Next(it.resultIt) { for graph.Next(it.resultIt) {
it.runstats.ContainsNext += 1 it.runstats.ContainsNext += 1
link := it.resultIt.Result() link := it.resultIt.Result()
if glog.V(4) { if glog.V(4) {
glog.V(4).Infoln("Quad is", it.ts.Quad(link)) glog.V(4).Infoln("Quad is", it.qs.Quad(link))
} }
if it.primaryIt.Contains(link) { if it.primaryIt.Contains(link) {
it.result = it.ts.TripleDirection(link, it.dir) it.result = it.qs.QuadDirection(link, it.dir)
return true return true
} }
} }
@ -192,7 +192,7 @@ func (it *HasA) NextPath() bool {
} }
// Next advances the iterator. This is simpler than Contains. We have a // Next advances the iterator. This is simpler than Contains. We have a
// subiterator we can get a value from, and we can take that resultant triple, // subiterator we can get a value from, and we can take that resultant quad,
// pull our direction out of it, and return that. // pull our direction out of it, and return that.
func (it *HasA) Next() bool { func (it *HasA) Next() bool {
graph.NextLogIn(it) graph.NextLogIn(it)
@ -206,7 +206,7 @@ func (it *HasA) Next() bool {
return graph.NextLogOut(it, 0, false) return graph.NextLogOut(it, 0, false)
} }
tID := it.primaryIt.Result() tID := it.primaryIt.Result()
val := it.ts.TripleDirection(tID, it.dir) val := it.qs.QuadDirection(tID, it.dir)
it.result = val it.result = val
return graph.NextLogOut(it, val, true) return graph.NextLogOut(it, val, true)
} }
@ -217,20 +217,20 @@ func (it *HasA) Result() graph.Value {
// GetStats() returns the statistics on the HasA iterator. This is curious. Next // GetStats() returns the statistics on the HasA iterator. This is curious. Next
// cost is easy, it's an extra call or so on top of the subiterator Next cost. // cost is easy, it's an extra call or so on top of the subiterator Next cost.
// ContainsCost involves going to the graph.TripleStore, iterating out values, and hoping // ContainsCost involves going to the graph.QuadStore, iterating out values, and hoping
// one sticks -- potentially expensive, depending on fanout. Size, however, is // one sticks -- potentially expensive, depending on fanout. Size, however, is
// potentially smaller. we know at worst it's the size of the subiterator, but // potentially smaller. we know at worst it's the size of the subiterator, but
// if there are many repeated values, it could be much smaller in totality. // if there are many repeated values, it could be much smaller in totality.
func (it *HasA) Stats() graph.IteratorStats { func (it *HasA) Stats() graph.IteratorStats {
subitStats := it.primaryIt.Stats() subitStats := it.primaryIt.Stats()
// TODO(barakmich): These should really come from the triplestore itself // TODO(barakmich): These should really come from the quadstore itself
// and be optimized. // and be optimized.
faninFactor := int64(1) faninFactor := int64(1)
fanoutFactor := int64(30) fanoutFactor := int64(30)
nextConstant := int64(2) nextConstant := int64(2)
tripleConstant := int64(1) quadConstant := int64(1)
return graph.IteratorStats{ return graph.IteratorStats{
NextCost: tripleConstant + subitStats.NextCost, NextCost: quadConstant + subitStats.NextCost,
ContainsCost: (fanoutFactor * nextConstant) * subitStats.ContainsCost, ContainsCost: (fanoutFactor * nextConstant) * subitStats.ContainsCost,
Size: faninFactor * subitStats.Size, Size: faninFactor * subitStats.Size,
Next: it.runstats.Next, Next: it.runstats.Next,

20
graph/iterator/linksto_iterator.go
View file

@ -37,13 +37,13 @@ import (
"github.com/google/cayley/quad" "github.com/google/cayley/quad"
) )
// A LinksTo has a reference back to the graph.TripleStore (to create the iterators // A LinksTo has a reference back to the graph.QuadStore (to create the iterators
// for each node) the subiterator, and the direction the iterator comes from. // for each node) the subiterator, and the direction the iterator comes from.
// `next_it` is the tempoarary iterator held per result in `primary_it`. // `next_it` is the tempoarary iterator held per result in `primary_it`.
type LinksTo struct { type LinksTo struct {
uid uint64 uid uint64
tags graph.Tagger tags graph.Tagger
ts graph.TripleStore qs graph.QuadStore
primaryIt graph.Iterator primaryIt graph.Iterator
dir quad.Direction dir quad.Direction
nextIt graph.Iterator nextIt graph.Iterator
@ -53,10 +53,10 @@ type LinksTo struct {
// Construct a new LinksTo iterator around a direction and a subiterator of // Construct a new LinksTo iterator around a direction and a subiterator of
// nodes. // nodes.
func NewLinksTo(ts graph.TripleStore, it graph.Iterator, d quad.Direction) *LinksTo { func NewLinksTo(qs graph.QuadStore, it graph.Iterator, d quad.Direction) *LinksTo {
return &LinksTo{ return &LinksTo{
uid: NextUID(), uid: NextUID(),
ts: ts, qs: qs,
primaryIt: it, primaryIt: it,
dir: d, dir: d,
nextIt: &Null{}, nextIt: &Null{},
@ -80,7 +80,7 @@ func (it *LinksTo) Tagger() *graph.Tagger {
} }
func (it *LinksTo) Clone() graph.Iterator { func (it *LinksTo) Clone() graph.Iterator {
out := NewLinksTo(it.ts, it.primaryIt.Clone(), it.dir) out := NewLinksTo(it.qs, it.primaryIt.Clone(), it.dir)
out.tags.CopyFrom(it) out.tags.CopyFrom(it)
return out return out
} }
@ -120,7 +120,7 @@ func (it *LinksTo) DebugString(indent int) string {
func (it *LinksTo) Contains(val graph.Value) bool { func (it *LinksTo) Contains(val graph.Value) bool {
graph.ContainsLogIn(it, val) graph.ContainsLogIn(it, val)
it.runstats.Contains += 1 it.runstats.Contains += 1
node := it.ts.TripleDirection(val, it.dir) node := it.qs.QuadDirection(val, it.dir)
if it.primaryIt.Contains(node) { if it.primaryIt.Contains(node) {
it.result = val it.result = val
return graph.ContainsLogOut(it, val, true) return graph.ContainsLogOut(it, val, true)
@ -143,10 +143,10 @@ func (it *LinksTo) Optimize() (graph.Iterator, bool) {
return it.primaryIt, true return it.primaryIt, true
} }
} }
// Ask the graph.TripleStore if we can be replaced. Often times, this is a great // Ask the graph.QuadStore if we can be replaced. Often times, this is a great
// optimization opportunity (there's a fixed iterator underneath us, for // optimization opportunity (there's a fixed iterator underneath us, for
// example). // example).
newReplacement, hasOne := it.ts.OptimizeIterator(it) newReplacement, hasOne := it.qs.OptimizeIterator(it)
if hasOne { if hasOne {
it.Close() it.Close()
return newReplacement, true return newReplacement, true
@ -170,7 +170,7 @@ func (it *LinksTo) Next() bool {
return graph.NextLogOut(it, 0, false) return graph.NextLogOut(it, 0, false)
} }
it.nextIt.Close() it.nextIt.Close()
it.nextIt = it.ts.TripleIterator(it.dir, it.primaryIt.Result()) it.nextIt = it.qs.QuadIterator(it.dir, it.primaryIt.Result())
// Recurse -- return the first in the next set. // Recurse -- return the first in the next set.
return it.Next() return it.Next()
@ -197,7 +197,7 @@ func (it *LinksTo) Type() graph.Type { return graph.LinksTo }
// Return a guess as to how big or costly it is to next the iterator. // Return a guess as to how big or costly it is to next the iterator.
func (it *LinksTo) Stats() graph.IteratorStats { func (it *LinksTo) Stats() graph.IteratorStats {
subitStats := it.primaryIt.Stats() subitStats := it.primaryIt.Stats()
// TODO(barakmich): These should really come from the triplestore itself // TODO(barakmich): These should really come from the quadstore itself
fanoutFactor := int64(20) fanoutFactor := int64(20)
checkConstant := int64(1) checkConstant := int64(1)
nextConstant := int64(2) nextConstant := int64(2)

12
graph/iterator/linksto_iterator_test.go
View file

@ -21,23 +21,23 @@ import (
) )
func TestLinksTo(t *testing.T) { func TestLinksTo(t *testing.T) {
ts := &store{ qs := &store{
data: []string{1: "cool"}, data: []string{1: "cool"},
iter: newFixed(), iter: newFixed(),
} }
ts.iter.(*Fixed).Add(2) qs.iter.(*Fixed).Add(2)
fixed := newFixed() fixed := newFixed()
val := ts.ValueOf("cool") val := qs.ValueOf("cool")
if val != 1 { if val != 1 {
t.Fatalf("Failed to return correct value, got:%v expect:1", val) t.Fatalf("Failed to return correct value, got:%v expect:1", val)
} }
fixed.Add(val) fixed.Add(val)
lto := NewLinksTo(ts, fixed, quad.Object) lto := NewLinksTo(qs, fixed, quad.Object)
if !lto.Next() { if !lto.Next() {
t.Error("At least one triple matches the fixed object") t.Error("At least one quad matches the fixed object")
} }
val = lto.Result() val = lto.Result()
if val != 2 { if val != 2 {
t.Errorf("Quad index 2, such as %s, should match %s", ts.Quad(2), ts.Quad(val)) t.Errorf("Quad index 2, such as %s, should match %s", qs.Quad(2), qs.Quad(val))
} }
} }

12
graph/iterator/mock_ts_test.go
View file

@ -14,14 +14,12 @@
package iterator package iterator
// A quickly mocked version of the TripleStore interface, for use in tests.
// Can better used Mock.Called but will fill in as needed.
import ( import (
"github.com/google/cayley/graph" "github.com/google/cayley/graph"
"github.com/google/cayley/quad" "github.com/google/cayley/quad"
) )
// store is a mocked version of the QuadStore interface, for use in tests.
type store struct { type store struct {
data []string data []string
iter graph.Iterator iter graph.Iterator
@ -40,13 +38,13 @@ func (qs *store) ApplyDeltas([]graph.Delta) error { return nil }
func (qs *store) Quad(graph.Value) quad.Quad { return quad.Quad{} } func (qs *store) Quad(graph.Value) quad.Quad { return quad.Quad{} }
func (qs *store) TripleIterator(d quad.Direction, i graph.Value) graph.Iterator { func (qs *store) QuadIterator(d quad.Direction, i graph.Value) graph.Iterator {
return qs.iter return qs.iter
} }
func (qs *store) NodesAllIterator() graph.Iterator { return &Null{} } func (qs *store) NodesAllIterator() graph.Iterator { return &Null{} }
func (qs *store) TriplesAllIterator() graph.Iterator { return &Null{} } func (qs *store) QuadsAllIterator() graph.Iterator { return &Null{} }
func (qs *store) NameOf(v graph.Value) string { func (qs *store) NameOf(v graph.Value) string {
i := v.(int) i := v.(int)
@ -72,6 +70,6 @@ func (qs *store) FixedIterator() graph.FixedIterator {
func (qs *store) Close() {} func (qs *store) Close() {}
func (qs *store) TripleDirection(graph.Value, quad.Direction) graph.Value { return 0 } func (qs *store) QuadDirection(graph.Value, quad.Direction) graph.Value { return 0 }
func (qs *store) RemoveTriple(t quad.Quad) {} func (qs *store) RemoveQuad(t quad.Quad) {}

98
graph/iterator/query_shape.go
View file

@ -37,47 +37,47 @@ type Link struct {
type queryShape struct { type queryShape struct {
nodes []Node nodes []Node
links []Link links []Link
ts graph.TripleStore qs graph.QuadStore
nodeId int nodeId int
hasaIds []int hasaIds []int
hasaDirs []quad.Direction hasaDirs []quad.Direction
} }
func OutputQueryShapeForIterator(it graph.Iterator, ts graph.TripleStore, outputMap map[string]interface{}) { func OutputQueryShapeForIterator(it graph.Iterator, qs graph.QuadStore, outputMap map[string]interface{}) {
qs := &queryShape{ s := &queryShape{
ts: ts, qs: qs,
nodeId: 1, nodeId: 1,
} }
node := qs.MakeNode(it.Clone()) node := s.MakeNode(it.Clone())
qs.AddNode(node) s.AddNode(node)
outputMap["nodes"] = qs.nodes outputMap["nodes"] = s.nodes
outputMap["links"] = qs.links outputMap["links"] = s.links
} }
func (qs *queryShape) AddNode(n *Node) { func (s *queryShape) AddNode(n *Node) {
qs.nodes = append(qs.nodes, *n) s.nodes = append(s.nodes, *n)
} }
func (qs *queryShape) AddLink(l *Link) { func (s *queryShape) AddLink(l *Link) {
qs.links = append(qs.links, *l) s.links = append(s.links, *l)
} }
func (qs *queryShape) LastHasa() (int, quad.Direction) { func (s *queryShape) LastHasa() (int, quad.Direction) {
return qs.hasaIds[len(qs.hasaIds)-1], qs.hasaDirs[len(qs.hasaDirs)-1] return s.hasaIds[len(s.hasaIds)-1], s.hasaDirs[len(s.hasaDirs)-1]
} }
func (qs *queryShape) PushHasa(i int, d quad.Direction) { func (s *queryShape) PushHasa(i int, d quad.Direction) {
qs.hasaIds = append(qs.hasaIds, i) s.hasaIds = append(s.hasaIds, i)
qs.hasaDirs = append(qs.hasaDirs, d) s.hasaDirs = append(s.hasaDirs, d)
} }
func (qs *queryShape) RemoveHasa() { func (s *queryShape) RemoveHasa() {
qs.hasaIds = qs.hasaIds[:len(qs.hasaIds)-1] s.hasaIds = s.hasaIds[:len(s.hasaIds)-1]
qs.hasaDirs = qs.hasaDirs[:len(qs.hasaDirs)-1] s.hasaDirs = s.hasaDirs[:len(s.hasaDirs)-1]
} }
func (qs *queryShape) StealNode(left *Node, right *Node) { func (s *queryShape) StealNode(left *Node, right *Node) {
for _, v := range right.Values { for _, v := range right.Values {
left.Values = append(left.Values, v) left.Values = append(left.Values, v)
} }
@ -86,7 +86,7 @@ func (qs *queryShape) StealNode(left *Node, right *Node) {
} }
left.IsLinkNode = left.IsLinkNode || right.IsLinkNode left.IsLinkNode = left.IsLinkNode || right.IsLinkNode
left.IsFixed = left.IsFixed || right.IsFixed left.IsFixed = left.IsFixed || right.IsFixed
for i, link := range qs.links { for i, link := range s.links {
rewrite := false rewrite := false
if link.LinkNode == right.Id { if link.LinkNode == right.Id {
link.LinkNode = left.Id link.LinkNode = left.Id
@ -101,13 +101,13 @@ func (qs *queryShape) StealNode(left *Node, right *Node) {
rewrite = true rewrite = true
} }
if rewrite { if rewrite {
qs.links = append(append(qs.links[:i], qs.links[i+1:]...), link) s.links = append(append(s.links[:i], s.links[i+1:]...), link)
} }
} }
} }
func (qs *queryShape) MakeNode(it graph.Iterator) *Node { func (s *queryShape) MakeNode(it graph.Iterator) *Node {
n := Node{Id: qs.nodeId} n := Node{Id: s.nodeId}
for _, tag := range it.Tagger().Tags() { for _, tag := range it.Tagger().Tags() {
n.Tags = append(n.Tags, tag) n.Tags = append(n.Tags, tag)
} }
@ -118,56 +118,56 @@ func (qs *queryShape) MakeNode(it graph.Iterator) *Node {
switch it.Type() { switch it.Type() {
case graph.And: case graph.And:
for _, sub := range it.SubIterators() { for _, sub := range it.SubIterators() {
qs.nodeId++ s.nodeId++
newNode := qs.MakeNode(sub) newNode := s.MakeNode(sub)
if sub.Type() != graph.Or { if sub.Type() != graph.Or {
qs.StealNode(&n, newNode) s.StealNode(&n, newNode)
} else { } else {
qs.AddNode(newNode) s.AddNode(newNode)
qs.AddLink(&Link{n.Id, newNode.Id, 0, 0}) s.AddLink(&Link{n.Id, newNode.Id, 0, 0})
} }
} }
case graph.Fixed: case graph.Fixed:
n.IsFixed = true n.IsFixed = true
for graph.Next(it) { for graph.Next(it) {
n.Values = append(n.Values, qs.ts.NameOf(it.Result())) n.Values = append(n.Values, s.qs.NameOf(it.Result()))
} }
case graph.HasA: case graph.HasA:
hasa := it.(*HasA) hasa := it.(*HasA)
qs.PushHasa(n.Id, hasa.dir) s.PushHasa(n.Id, hasa.dir)
qs.nodeId++ s.nodeId++
newNode := qs.MakeNode(hasa.primaryIt) newNode := s.MakeNode(hasa.primaryIt)
qs.AddNode(newNode) s.AddNode(newNode)
qs.RemoveHasa() s.RemoveHasa()
case graph.Or: case graph.Or:
for _, sub := range it.SubIterators() { for _, sub := range it.SubIterators() {
qs.nodeId++ s.nodeId++
newNode := qs.MakeNode(sub) newNode := s.MakeNode(sub)
if sub.Type() == graph.Or { if sub.Type() == graph.Or {
qs.StealNode(&n, newNode) s.StealNode(&n, newNode)
} else { } else {
qs.AddNode(newNode) s.AddNode(newNode)
qs.AddLink(&Link{n.Id, newNode.Id, 0, 0}) s.AddLink(&Link{n.Id, newNode.Id, 0, 0})
} }
} }
case graph.LinksTo: case graph.LinksTo:
n.IsLinkNode = true n.IsLinkNode = true
lto := it.(*LinksTo) lto := it.(*LinksTo)
qs.nodeId++ s.nodeId++
newNode := qs.MakeNode(lto.primaryIt) newNode := s.MakeNode(lto.primaryIt)
hasaID, hasaDir := qs.LastHasa() hasaID, hasaDir := s.LastHasa()
if (hasaDir == quad.Subject && lto.dir == quad.Object) || if (hasaDir == quad.Subject && lto.dir == quad.Object) ||
(hasaDir == quad.Object && lto.dir == quad.Subject) { (hasaDir == quad.Object && lto.dir == quad.Subject) {
qs.AddNode(newNode) s.AddNode(newNode)
if hasaDir == quad.Subject { if hasaDir == quad.Subject {
qs.AddLink(&Link{hasaID, newNode.Id, 0, n.Id}) s.AddLink(&Link{hasaID, newNode.Id, 0, n.Id})
} else { } else {
qs.AddLink(&Link{newNode.Id, hasaID, 0, n.Id}) s.AddLink(&Link{newNode.Id, hasaID, 0, n.Id})
} }
} else if lto.primaryIt.Type() == graph.Fixed { } else if lto.primaryIt.Type() == graph.Fixed {
qs.StealNode(&n, newNode) s.StealNode(&n, newNode)
} else { } else {
qs.AddNode(newNode) s.AddNode(newNode)
} }
case graph.Optional: case graph.Optional:
// Unsupported, for the moment // Unsupported, for the moment

36
graph/iterator/query_shape_test.go
View file

@ -22,23 +22,23 @@ import (
"github.com/google/cayley/quad" "github.com/google/cayley/quad"
) )
func hasaWithTag(ts graph.TripleStore, tag string, target string) *HasA { func hasaWithTag(qs graph.QuadStore, tag string, target string) *HasA {
and := NewAnd() and := NewAnd()
obj := ts.FixedIterator() obj := qs.FixedIterator()
obj.Add(ts.ValueOf(target)) obj.Add(qs.ValueOf(target))
obj.Tagger().Add(tag) obj.Tagger().Add(tag)
and.AddSubIterator(NewLinksTo(ts, obj, quad.Object)) and.AddSubIterator(NewLinksTo(qs, obj, quad.Object))
pred := ts.FixedIterator() pred := qs.FixedIterator()
pred.Add(ts.ValueOf("status")) pred.Add(qs.ValueOf("status"))
and.AddSubIterator(NewLinksTo(ts, pred, quad.Predicate)) and.AddSubIterator(NewLinksTo(qs, pred, quad.Predicate))
return NewHasA(ts, and, quad.Subject) return NewHasA(qs, and, quad.Subject)
} }
func TestQueryShape(t *testing.T) { func TestQueryShape(t *testing.T) {
ts := &store{ qs := &store{
data: []string{ data: []string{
1: "cool", 1: "cool",
2: "status", 2: "status",
@ -48,11 +48,11 @@ func TestQueryShape(t *testing.T) {
} }
// Given a single linkage iterator's shape. // Given a single linkage iterator's shape.
hasa := hasaWithTag(ts, "tag", "cool") hasa := hasaWithTag(qs, "tag", "cool")
hasa.Tagger().Add("top") hasa.Tagger().Add("top")
shape := make(map[string]interface{}) shape := make(map[string]interface{})
OutputQueryShapeForIterator(hasa, ts, shape) OutputQueryShapeForIterator(hasa, qs, shape)
nodes := shape["nodes"].([]Node) nodes := shape["nodes"].([]Node)
if len(nodes) != 3 { if len(nodes) != 3 {
@ -93,23 +93,23 @@ func TestQueryShape(t *testing.T) {
// Given a name-of-an-and-iterator's shape. // Given a name-of-an-and-iterator's shape.
andInternal := NewAnd() andInternal := NewAnd()
hasa1 := hasaWithTag(ts, "tag1", "cool") hasa1 := hasaWithTag(qs, "tag1", "cool")
hasa1.Tagger().Add("hasa1") hasa1.Tagger().Add("hasa1")
andInternal.AddSubIterator(hasa1) andInternal.AddSubIterator(hasa1)
hasa2 := hasaWithTag(ts, "tag2", "fun") hasa2 := hasaWithTag(qs, "tag2", "fun")
hasa2.Tagger().Add("hasa2") hasa2.Tagger().Add("hasa2")
andInternal.AddSubIterator(hasa2) andInternal.AddSubIterator(hasa2)
pred := ts.FixedIterator() pred := qs.FixedIterator()
pred.Add(ts.ValueOf("name")) pred.Add(qs.ValueOf("name"))
and := NewAnd() and := NewAnd()
and.AddSubIterator(NewLinksTo(ts, andInternal, quad.Subject)) and.AddSubIterator(NewLinksTo(qs, andInternal, quad.Subject))
and.AddSubIterator(NewLinksTo(ts, pred, quad.Predicate)) and.AddSubIterator(NewLinksTo(qs, pred, quad.Predicate))
shape = make(map[string]interface{}) shape = make(map[string]interface{})
OutputQueryShapeForIterator(NewHasA(ts, and, quad.Object), ts, shape) OutputQueryShapeForIterator(NewHasA(qs, and, quad.Object), qs, shape)
links = shape["links"].([]Link) links = shape["links"].([]Link)
if len(links) != 3 { if len(links) != 3 {

10
graph/iterator/value_comparison_iterator.go
View file

@ -51,17 +51,17 @@ type Comparison struct {
subIt graph.Iterator subIt graph.Iterator
op Operator op Operator
val interface{} val interface{}
ts graph.TripleStore qs graph.QuadStore
result graph.Value result graph.Value
} }
func NewComparison(sub graph.Iterator, op Operator, val interface{}, ts graph.TripleStore) *Comparison { func NewComparison(sub graph.Iterator, op Operator, val interface{}, qs graph.QuadStore) *Comparison {
return &Comparison{ return &Comparison{
uid: NextUID(), uid: NextUID(),
subIt: sub, subIt: sub,
op: op, op: op,
val: val, val: val,
ts: ts, qs: qs,
} }
} }
@ -73,7 +73,7 @@ func (it *Comparison) UID() uint64 {
// and our operator, determine whether or not we meet the requirement. // and our operator, determine whether or not we meet the requirement.
func (it *Comparison) doComparison(val graph.Value) bool { func (it *Comparison) doComparison(val graph.Value) bool {
//TODO(barakmich): Implement string comparison. //TODO(barakmich): Implement string comparison.
nodeStr := it.ts.NameOf(val) nodeStr := it.qs.NameOf(val)
switch cVal := it.val.(type) { switch cVal := it.val.(type) {
case int: case int:
cInt := int64(cVal) cInt := int64(cVal)
@ -122,7 +122,7 @@ func (it *Comparison) Tagger() *graph.Tagger {
} }
func (it *Comparison) Clone() graph.Iterator { func (it *Comparison) Clone() graph.Iterator {
out := NewComparison(it.subIt.Clone(), it.op, it.val, it.ts) out := NewComparison(it.subIt.Clone(), it.op, it.val, it.qs)
out.tags.CopyFrom(it) out.tags.CopyFrom(it)
return out return out
} }

6
graph/iterator/value_comparison_iterator_test.go
View file

@ -65,12 +65,12 @@ var comparisonTests = []struct {
func TestValueComparison(t *testing.T) { func TestValueComparison(t *testing.T) {
for _, test := range comparisonTests { for _, test := range comparisonTests {
ts := simpleStore qs := simpleStore
vc := NewComparison(simpleFixedIterator(), test.operator, test.operand, ts) vc := NewComparison(simpleFixedIterator(), test.operator, test.operand, qs)
var got []string var got []string
for vc.Next() { for vc.Next() {
got = append(got, ts.NameOf(vc.Result())) got = append(got, qs.NameOf(vc.Result()))
} }
if !reflect.DeepEqual(got, test.expect) { if !reflect.DeepEqual(got, test.expect) {
t.Errorf("Failed to show %s, got:%q expect:%q", test.message, got, test.expect) t.Errorf("Failed to show %s, got:%q expect:%q", test.message, got, test.expect)

14
graph/leveldb/all_iterator.go
View file

@ -34,12 +34,12 @@ type AllIterator struct {
dir quad.Direction dir quad.Direction
open bool open bool
iter ldbit.Iterator iter ldbit.Iterator
ts *TripleStore qs *QuadStore
ro *opt.ReadOptions ro *opt.ReadOptions
result graph.Value result graph.Value
} }
func NewAllIterator(prefix string, d quad.Direction, ts *TripleStore) *AllIterator { func NewAllIterator(prefix string, d quad.Direction, qs *QuadStore) *AllIterator {
opts := &opt.ReadOptions{ opts := &opt.ReadOptions{
DontFillCache: true, DontFillCache: true,
} }
@ -47,11 +47,11 @@ func NewAllIterator(prefix string, d quad.Direction, ts *TripleStore) *AllIterat
it := AllIterator{ it := AllIterator{
uid: iterator.NextUID(), uid: iterator.NextUID(),
ro: opts, ro: opts,
iter: ts.db.NewIterator(nil, opts), iter: qs.db.NewIterator(nil, opts),
prefix: []byte(prefix), prefix: []byte(prefix),
dir: d, dir: d,
open: true, open: true,
ts: ts, qs: qs,
} }
it.iter.Seek(it.prefix) it.iter.Seek(it.prefix)
@ -71,7 +71,7 @@ func (it *AllIterator) UID() uint64 {
func (it *AllIterator) Reset() { func (it *AllIterator) Reset() {
if !it.open { if !it.open {
it.iter = it.ts.db.NewIterator(nil, it.ro) it.iter = it.qs.db.NewIterator(nil, it.ro)
it.open = true it.open = true
} }
it.iter.Seek(it.prefix) it.iter.Seek(it.prefix)
@ -96,7 +96,7 @@ func (it *AllIterator) TagResults(dst map[string]graph.Value) {
} }
func (it *AllIterator) Clone() graph.Iterator { func (it *AllIterator) Clone() graph.Iterator {
out := NewAllIterator(string(it.prefix), it.dir, it.ts) out := NewAllIterator(string(it.prefix), it.dir, it.qs)
out.tags.CopyFrom(it) out.tags.CopyFrom(it)
return out return out
} }
@ -151,7 +151,7 @@ func (it *AllIterator) Close() {
} }
func (it *AllIterator) Size() (int64, bool) { func (it *AllIterator) Size() (int64, bool) {
size, err := it.ts.SizeOfPrefix(it.prefix) size, err := it.qs.SizeOfPrefix(it.prefix)
if err == nil { if err == nil {
return size, false return size, false
} }

8
graph/leveldb/iterator.go
View file

@ -37,13 +37,13 @@ type Iterator struct {
dir quad.Direction dir quad.Direction
open bool open bool
iter ldbit.Iterator iter ldbit.Iterator
qs *TripleStore qs *QuadStore
ro *opt.ReadOptions ro *opt.ReadOptions
originalPrefix string originalPrefix string
result graph.Value result graph.Value
} }
func NewIterator(prefix string, d quad.Direction, value graph.Value, qs *TripleStore) *Iterator { func NewIterator(prefix string, d quad.Direction, value graph.Value, qs *QuadStore) *Iterator {
vb := value.(Token) vb := value.(Token)
p := make([]byte, 0, 2+hashSize) p := make([]byte, 0, 2+hashSize)
p = append(p, []byte(prefix)...) p = append(p, []byte(prefix)...)
@ -173,7 +173,7 @@ func (it *Iterator) SubIterators() []graph.Iterator {
return nil return nil
} }
func PositionOf(prefix []byte, d quad.Direction, qs *TripleStore) int { func PositionOf(prefix []byte, d quad.Direction, qs *QuadStore) int {
if bytes.Equal(prefix, []byte("sp")) { if bytes.Equal(prefix, []byte("sp")) {
switch d { switch d {
case quad.Subject: case quad.Subject:
@ -232,7 +232,7 @@ func (it *Iterator) Contains(v graph.Value) bool {
} }
offset := PositionOf(val[0:2], it.dir, it.qs) offset := PositionOf(val[0:2], it.dir, it.qs)
if bytes.HasPrefix(val[offset:], it.checkId[1:]) { if bytes.HasPrefix(val[offset:], it.checkId[1:]) {
// You may ask, why don't we check to see if it's a valid (not deleted) triple // You may ask, why don't we check to see if it's a valid (not deleted) quad
// again? // again?
// //
// We've already done that -- in order to get the graph.Value token in the // We've already done that -- in order to get the graph.Value token in the

104
graph/leveldb/leveldb_test.go
View file

@ -27,8 +27,8 @@ import (
"github.com/google/cayley/writer" "github.com/google/cayley/writer"
) )
func makeTripleSet() []quad.Quad { func makeQuadSet() []quad.Quad {
tripleSet := []quad.Quad{ quadSet := []quad.Quad{
{"A", "follows", "B", ""}, {"A", "follows", "B", ""},
{"C", "follows", "B", ""}, {"C", "follows", "B", ""},
{"C", "follows", "D", ""}, {"C", "follows", "D", ""},
@ -41,10 +41,10 @@ func makeTripleSet() []quad.Quad {
{"D", "status", "cool", "status_graph"}, {"D", "status", "cool", "status_graph"},
{"G", "status", "cool", "status_graph"}, {"G", "status", "cool", "status_graph"},
} }
return tripleSet return quadSet
} }
func iteratedTriples(qs graph.TripleStore, it graph.Iterator) []quad.Quad { func iteratedQuads(qs graph.QuadStore, it graph.Iterator) []quad.Quad {
var res ordered var res ordered
for graph.Next(it) { for graph.Next(it) {
res = append(res, qs.Quad(it.Result())) res = append(res, qs.Quad(it.Result()))
@ -80,7 +80,7 @@ func (o ordered) Less(i, j int) bool {
} }
func (o ordered) Swap(i, j int) { o[i], o[j] = o[j], o[i] } func (o ordered) Swap(i, j int) { o[i], o[j] = o[j], o[i] }
func iteratedNames(qs graph.TripleStore, it graph.Iterator) []string { func iteratedNames(qs graph.QuadStore, it graph.Iterator) []string {
var res []string var res []string
for graph.Next(it) { for graph.Next(it) {
res = append(res, qs.NameOf(it.Result())) res = append(res, qs.NameOf(it.Result()))
@ -101,9 +101,9 @@ func TestCreateDatabase(t *testing.T) {
t.Fatal("Failed to create LevelDB database.") t.Fatal("Failed to create LevelDB database.")
} }
qs, err := newTripleStore(tmpDir, nil) qs, err := newQuadStore(tmpDir, nil)
if qs == nil || err != nil { if qs == nil || err != nil {
t.Error("Failed to create leveldb TripleStore.") t.Error("Failed to create leveldb QuadStore.")
} }
if s := qs.Size(); s != 0 { if s := qs.Size(); s != 0 {
t.Errorf("Unexpected size, got:%d expected:0", s) t.Errorf("Unexpected size, got:%d expected:0", s)
@ -131,9 +131,9 @@ func TestLoadDatabase(t *testing.T) {
t.Fatal("Failed to create LevelDB database.") t.Fatal("Failed to create LevelDB database.")
} }
qs, err := newTripleStore(tmpDir, nil) qs, err := newQuadStore(tmpDir, nil)
if qs == nil || err != nil { if qs == nil || err != nil {
t.Error("Failed to create leveldb TripleStore.") t.Error("Failed to create leveldb QuadStore.")
} }
w, _ := writer.NewSingleReplication(qs, nil) w, _ := writer.NewSingleReplication(qs, nil)
@ -144,7 +144,7 @@ func TestLoadDatabase(t *testing.T) {
} }
} }
if s := qs.Size(); s != 1 { if s := qs.Size(); s != 1 {
t.Errorf("Unexpected triplestore size, got:%d expect:1", s) t.Errorf("Unexpected quadstore size, got:%d expect:1", s)
} }
qs.Close() qs.Close()
@ -152,31 +152,31 @@ func TestLoadDatabase(t *testing.T) {
if err != nil { if err != nil {
t.Fatal("Failed to create LevelDB database.") t.Fatal("Failed to create LevelDB database.")
} }
qs, err = newTripleStore(tmpDir, nil) qs, err = newQuadStore(tmpDir, nil)
if qs == nil || err != nil { if qs == nil || err != nil {
t.Error("Failed to create leveldb TripleStore.") t.Error("Failed to create leveldb QuadStore.")
} }
w, _ = writer.NewSingleReplication(qs, nil) w, _ = writer.NewSingleReplication(qs, nil)
ts2, didConvert := qs.(*TripleStore) ts2, didConvert := qs.(*QuadStore)
if !didConvert { if !didConvert {
t.Errorf("Could not convert from generic to LevelDB TripleStore") t.Errorf("Could not convert from generic to LevelDB QuadStore")
} }
w.AddQuadSet(makeTripleSet()) w.AddQuadSet(makeQuadSet())
if s := qs.Size(); s != 11 { if s := qs.Size(); s != 11 {
t.Errorf("Unexpected triplestore size, got:%d expect:11", s) t.Errorf("Unexpected quadstore size, got:%d expect:11", s)
} }
if s := ts2.SizeOf(qs.ValueOf("B")); s != 5 { if s := ts2.SizeOf(qs.ValueOf("B")); s != 5 {
t.Errorf("Unexpected triplestore size, got:%d expect:5", s) t.Errorf("Unexpected quadstore size, got:%d expect:5", s)
} }
w.RemoveQuad(quad.Quad{"A", "follows", "B", ""}) w.RemoveQuad(quad.Quad{"A", "follows", "B", ""})
if s := qs.Size(); s != 10 { if s := qs.Size(); s != 10 {
t.Errorf("Unexpected triplestore size after RemoveTriple, got:%d expect:10", s) t.Errorf("Unexpected quadstore size after RemoveQuad, got:%d expect:10", s)
} }
if s := ts2.SizeOf(qs.ValueOf("B")); s != 4 { if s := ts2.SizeOf(qs.ValueOf("B")); s != 4 {
t.Errorf("Unexpected triplestore size, got:%d expect:4", s) t.Errorf("Unexpected quadstore size, got:%d expect:4", s)
} }
qs.Close() qs.Close()
@ -195,13 +195,13 @@ func TestIterator(t *testing.T) {
t.Fatal("Failed to create LevelDB database.") t.Fatal("Failed to create LevelDB database.")
} }
qs, err := newTripleStore(tmpDir, nil) qs, err := newQuadStore(tmpDir, nil)
if qs == nil || err != nil { if qs == nil || err != nil {
t.Error("Failed to create leveldb TripleStore.") t.Error("Failed to create leveldb QuadStore.")
} }
w, _ := writer.NewSingleReplication(qs, nil) w, _ := writer.NewSingleReplication(qs, nil)
w.AddQuadSet(makeTripleSet()) w.AddQuadSet(makeQuadSet())
var it graph.Iterator var it graph.Iterator
it = qs.NodesAllIterator() it = qs.NodesAllIterator()
@ -262,19 +262,19 @@ func TestIterator(t *testing.T) {
*/ */
it.Reset() it.Reset()
it = qs.TriplesAllIterator() it = qs.QuadsAllIterator()
graph.Next(it) graph.Next(it)
triple := qs.Quad(it.Result()) q := qs.Quad(it.Result())
set := makeTripleSet() set := makeQuadSet()
var ok bool var ok bool
for _, t := range set { for _, t := range set {
if t.String() == triple.String() { if t.String() == q.String() {
ok = true ok = true
break break
} }
} }
if !ok { if !ok {
t.Errorf("Failed to find %q during iteration, got:%q", triple, set) t.Errorf("Failed to find %q during iteration, got:%q", q, set)
} }
qs.Close() qs.Close()
@ -290,14 +290,14 @@ func TestSetIterator(t *testing.T) {
t.Fatalf("Failed to create working directory") t.Fatalf("Failed to create working directory")
} }
qs, err := newTripleStore(tmpDir, nil) qs, err := newQuadStore(tmpDir, nil)
if qs == nil || err != nil { if qs == nil || err != nil {
t.Error("Failed to create leveldb TripleStore.") t.Error("Failed to create leveldb QuadStore.")
} }
defer qs.Close() defer qs.Close()
w, _ := writer.NewSingleReplication(qs, nil) w, _ := writer.NewSingleReplication(qs, nil)
w.AddQuadSet(makeTripleSet()) w.AddQuadSet(makeQuadSet())
expect := []quad.Quad{ expect := []quad.Quad{
{"C", "follows", "B", ""}, {"C", "follows", "B", ""},
@ -306,46 +306,46 @@ func TestSetIterator(t *testing.T) {
sort.Sort(ordered(expect)) sort.Sort(ordered(expect))
// Subject iterator. // Subject iterator.
it := qs.TripleIterator(quad.Subject, qs.ValueOf("C")) it := qs.QuadIterator(quad.Subject, qs.ValueOf("C"))
if got := iteratedTriples(qs, it); !reflect.DeepEqual(got, expect) { if got := iteratedQuads(qs, it); !reflect.DeepEqual(got, expect) {
t.Errorf("Failed to get expected results, got:%v expect:%v", got, expect) t.Errorf("Failed to get expected results, got:%v expect:%v", got, expect)
} }
it.Reset() it.Reset()
and := iterator.NewAnd() and := iterator.NewAnd()
and.AddSubIterator(qs.TriplesAllIterator()) and.AddSubIterator(qs.QuadsAllIterator())
and.AddSubIterator(it) and.AddSubIterator(it)
if got := iteratedTriples(qs, and); !reflect.DeepEqual(got, expect) { if got := iteratedQuads(qs, and); !reflect.DeepEqual(got, expect) {
t.Errorf("Failed to get confirm expected results, got:%v expect:%v", got, expect) t.Errorf("Failed to get confirm expected results, got:%v expect:%v", got, expect)
} }
// Object iterator. // Object iterator.
it = qs.TripleIterator(quad.Object, qs.ValueOf("F")) it = qs.QuadIterator(quad.Object, qs.ValueOf("F"))
expect = []quad.Quad{ expect = []quad.Quad{
{"B", "follows", "F", ""}, {"B", "follows", "F", ""},
{"E", "follows", "F", ""}, {"E", "follows", "F", ""},
} }
sort.Sort(ordered(expect)) sort.Sort(ordered(expect))
if got := iteratedTriples(qs, it); !reflect.DeepEqual(got, expect) { if got := iteratedQuads(qs, it); !reflect.DeepEqual(got, expect) {
t.Errorf("Failed to get expected results, got:%v expect:%v", got, expect) t.Errorf("Failed to get expected results, got:%v expect:%v", got, expect)
} }
and = iterator.NewAnd() and = iterator.NewAnd()
and.AddSubIterator(qs.TripleIterator(quad.Subject, qs.ValueOf("B"))) and.AddSubIterator(qs.QuadIterator(quad.Subject, qs.ValueOf("B")))
and.AddSubIterator(it) and.AddSubIterator(it)
expect = []quad.Quad{ expect = []quad.Quad{
{"B", "follows", "F", ""}, {"B", "follows", "F", ""},
} }
if got := iteratedTriples(qs, and); !reflect.DeepEqual(got, expect) { if got := iteratedQuads(qs, and); !reflect.DeepEqual(got, expect) {
t.Errorf("Failed to get confirm expected results, got:%v expect:%v", got, expect) t.Errorf("Failed to get confirm expected results, got:%v expect:%v", got, expect)
} }
// Predicate iterator. // Predicate iterator.
it = qs.TripleIterator(quad.Predicate, qs.ValueOf("status")) it = qs.QuadIterator(quad.Predicate, qs.ValueOf("status"))
expect = []quad.Quad{ expect = []quad.Quad{
{"B", "status", "cool", "status_graph"}, {"B", "status", "cool", "status_graph"},
@ -353,12 +353,12 @@ func TestSetIterator(t *testing.T) {
{"G", "status", "cool", "status_graph"}, {"G", "status", "cool", "status_graph"},
} }
sort.Sort(ordered(expect)) sort.Sort(ordered(expect))
if got := iteratedTriples(qs, it); !reflect.DeepEqual(got, expect) { if got := iteratedQuads(qs, it); !reflect.DeepEqual(got, expect) {
t.Errorf("Failed to get expected results from predicate iterator, got:%v expect:%v", got, expect) t.Errorf("Failed to get expected results from predicate iterator, got:%v expect:%v", got, expect)
} }
// Label iterator. // Label iterator.
it = qs.TripleIterator(quad.Label, qs.ValueOf("status_graph")) it = qs.QuadIterator(quad.Label, qs.ValueOf("status_graph"))
expect = []quad.Quad{ expect = []quad.Quad{
{"B", "status", "cool", "status_graph"}, {"B", "status", "cool", "status_graph"},
@ -366,20 +366,20 @@ func TestSetIterator(t *testing.T) {
{"G", "status", "cool", "status_graph"}, {"G", "status", "cool", "status_graph"},
} }
sort.Sort(ordered(expect)) sort.Sort(ordered(expect))
if got := iteratedTriples(qs, it); !reflect.DeepEqual(got, expect) { if got := iteratedQuads(qs, it); !reflect.DeepEqual(got, expect) {
t.Errorf("Failed to get expected results from predicate iterator, got:%v expect:%v", got, expect) t.Errorf("Failed to get expected results from predicate iterator, got:%v expect:%v", got, expect)
} }
it.Reset() it.Reset()
// Order is important // Order is important
and = iterator.NewAnd() and = iterator.NewAnd()
and.AddSubIterator(qs.TripleIterator(quad.Subject, qs.ValueOf("B"))) and.AddSubIterator(qs.QuadIterator(quad.Subject, qs.ValueOf("B")))
and.AddSubIterator(it) and.AddSubIterator(it)
expect = []quad.Quad{ expect = []quad.Quad{
{"B", "status", "cool", "status_graph"}, {"B", "status", "cool", "status_graph"},
} }
if got := iteratedTriples(qs, and); !reflect.DeepEqual(got, expect) { if got := iteratedQuads(qs, and); !reflect.DeepEqual(got, expect) {
t.Errorf("Failed to get confirm expected results, got:%v expect:%v", got, expect) t.Errorf("Failed to get confirm expected results, got:%v expect:%v", got, expect)
} }
it.Reset() it.Reset()
@ -387,12 +387,12 @@ func TestSetIterator(t *testing.T) {
// Order is important // Order is important
and = iterator.NewAnd() and = iterator.NewAnd()
and.AddSubIterator(it) and.AddSubIterator(it)
and.AddSubIterator(qs.TripleIterator(quad.Subject, qs.ValueOf("B"))) and.AddSubIterator(qs.QuadIterator(quad.Subject, qs.ValueOf("B")))
expect = []quad.Quad{ expect = []quad.Quad{
{"B", "status", "cool", "status_graph"}, {"B", "status", "cool", "status_graph"},
} }
if got := iteratedTriples(qs, and); !reflect.DeepEqual(got, expect) { if got := iteratedQuads(qs, and); !reflect.DeepEqual(got, expect) {
t.Errorf("Failed to get confirm expected results, got:%v expect:%v", got, expect) t.Errorf("Failed to get confirm expected results, got:%v expect:%v", got, expect)
} }
} }
@ -405,13 +405,13 @@ func TestOptimize(t *testing.T) {
if err != nil { if err != nil {
t.Fatalf("Failed to create working directory") t.Fatalf("Failed to create working directory")
} }
qs, err := newTripleStore(tmpDir, nil) qs, err := newQuadStore(tmpDir, nil)
if qs == nil || err != nil { if qs == nil || err != nil {
t.Error("Failed to create leveldb TripleStore.") t.Error("Failed to create leveldb QuadStore.")
} }
w, _ := writer.NewSingleReplication(qs, nil) w, _ := writer.NewSingleReplication(qs, nil)
w.AddQuadSet(makeTripleSet()) w.AddQuadSet(makeQuadSet())
// With an linksto-fixed pair // With an linksto-fixed pair
fixed := qs.FixedIterator() fixed := qs.FixedIterator()
@ -428,9 +428,9 @@ func TestOptimize(t *testing.T) {
t.Errorf("Optimized iterator type does not match original, got:%v expect:%v", newIt.Type(), Type()) t.Errorf("Optimized iterator type does not match original, got:%v expect:%v", newIt.Type(), Type())
} }
newTriples := iteratedTriples(qs, newIt) newQuads := iteratedQuads(qs, newIt)
oldTriples := iteratedTriples(qs, oldIt) oldQuads := iteratedQuads(qs, oldIt)
if !reflect.DeepEqual(newTriples, oldTriples) { if !reflect.DeepEqual(newQuads, oldQuads) {
t.Errorf("Optimized iteration does not match original") t.Errorf("Optimized iteration does not match original")
} }

104
graph/leveldb/triplestore.go → graph/leveldb/quadstore.go
View file

@ -36,7 +36,7 @@ import (
) )
func init() { func init() {
graph.RegisterTripleStore("leveldb", true, newTripleStore, createNewLevelDB) graph.RegisterQuadStore("leveldb", true, newQuadStore, createNewLevelDB)
} }
const ( const (
@ -57,7 +57,7 @@ func (t Token) Key() interface{} {
return string(t) return string(t)
} }
type TripleStore struct { type QuadStore struct {
dbOpts *opt.Options dbOpts *opt.Options
db *leveldb.DB db *leveldb.DB
path string path string
@ -76,7 +76,7 @@ func createNewLevelDB(path string, _ graph.Options) error {
return err return err
} }
defer db.Close() defer db.Close()
qs := &TripleStore{} qs := &QuadStore{}
qs.db = db qs.db = db
qs.writeopts = &opt.WriteOptions{ qs.writeopts = &opt.WriteOptions{
Sync: true, Sync: true,
@ -85,8 +85,8 @@ func createNewLevelDB(path string, _ graph.Options) error {
return nil return nil
} }
func newTripleStore(path string, options graph.Options) (graph.TripleStore, error) { func newQuadStore(path string, options graph.Options) (graph.QuadStore, error) {
var qs TripleStore var qs QuadStore
var err error var err error
qs.path = path qs.path = path
cache_size := DefaultCacheSize cache_size := DefaultCacheSize
@ -121,7 +121,7 @@ func newTripleStore(path string, options graph.Options) (graph.TripleStore, erro
return &qs, nil return &qs, nil
} }
func (qs *TripleStore) GetStats() string { func (qs *QuadStore) GetStats() string {
out := "" out := ""
stats, err := qs.db.GetProperty("leveldb.stats") stats, err := qs.db.GetProperty("leveldb.stats")
if err == nil { if err == nil {
@ -131,36 +131,46 @@ func (qs *TripleStore) GetStats() string {
return out return out
} }
func (qs *TripleStore) Size() int64 { func (qs *QuadStore) Size() int64 {
return qs.size return qs.size
} }
func (qs *TripleStore) Horizon() int64 { func (qs *QuadStore) Horizon() int64 {
return qs.horizon return qs.horizon
} }
func (qa *TripleStore) createDeltaKeyFor(d graph.Delta) []byte { func (qa *QuadStore) createDeltaKeyFor(d graph.Delta) []byte {
key := make([]byte, 0, 19) key := make([]byte, 0, 19)
key = append(key, 'd') key = append(key, 'd')
key = append(key, []byte(fmt.Sprintf("%018x", d.ID))...) key = append(key, []byte(fmt.Sprintf("%018x", d.ID))...)
return key return key
} }
func (qs *TripleStore) createKeyFor(d [4]quad.Direction, triple quad.Quad) []byte { func hashOf(s string) []byte {
key := make([]byte, 0, 2+(hashSize*3)) h := hashPool.Get().(hash.Hash)
// TODO(kortschak) Remove dependence on String() method. h.Reset()
key = append(key, []byte{d[0].Prefix(), d[1].Prefix()}...) defer hashPool.Put(h)
key = append(key, qs.convertStringToByteHash(triple.Get(d[0]))...) key := make([]byte, 0, hashSize)
key = append(key, qs.convertStringToByteHash(triple.Get(d[1]))...) h.Write([]byte(s))
key = append(key, qs.convertStringToByteHash(triple.Get(d[2]))...) key = h.Sum(key)
key = append(key, qs.convertStringToByteHash(triple.Get(d[3]))...)
return key return key
} }
func (qs *TripleStore) createValueKeyFor(s string) []byte { func (qs *QuadStore) createKeyFor(d [4]quad.Direction, q quad.Quad) []byte {
key := make([]byte, 0, 2+(hashSize*3))
// TODO(kortschak) Remove dependence on String() method.
key = append(key, []byte{d[0].Prefix(), d[1].Prefix()}...)
key = append(key, hashOf(q.Get(d[0]))...)
key = append(key, hashOf(q.Get(d[1]))...)
key = append(key, hashOf(q.Get(d[2]))...)
key = append(key, hashOf(q.Get(d[3]))...)
return key
}
func (qs *QuadStore) createValueKeyFor(s string) []byte {
key := make([]byte, 0, 1+hashSize) key := make([]byte, 0, 1+hashSize)
key = append(key, []byte("z")...) key = append(key, []byte("z")...)
key = append(key, qs.convertStringToByteHash(s)...) key = append(key, hashOf(s)...)
return key return key
} }
@ -177,7 +187,7 @@ var (
cps = [4]quad.Direction{quad.Label, quad.Predicate, quad.Subject, quad.Object} cps = [4]quad.Direction{quad.Label, quad.Predicate, quad.Subject, quad.Object}
) )
func (qs *TripleStore) ApplyDeltas(deltas []graph.Delta) error { func (qs *QuadStore) ApplyDeltas(deltas []graph.Delta) error {
batch := &leveldb.Batch{} batch := &leveldb.Batch{}
resizeMap := make(map[string]int64) resizeMap := make(map[string]int64)
size_change := int64(0) size_change := int64(0)
@ -214,14 +224,14 @@ func (qs *TripleStore) ApplyDeltas(deltas []graph.Delta) error {
} }
err := qs.db.Write(batch, qs.writeopts) err := qs.db.Write(batch, qs.writeopts)
if err != nil { if err != nil {
glog.Error("Couldn't write to DB for tripleset.") glog.Error("Couldn't write to DB for quadset.")
return err return err
} }
qs.size += size_change qs.size += size_change
return nil return nil
} }
func (qs *TripleStore) buildQuadWrite(batch *leveldb.Batch, q quad.Quad, id int64, isAdd bool) error { func (qs *QuadStore) buildQuadWrite(batch *leveldb.Batch, q quad.Quad, id int64, isAdd bool) error {
var entry IndexEntry var entry IndexEntry
data, err := qs.db.Get(qs.createKeyFor(spo, q), qs.readopts) data, err := qs.db.Get(qs.createKeyFor(spo, q), qs.readopts)
if err != nil && err != leveldb.ErrNotFound { if err != nil && err != leveldb.ErrNotFound {
@ -263,7 +273,7 @@ type ValueData struct {
Size int64 Size int64
} }
func (qs *TripleStore) UpdateValueKeyBy(name string, amount int64, batch *leveldb.Batch) error { func (qs *QuadStore) UpdateValueKeyBy(name string, amount int64, batch *leveldb.Batch) error {
value := &ValueData{name, amount} value := &ValueData{name, amount}
key := qs.createValueKeyFor(name) key := qs.createValueKeyFor(name)
b, err := qs.db.Get(key, qs.readopts) b, err := qs.db.Get(key, qs.readopts)
@ -303,7 +313,7 @@ func (qs *TripleStore) UpdateValueKeyBy(name string, amount int64, batch *leveld
return nil return nil
} }
func (qs *TripleStore) Close() { func (qs *QuadStore) Close() {
buf := new(bytes.Buffer) buf := new(bytes.Buffer)
err := binary.Write(buf, binary.LittleEndian, qs.size) err := binary.Write(buf, binary.LittleEndian, qs.size)
if err == nil { if err == nil {
@ -328,40 +338,30 @@ func (qs *TripleStore) Close() {
qs.open = false qs.open = false
} }
func (qs *TripleStore) Quad(k graph.Value) quad.Quad { func (qs *QuadStore) Quad(k graph.Value) quad.Quad {
var triple quad.Quad var q quad.Quad
b, err := qs.db.Get(k.(Token), qs.readopts) b, err := qs.db.Get(k.(Token), qs.readopts)
if err != nil && err != leveldb.ErrNotFound { if err != nil && err != leveldb.ErrNotFound {
glog.Error("Error: couldn't get triple from DB.") glog.Error("Error: couldn't get quad from DB.")
return quad.Quad{} return quad.Quad{}
} }
if err == leveldb.ErrNotFound { if err == leveldb.ErrNotFound {
// No harm, no foul. // No harm, no foul.
return quad.Quad{} return quad.Quad{}
} }
err = json.Unmarshal(b, &triple) err = json.Unmarshal(b, &q)
if err != nil { if err != nil {
glog.Error("Error: couldn't reconstruct triple.") glog.Error("Error: couldn't reconstruct quad.")
return quad.Quad{} return quad.Quad{}
} }
return triple return q
} }
func (qs *TripleStore) convertStringToByteHash(s string) []byte { func (qs *QuadStore) ValueOf(s string) graph.Value {
h := hashPool.Get().(hash.Hash)
h.Reset()
defer hashPool.Put(h)
key := make([]byte, 0, hashSize)
h.Write([]byte(s))
key = h.Sum(key)
return key
}
func (qs *TripleStore) ValueOf(s string) graph.Value {
return Token(qs.createValueKeyFor(s)) return Token(qs.createValueKeyFor(s))
} }
func (qs *TripleStore) valueData(value_key []byte) ValueData { func (qs *QuadStore) valueData(value_key []byte) ValueData {
var out ValueData var out ValueData
if glog.V(3) { if glog.V(3) {
glog.V(3).Infof("%s %v", string(value_key[0]), value_key) glog.V(3).Infof("%s %v", string(value_key[0]), value_key)
@ -381,7 +381,7 @@ func (qs *TripleStore) valueData(value_key []byte) ValueData {
return out return out
} }
func (qs *TripleStore) NameOf(k graph.Value) string { func (qs *QuadStore) NameOf(k graph.Value) string {
if k == nil { if k == nil {
glog.V(2).Info("k was nil") glog.V(2).Info("k was nil")
return "" return ""
@ -389,14 +389,14 @@ func (qs *TripleStore) NameOf(k graph.Value) string {
return qs.valueData(k.(Token)).Name return qs.valueData(k.(Token)).Name
} }
func (qs *TripleStore) SizeOf(k graph.Value) int64 { func (qs *QuadStore) SizeOf(k graph.Value) int64 {
if k == nil { if k == nil {
return 0 return 0
} }
return int64(qs.valueData(k.(Token)).Size) return int64(qs.valueData(k.(Token)).Size)
} }
func (qs *TripleStore) getInt64ForKey(key string, empty int64) (int64, error) { func (qs *QuadStore) getInt64ForKey(key string, empty int64) (int64, error) {
var out int64 var out int64
b, err := qs.db.Get([]byte(key), qs.readopts) b, err := qs.db.Get([]byte(key), qs.readopts)
if err != nil && err != leveldb.ErrNotFound { if err != nil && err != leveldb.ErrNotFound {
@ -416,7 +416,7 @@ func (qs *TripleStore) getInt64ForKey(key string, empty int64) (int64, error) {
return out, nil return out, nil
} }
func (qs *TripleStore) getMetadata() error { func (qs *QuadStore) getMetadata() error {
var err error var err error
qs.size, err = qs.getInt64ForKey("__size", 0) qs.size, err = qs.getInt64ForKey("__size", 0)
if err != nil { if err != nil {
@ -426,7 +426,7 @@ func (qs *TripleStore) getMetadata() error {
return err return err
} }
func (qs *TripleStore) SizeOfPrefix(pre []byte) (int64, error) { func (qs *QuadStore) SizeOfPrefix(pre []byte) (int64, error) {
limit := make([]byte, len(pre)) limit := make([]byte, len(pre))
copy(limit, pre) copy(limit, pre)
end := len(limit) - 1 end := len(limit) - 1
@ -441,7 +441,7 @@ func (qs *TripleStore) SizeOfPrefix(pre []byte) (int64, error) {
return 0, nil return 0, nil
} }
func (qs *TripleStore) TripleIterator(d quad.Direction, val graph.Value) graph.Iterator { func (qs *QuadStore) QuadIterator(d quad.Direction, val graph.Value) graph.Iterator {
var prefix string var prefix string
switch d { switch d {
case quad.Subject: case quad.Subject:
@ -458,15 +458,15 @@ func (qs *TripleStore) TripleIterator(d quad.Direction, val graph.Value) graph.I
return NewIterator(prefix, d, val, qs) return NewIterator(prefix, d, val, qs)
} }
func (qs *TripleStore) NodesAllIterator() graph.Iterator { func (qs *QuadStore) NodesAllIterator() graph.Iterator {
return NewAllIterator("z", quad.Any, qs) return NewAllIterator("z", quad.Any, qs)
} }
func (qs *TripleStore) TriplesAllIterator() graph.Iterator { func (qs *QuadStore) QuadsAllIterator() graph.Iterator {
return NewAllIterator("po", quad.Predicate, qs) return NewAllIterator("po", quad.Predicate, qs)
} }
func (qs *TripleStore) TripleDirection(val graph.Value, d quad.Direction) graph.Value { func (qs *QuadStore) QuadDirection(val graph.Value, d quad.Direction) graph.Value {
v := val.(Token) v := val.(Token)
offset := PositionOf(v[0:2], d, qs) offset := PositionOf(v[0:2], d, qs)
if offset != -1 { if offset != -1 {
@ -480,6 +480,6 @@ func compareBytes(a, b graph.Value) bool {
return bytes.Equal(a.(Token), b.(Token)) return bytes.Equal(a.(Token), b.(Token))
} }
func (qs *TripleStore) FixedIterator() graph.FixedIterator { func (qs *QuadStore) FixedIterator() graph.FixedIterator {
return iterator.NewFixedIteratorWithCompare(compareBytes) return iterator.NewFixedIteratorWithCompare(compareBytes)
} }

8
graph/leveldb/triplestore_iterator_optimize.go → graph/leveldb/quadstore_iterator_optimize.go
View file

@ -19,16 +19,16 @@ import (
"github.com/google/cayley/graph/iterator" "github.com/google/cayley/graph/iterator"
) )
func (ts *TripleStore) OptimizeIterator(it graph.Iterator) (graph.Iterator, bool) { func (qs *QuadStore) OptimizeIterator(it graph.Iterator) (graph.Iterator, bool) {
switch it.Type() { switch it.Type() {
case graph.LinksTo: case graph.LinksTo:
return ts.optimizeLinksTo(it.(*iterator.LinksTo)) return qs.optimizeLinksTo(it.(*iterator.LinksTo))
} }
return it, false return it, false
} }
func (ts *TripleStore) optimizeLinksTo(it *iterator.LinksTo) (graph.Iterator, bool) { func (qs *QuadStore) optimizeLinksTo(it *iterator.LinksTo) (graph.Iterator, bool) {
subs := it.SubIterators() subs := it.SubIterators()
if len(subs) != 1 { if len(subs) != 1 {
return it, false return it, false
@ -41,7 +41,7 @@ func (ts *TripleStore) optimizeLinksTo(it *iterator.LinksTo) (graph.Iterator, bo
panic("unexpected size during optimize") panic("unexpected size during optimize")
} }
val := primary.Result() val := primary.Result()
newIt := ts.TripleIterator(it.Direction(), val) newIt := qs.QuadIterator(it.Direction(), val)
nt := newIt.Tagger() nt := newIt.Tagger()
nt.CopyFrom(it) nt.CopyFrom(it)
for _, tag := range primary.Tagger().Tags() { for _, tag := range primary.Tagger().Tags() {

40
graph/memstore/all_iterator.go
View file

@ -21,48 +21,50 @@ import (
type AllIterator struct { type AllIterator struct {
iterator.Int64 iterator.Int64
ts *TripleStore qs *QuadStore
} }
type NodesAllIterator AllIterator type (
type QuadsAllIterator AllIterator nodesAllIterator AllIterator
quadsAllIterator AllIterator
)
func NewMemstoreNodesAllIterator(ts *TripleStore) *NodesAllIterator { func newNodesAllIterator(qs *QuadStore) *nodesAllIterator {
var out NodesAllIterator var out nodesAllIterator
out.Int64 = *iterator.NewInt64(1, ts.idCounter-1) out.Int64 = *iterator.NewInt64(1, qs.idCounter-1)
out.ts = ts out.qs = qs
return &out return &out
} }
// No subiterators. // No subiterators.
func (it *NodesAllIterator) SubIterators() []graph.Iterator { func (it *nodesAllIterator) SubIterators() []graph.Iterator {
return nil return nil
} }
func (it *NodesAllIterator) Next() bool { func (it *nodesAllIterator) Next() bool {
if !it.Int64.Next() { if !it.Int64.Next() {
return false return false
} }
_, ok := it.ts.revIdMap[it.Int64.Result().(int64)] _, ok := it.qs.revIdMap[it.Int64.Result().(int64)]
if !ok { if !ok {
return it.Next() return it.Next()
} }
return true return true
} }
func NewMemstoreQuadsAllIterator(ts *TripleStore) *QuadsAllIterator { func newQuadsAllIterator(qs *QuadStore) *quadsAllIterator {
var out QuadsAllIterator var out quadsAllIterator
out.Int64 = *iterator.NewInt64(1, ts.quadIdCounter-1) out.Int64 = *iterator.NewInt64(1, qs.quadIdCounter-1)
out.ts = ts out.qs = qs
return &out return &out
} }
func (qit *QuadsAllIterator) Next() bool { func (it *quadsAllIterator) Next() bool {
out := qit.Int64.Next() out := it.Int64.Next()
if out { if out {
i64 := qit.Int64.Result().(int64) i64 := it.Int64.Result().(int64)
if qit.ts.log[i64].DeletedBy != 0 || qit.ts.log[i64].Action == graph.Delete { if it.qs.log[i64].DeletedBy != 0 || it.qs.log[i64].Action == graph.Delete {
return qit.Next() return it.Next()
} }
} }
return out return out

10
graph/memstore/iterator.go
View file

@ -26,7 +26,7 @@ import (
type Iterator struct { type Iterator struct {
uid uint64 uid uint64
ts *TripleStore qs *QuadStore
tags graph.Tagger tags graph.Tagger
tree *b.Tree tree *b.Tree
iter *b.Enumerator iter *b.Enumerator
@ -38,14 +38,14 @@ func cmp(a, b int64) int {
return int(a - b) return int(a - b)
} }
func NewIterator(tree *b.Tree, data string, ts *TripleStore) *Iterator { func NewIterator(tree *b.Tree, data string, qs *QuadStore) *Iterator {
iter, err := tree.SeekFirst() iter, err := tree.SeekFirst()
if err != nil { if err != nil {
iter = nil iter = nil
} }
return &Iterator{ return &Iterator{
uid: iterator.NextUID(), uid: iterator.NextUID(),
ts: ts, qs: qs,
tree: tree, tree: tree,
iter: iter, iter: iter,
data: data, data: data,
@ -96,7 +96,7 @@ func (it *Iterator) Clone() graph.Iterator {
m := &Iterator{ m := &Iterator{
uid: iterator.NextUID(), uid: iterator.NextUID(),
ts: it.ts, qs: it.qs,
tree: it.tree, tree: it.tree,
iter: iter, iter: iter,
data: it.data, data: it.data,
@ -109,7 +109,7 @@ func (it *Iterator) Clone() graph.Iterator {
func (it *Iterator) Close() {} func (it *Iterator) Close() {}
func (it *Iterator) checkValid(index int64) bool { func (it *Iterator) checkValid(index int64) bool {
return it.ts.log[index].DeletedBy == 0 return it.qs.log[index].DeletedBy == 0
} }
func (it *Iterator) Next() bool { func (it *Iterator) Next() bool {

112
graph/memstore/triplestore.go → graph/memstore/quadstore.go
View file

@ -26,8 +26,8 @@ import (
) )
func init() { func init() {
graph.RegisterTripleStore("memstore", false, func(string, graph.Options) (graph.TripleStore, error) { graph.RegisterQuadStore("memstore", false, func(string, graph.Options) (graph.QuadStore, error) {
return newTripleStore(), nil return newQuadStore(), nil
}, nil) }, nil)
} }
@ -69,7 +69,7 @@ type LogEntry struct {
DeletedBy int64 DeletedBy int64
} }
type TripleStore struct { type QuadStore struct {
idCounter int64 idCounter int64
quadIdCounter int64 quadIdCounter int64
idMap map[string]int64 idMap map[string]int64
@ -80,8 +80,8 @@ type TripleStore struct {
// vip_index map[string]map[int64]map[string]map[int64]*b.Tree // vip_index map[string]map[int64]map[string]map[int64]*b.Tree
} }
func newTripleStore() *TripleStore { func newQuadStore() *QuadStore {
return &TripleStore{ return &QuadStore{
idMap: make(map[string]int64), idMap: make(map[string]int64),
revIdMap: make(map[int64]string), revIdMap: make(map[int64]string),
@ -94,13 +94,13 @@ func newTripleStore() *TripleStore {
} }
} }
func (ts *TripleStore) ApplyDeltas(deltas []graph.Delta) error { func (qs *QuadStore) ApplyDeltas(deltas []graph.Delta) error {
for _, d := range deltas { for _, d := range deltas {
var err error var err error
if d.Action == graph.Add { if d.Action == graph.Add {
err = ts.AddDelta(d) err = qs.AddDelta(d)
} else { } else {
err = ts.RemoveDelta(d) err = qs.RemoveDelta(d)
} }
if err != nil { if err != nil {
return err return err
@ -111,7 +111,7 @@ func (ts *TripleStore) ApplyDeltas(deltas []graph.Delta) error {
const maxInt = int(^uint(0) >> 1) const maxInt = int(^uint(0) >> 1)
func (ts *TripleStore) indexOf(t quad.Quad) (int64, bool) { func (qs *QuadStore) indexOf(t quad.Quad) (int64, bool) {
min := maxInt min := maxInt
var tree *b.Tree var tree *b.Tree
for d := quad.Subject; d <= quad.Label; d++ { for d := quad.Subject; d <= quad.Label; d++ {
@ -119,12 +119,12 @@ func (ts *TripleStore) indexOf(t quad.Quad) (int64, bool) {
if d == quad.Label && sid == "" { if d == quad.Label && sid == "" {
continue continue
} }
id, ok := ts.idMap[sid] id, ok := qs.idMap[sid]
// If we've never heard about a node, it must not exist // If we've never heard about a node, it must not exist
if !ok { if !ok {
return 0, false return 0, false
} }
index, ok := ts.index.Get(d, id) index, ok := qs.index.Get(d, id)
if !ok { if !ok {
// If it's never been indexed in this direction, it can't exist. // If it's never been indexed in this direction, it can't exist.
return 0, false return 0, false
@ -133,35 +133,35 @@ func (ts *TripleStore) indexOf(t quad.Quad) (int64, bool) {
min, tree = l, index min, tree = l, index
} }
} }
it := NewIterator(tree, "", ts) it := NewIterator(tree, "", qs)
for it.Next() { for it.Next() {
val := it.Result() val := it.Result()
if t == ts.log[val.(int64)].Quad { if t == qs.log[val.(int64)].Quad {
return val.(int64), true return val.(int64), true
} }
} }
return 0, false return 0, false
} }
func (ts *TripleStore) AddDelta(d graph.Delta) error { func (qs *QuadStore) AddDelta(d graph.Delta) error {
if _, exists := ts.indexOf(d.Quad); exists { if _, exists := qs.indexOf(d.Quad); exists {
return graph.ErrQuadExists return graph.ErrQuadExists
} }
qid := ts.quadIdCounter qid := qs.quadIdCounter
ts.log = append(ts.log, LogEntry{Delta: d}) qs.log = append(qs.log, LogEntry{Delta: d})
ts.size++ qs.size++
ts.quadIdCounter++ qs.quadIdCounter++
for dir := quad.Subject; dir <= quad.Label; dir++ { for dir := quad.Subject; dir <= quad.Label; dir++ {
sid := d.Quad.Get(dir) sid := d.Quad.Get(dir)
if dir == quad.Label && sid == "" { if dir == quad.Label && sid == "" {
continue continue
} }
if _, ok := ts.idMap[sid]; !ok { if _, ok := qs.idMap[sid]; !ok {
ts.idMap[sid] = ts.idCounter qs.idMap[sid] = qs.idCounter
ts.revIdMap[ts.idCounter] = sid qs.revIdMap[qs.idCounter] = sid
ts.idCounter++ qs.idCounter++
} }
} }
@ -169,8 +169,8 @@ func (ts *TripleStore) AddDelta(d graph.Delta) error {
if dir == quad.Label && d.Quad.Get(dir) == "" { if dir == quad.Label && d.Quad.Get(dir) == "" {
continue continue
} }
id := ts.idMap[d.Quad.Get(dir)] id := qs.idMap[d.Quad.Get(dir)]
tree := ts.index.Tree(dir, id) tree := qs.index.Tree(dir, id)
tree.Set(qid, struct{}{}) tree.Set(qid, struct{}{})
} }
@ -178,43 +178,43 @@ func (ts *TripleStore) AddDelta(d graph.Delta) error {
return nil return nil
} }
func (ts *TripleStore) RemoveDelta(d graph.Delta) error { func (qs *QuadStore) RemoveDelta(d graph.Delta) error {
prevQuadID, exists := ts.indexOf(d.Quad) prevQuadID, exists := qs.indexOf(d.Quad)
if !exists { if !exists {
return graph.ErrQuadNotExist return graph.ErrQuadNotExist
} }
quadID := ts.quadIdCounter quadID := qs.quadIdCounter
ts.log = append(ts.log, LogEntry{Delta: d}) qs.log = append(qs.log, LogEntry{Delta: d})
ts.log[prevQuadID].DeletedBy = quadID qs.log[prevQuadID].DeletedBy = quadID
ts.size-- qs.size--
ts.quadIdCounter++ qs.quadIdCounter++
return nil return nil
} }
func (ts *TripleStore) Quad(index graph.Value) quad.Quad { func (qs *QuadStore) Quad(index graph.Value) quad.Quad {
return ts.log[index.(int64)].Quad return qs.log[index.(int64)].Quad
} }
func (ts *TripleStore) TripleIterator(d quad.Direction, value graph.Value) graph.Iterator { func (qs *QuadStore) QuadIterator(d quad.Direction, value graph.Value) graph.Iterator {
index, ok := ts.index.Get(d, value.(int64)) index, ok := qs.index.Get(d, value.(int64))
data := fmt.Sprintf("dir:%s val:%d", d, value.(int64)) data := fmt.Sprintf("dir:%s val:%d", d, value.(int64))
if ok { if ok {
return NewIterator(index, data, ts) return NewIterator(index, data, qs)
} }
return &iterator.Null{} return &iterator.Null{}
} }
func (ts *TripleStore) Horizon() int64 { func (qs *QuadStore) Horizon() int64 {
return ts.log[len(ts.log)-1].ID return qs.log[len(qs.log)-1].ID
} }
func (ts *TripleStore) Size() int64 { func (qs *QuadStore) Size() int64 {
return ts.size return qs.size
} }
func (ts *TripleStore) DebugPrint() { func (qs *QuadStore) DebugPrint() {
for i, l := range ts.log { for i, l := range qs.log {
if i == 0 { if i == 0 {
continue continue
} }
@ -222,29 +222,29 @@ func (ts *TripleStore) DebugPrint() {
} }
} }
func (ts *TripleStore) ValueOf(name string) graph.Value { func (qs *QuadStore) ValueOf(name string) graph.Value {
return ts.idMap[name] return qs.idMap[name]
} }
func (ts *TripleStore) NameOf(id graph.Value) string { func (qs *QuadStore) NameOf(id graph.Value) string {
return ts.revIdMap[id.(int64)] return qs.revIdMap[id.(int64)]
} }
func (ts *TripleStore) TriplesAllIterator() graph.Iterator { func (qs *QuadStore) QuadsAllIterator() graph.Iterator {
return NewMemstoreQuadsAllIterator(ts) return newQuadsAllIterator(qs)
} }
func (ts *TripleStore) FixedIterator() graph.FixedIterator { func (qs *QuadStore) FixedIterator() graph.FixedIterator {
return iterator.NewFixedIteratorWithCompare(iterator.BasicEquality) return iterator.NewFixedIteratorWithCompare(iterator.BasicEquality)
} }
func (ts *TripleStore) TripleDirection(val graph.Value, d quad.Direction) graph.Value { func (qs *QuadStore) QuadDirection(val graph.Value, d quad.Direction) graph.Value {
name := ts.Quad(val).Get(d) name := qs.Quad(val).Get(d)
return ts.ValueOf(name) return qs.ValueOf(name)
} }
func (ts *TripleStore) NodesAllIterator() graph.Iterator { func (qs *QuadStore) NodesAllIterator() graph.Iterator {
return NewMemstoreNodesAllIterator(ts) return newNodesAllIterator(qs)
} }
func (ts *TripleStore) Close() {} func (qs *QuadStore) Close() {}

8
graph/memstore/triplestore_iterator_optimize.go → graph/memstore/quadstore_iterator_optimize.go
View file

@ -19,16 +19,16 @@ import (
"github.com/google/cayley/graph/iterator" "github.com/google/cayley/graph/iterator"
) )
func (ts *TripleStore) OptimizeIterator(it graph.Iterator) (graph.Iterator, bool) { func (qs *QuadStore) OptimizeIterator(it graph.Iterator) (graph.Iterator, bool) {
switch it.Type() { switch it.Type() {
case graph.LinksTo: case graph.LinksTo:
return ts.optimizeLinksTo(it.(*iterator.LinksTo)) return qs.optimizeLinksTo(it.(*iterator.LinksTo))
} }
return it, false return it, false
} }
func (ts *TripleStore) optimizeLinksTo(it *iterator.LinksTo) (graph.Iterator, bool) { func (qs *QuadStore) optimizeLinksTo(it *iterator.LinksTo) (graph.Iterator, bool) {
subs := it.SubIterators() subs := it.SubIterators()
if len(subs) != 1 { if len(subs) != 1 {
return it, false return it, false
@ -41,7 +41,7 @@ func (ts *TripleStore) optimizeLinksTo(it *iterator.LinksTo) (graph.Iterator, bo
panic("unexpected size during optimize") panic("unexpected size during optimize")
} }
val := primary.Result() val := primary.Result()
newIt := ts.TripleIterator(it.Direction(), val) newIt := qs.QuadIterator(it.Direction(), val)
nt := newIt.Tagger() nt := newIt.Tagger()
nt.CopyFrom(it) nt.CopyFrom(it)
for _, tag := range primary.Tagger().Tags() { for _, tag := range primary.Tagger().Tags() {

66
graph/memstore/triplestore_test.go → graph/memstore/quadstore_test.go
View file

@ -52,14 +52,14 @@ var simpleGraph = []quad.Quad{
{"G", "status", "cool", "status_graph"}, {"G", "status", "cool", "status_graph"},
} }
func makeTestStore(data []quad.Quad) (*TripleStore, graph.QuadWriter, []pair) { func makeTestStore(data []quad.Quad) (*QuadStore, graph.QuadWriter, []pair) {
seen := make(map[string]struct{}) seen := make(map[string]struct{})
ts := newTripleStore() qs := newQuadStore()
var ( var (
val int64 val int64
ind []pair ind []pair
) )
writer, _ := writer.NewSingleReplication(ts, nil) writer, _ := writer.NewSingleReplication(qs, nil)
for _, t := range data { for _, t := range data {
for _, qp := range []string{t.Subject, t.Predicate, t.Object, t.Label} { for _, qp := range []string{t.Subject, t.Predicate, t.Object, t.Label} {
if _, ok := seen[qp]; !ok && qp != "" { if _, ok := seen[qp]; !ok && qp != "" {
@ -71,7 +71,7 @@ func makeTestStore(data []quad.Quad) (*TripleStore, graph.QuadWriter, []pair) {
writer.AddQuad(t) writer.AddQuad(t)
} }
return ts, writer, ind return qs, writer, ind
} }
type pair struct { type pair struct {
@ -80,12 +80,12 @@ type pair struct {
} }
func TestMemstore(t *testing.T) { func TestMemstore(t *testing.T) {
ts, _, index := makeTestStore(simpleGraph) qs, _, index := makeTestStore(simpleGraph)
if size := ts.Size(); size != int64(len(simpleGraph)) { if size := qs.Size(); size != int64(len(simpleGraph)) {
t.Errorf("Triple store has unexpected size, got:%d expected %d", size, len(simpleGraph)) t.Errorf("Quad store has unexpected size, got:%d expected %d", size, len(simpleGraph))
} }
for _, test := range index { for _, test := range index {
v := ts.ValueOf(test.query) v := qs.ValueOf(test.query)
switch v := v.(type) { switch v := v.(type) {
default: default:
t.Errorf("ValueOf(%q) returned unexpected type, got:%T expected int64", test.query, v) t.Errorf("ValueOf(%q) returned unexpected type, got:%T expected int64", test.query, v)
@ -98,21 +98,21 @@ func TestMemstore(t *testing.T) {
} }
func TestIteratorsAndNextResultOrderA(t *testing.T) { func TestIteratorsAndNextResultOrderA(t *testing.T) {
ts, _, _ := makeTestStore(simpleGraph) qs, _, _ := makeTestStore(simpleGraph)
fixed := ts.FixedIterator() fixed := qs.FixedIterator()
fixed.Add(ts.ValueOf("C")) fixed.Add(qs.ValueOf("C"))
fixed2 := ts.FixedIterator() fixed2 := qs.FixedIterator()
fixed2.Add(ts.ValueOf("follows")) fixed2.Add(qs.ValueOf("follows"))
all := ts.NodesAllIterator() all := qs.NodesAllIterator()
innerAnd := iterator.NewAnd() innerAnd := iterator.NewAnd()
innerAnd.AddSubIterator(iterator.NewLinksTo(ts, fixed2, quad.Predicate)) innerAnd.AddSubIterator(iterator.NewLinksTo(qs, fixed2, quad.Predicate))
innerAnd.AddSubIterator(iterator.NewLinksTo(ts, all, quad.Object)) innerAnd.AddSubIterator(iterator.NewLinksTo(qs, all, quad.Object))
hasa := iterator.NewHasA(ts, innerAnd, quad.Subject) hasa := iterator.NewHasA(qs, innerAnd, quad.Subject)
outerAnd := iterator.NewAnd() outerAnd := iterator.NewAnd()
outerAnd.AddSubIterator(fixed) outerAnd.AddSubIterator(fixed)
outerAnd.AddSubIterator(hasa) outerAnd.AddSubIterator(hasa)
@ -121,8 +121,8 @@ func TestIteratorsAndNextResultOrderA(t *testing.T) {
t.Error("Expected one matching subtree") t.Error("Expected one matching subtree")
} }
val := outerAnd.Result() val := outerAnd.Result()
if ts.NameOf(val) != "C" { if qs.NameOf(val) != "C" {
t.Errorf("Matching subtree should be %s, got %s", "barak", ts.NameOf(val)) t.Errorf("Matching subtree should be %s, got %s", "barak", qs.NameOf(val))
} }
var ( var (
@ -130,7 +130,7 @@ func TestIteratorsAndNextResultOrderA(t *testing.T) {
expect = []string{"B", "D"} expect = []string{"B", "D"}
) )
for { for {
got = append(got, ts.NameOf(all.Result())) got = append(got, qs.NameOf(all.Result()))
if !outerAnd.NextPath() { if !outerAnd.NextPath() {
break break
} }
@ -147,12 +147,12 @@ func TestIteratorsAndNextResultOrderA(t *testing.T) {
} }
func TestLinksToOptimization(t *testing.T) { func TestLinksToOptimization(t *testing.T) {
ts, _, _ := makeTestStore(simpleGraph) qs, _, _ := makeTestStore(simpleGraph)
fixed := ts.FixedIterator() fixed := qs.FixedIterator()
fixed.Add(ts.ValueOf("cool")) fixed.Add(qs.ValueOf("cool"))
lto := iterator.NewLinksTo(ts, fixed, quad.Object) lto := iterator.NewLinksTo(qs, fixed, quad.Object)
lto.Tagger().Add("foo") lto.Tagger().Add("foo")
newIt, changed := lto.Optimize() newIt, changed := lto.Optimize()
@ -174,22 +174,22 @@ func TestLinksToOptimization(t *testing.T) {
} }
} }
func TestRemoveTriple(t *testing.T) { func TestRemoveQuad(t *testing.T) {
ts, w, _ := makeTestStore(simpleGraph) qs, w, _ := makeTestStore(simpleGraph)
w.RemoveQuad(quad.Quad{"E", "follows", "F", ""}) w.RemoveQuad(quad.Quad{"E", "follows", "F", ""})
fixed := ts.FixedIterator() fixed := qs.FixedIterator()
fixed.Add(ts.ValueOf("E")) fixed.Add(qs.ValueOf("E"))
fixed2 := ts.FixedIterator() fixed2 := qs.FixedIterator()
fixed2.Add(ts.ValueOf("follows")) fixed2.Add(qs.ValueOf("follows"))
innerAnd := iterator.NewAnd() innerAnd := iterator.NewAnd()
innerAnd.AddSubIterator(iterator.NewLinksTo(ts, fixed, quad.Subject)) innerAnd.AddSubIterator(iterator.NewLinksTo(qs, fixed, quad.Subject))
innerAnd.AddSubIterator(iterator.NewLinksTo(ts, fixed2, quad.Predicate)) innerAnd.AddSubIterator(iterator.NewLinksTo(qs, fixed2, quad.Predicate))
hasa := iterator.NewHasA(ts, innerAnd, quad.Object) hasa := iterator.NewHasA(qs, innerAnd, quad.Object)
newIt, _ := hasa.Optimize() newIt, _ := hasa.Optimize()
if graph.Next(newIt) { if graph.Next(newIt) {

6
graph/mongo/iterator.go
View file

@ -30,7 +30,7 @@ import (
type Iterator struct { type Iterator struct {
uid uint64 uid uint64
tags graph.Tagger tags graph.Tagger
qs *TripleStore qs *QuadStore
dir quad.Direction dir quad.Direction
iter *mgo.Iter iter *mgo.Iter
hash string hash string
@ -42,7 +42,7 @@ type Iterator struct {
result graph.Value result graph.Value
} }
func NewIterator(qs *TripleStore, collection string, d quad.Direction, val graph.Value) *Iterator { func NewIterator(qs *QuadStore, collection string, d quad.Direction, val graph.Value) *Iterator {
name := qs.NameOf(val) name := qs.NameOf(val)
constraint := bson.M{d.String(): name} constraint := bson.M{d.String(): name}
@ -68,7 +68,7 @@ func NewIterator(qs *TripleStore, collection string, d quad.Direction, val graph
} }
} }
func NewAllIterator(qs *TripleStore, collection string) *Iterator { func NewAllIterator(qs *QuadStore, collection string) *Iterator {
size, err := qs.db.C(collection).Count() size, err := qs.db.C(collection).Count()
if err != nil { if err != nil {
// FIXME(kortschak) This should be passed back rather than just logging. // FIXME(kortschak) This should be passed back rather than just logging.

54
graph/mongo/triplestore.go → graph/mongo/quadstore.go
View file

@ -30,7 +30,7 @@ import (
) )
func init() { func init() {
graph.RegisterTripleStore("mongo", true, newTripleStore, createNewMongoGraph) graph.RegisterQuadStore("mongo", true, newQuadStore, createNewMongoGraph)
} }
const DefaultDBName = "cayley" const DefaultDBName = "cayley"
@ -42,7 +42,7 @@ var (
hashSize = sha1.Size hashSize = sha1.Size
) )
type TripleStore struct { type QuadStore struct {
session *mgo.Session session *mgo.Session
db *mgo.Database db *mgo.Database
idCache *IDLru idCache *IDLru
@ -84,8 +84,8 @@ func createNewMongoGraph(addr string, options graph.Options) error {
return nil return nil
} }
func newTripleStore(addr string, options graph.Options) (graph.TripleStore, error) { func newQuadStore(addr string, options graph.Options) (graph.QuadStore, error) {
var qs TripleStore var qs QuadStore
conn, err := mgo.Dial(addr) conn, err := mgo.Dial(addr)
if err != nil { if err != nil {
return nil, err return nil, err
@ -101,15 +101,15 @@ func newTripleStore(addr string, options graph.Options) (graph.TripleStore, erro
return &qs, nil return &qs, nil
} }
func (qs *TripleStore) getIdForQuad(t quad.Quad) string { func (qs *QuadStore) getIdForQuad(t quad.Quad) string {
id := qs.convertStringToByteHash(t.Subject) id := hashOf(t.Subject)
id += qs.convertStringToByteHash(t.Predicate) id += hashOf(t.Predicate)
id += qs.convertStringToByteHash(t.Object) id += hashOf(t.Object)
id += qs.convertStringToByteHash(t.Label) id += hashOf(t.Label)
return id return id
} }
func (qs *TripleStore) convertStringToByteHash(s string) string { func hashOf(s string) string {
h := hashPool.Get().(hash.Hash) h := hashPool.Get().(hash.Hash)
h.Reset() h.Reset()
defer hashPool.Put(h) defer hashPool.Put(h)
@ -133,7 +133,7 @@ type MongoLogEntry struct {
Timestamp int64 Timestamp int64
} }
func (qs *TripleStore) updateNodeBy(node_name string, inc int) error { func (qs *QuadStore) updateNodeBy(node_name string, inc int) error {
node := qs.ValueOf(node_name) node := qs.ValueOf(node_name)
doc := bson.M{ doc := bson.M{
"_id": node.(string), "_id": node.(string),
@ -153,7 +153,7 @@ func (qs *TripleStore) updateNodeBy(node_name string, inc int) error {
return err return err
} }
func (qs *TripleStore) updateQuad(q quad.Quad, id int64, proc graph.Procedure) error { func (qs *QuadStore) updateQuad(q quad.Quad, id int64, proc graph.Procedure) error {
var setname string var setname string
if proc == graph.Add { if proc == graph.Add {
setname = "Added" setname = "Added"
@ -173,7 +173,7 @@ func (qs *TripleStore) updateQuad(q quad.Quad, id int64, proc graph.Procedure) e
return err return err
} }
func (qs *TripleStore) checkValid(key string) bool { func (qs *QuadStore) checkValid(key string) bool {
var indexEntry struct { var indexEntry struct {
Added []int64 `bson:"Added"` Added []int64 `bson:"Added"`
Deleted []int64 `bson:"Deleted"` Deleted []int64 `bson:"Deleted"`
@ -192,7 +192,7 @@ func (qs *TripleStore) checkValid(key string) bool {
return true return true
} }
func (qs *TripleStore) updateLog(d graph.Delta) error { func (qs *QuadStore) updateLog(d graph.Delta) error {
var action string var action string
if d.Action == graph.Add { if d.Action == graph.Add {
action = "Add" action = "Add"
@ -212,7 +212,7 @@ func (qs *TripleStore) updateLog(d graph.Delta) error {
return err return err
} }
func (qs *TripleStore) ApplyDeltas(in []graph.Delta) error { func (qs *QuadStore) ApplyDeltas(in []graph.Delta) error {
qs.session.SetSafe(nil) qs.session.SetSafe(nil)
ids := make(map[string]int) ids := make(map[string]int)
// Pre-check the existence condition. // Pre-check the existence condition.
@ -266,7 +266,7 @@ func (qs *TripleStore) ApplyDeltas(in []graph.Delta) error {
return nil return nil
} }
func (qs *TripleStore) Quad(val graph.Value) quad.Quad { func (qs *QuadStore) Quad(val graph.Value) quad.Quad {
var q quad.Quad var q quad.Quad
err := qs.db.C("quads").FindId(val.(string)).One(&q) err := qs.db.C("quads").FindId(val.(string)).One(&q)
if err != nil { if err != nil {
@ -275,23 +275,23 @@ func (qs *TripleStore) Quad(val graph.Value) quad.Quad {
return q return q
} }
func (qs *TripleStore) TripleIterator(d quad.Direction, val graph.Value) graph.Iterator { func (qs *QuadStore) QuadIterator(d quad.Direction, val graph.Value) graph.Iterator {
return NewIterator(qs, "quads", d, val) return NewIterator(qs, "quads", d, val)
} }
func (qs *TripleStore) NodesAllIterator() graph.Iterator { func (qs *QuadStore) NodesAllIterator() graph.Iterator {
return NewAllIterator(qs, "nodes") return NewAllIterator(qs, "nodes")
} }
func (qs *TripleStore) TriplesAllIterator() graph.Iterator { func (qs *QuadStore) QuadsAllIterator() graph.Iterator {
return NewAllIterator(qs, "quads") return NewAllIterator(qs, "quads")
} }
func (qs *TripleStore) ValueOf(s string) graph.Value { func (qs *QuadStore) ValueOf(s string) graph.Value {
return qs.convertStringToByteHash(s) return hashOf(s)
} }
func (qs *TripleStore) NameOf(v graph.Value) string { func (qs *QuadStore) NameOf(v graph.Value) string {
val, ok := qs.idCache.Get(v.(string)) val, ok := qs.idCache.Get(v.(string))
if ok { if ok {
return val return val
@ -305,7 +305,7 @@ func (qs *TripleStore) NameOf(v graph.Value) string {
return node.Name return node.Name
} }
func (qs *TripleStore) Size() int64 { func (qs *QuadStore) Size() int64 {
// TODO(barakmich): Make size real; store it in the log, and retrieve it. // TODO(barakmich): Make size real; store it in the log, and retrieve it.
count, err := qs.db.C("quads").Count() count, err := qs.db.C("quads").Count()
if err != nil { if err != nil {
@ -315,7 +315,7 @@ func (qs *TripleStore) Size() int64 {
return int64(count) return int64(count)
} }
func (qs *TripleStore) Horizon() int64 { func (qs *QuadStore) Horizon() int64 {
var log MongoLogEntry var log MongoLogEntry
err := qs.db.C("log").Find(nil).Sort("-LogID").One(&log) err := qs.db.C("log").Find(nil).Sort("-LogID").One(&log)
if err != nil { if err != nil {
@ -331,15 +331,15 @@ func compareStrings(a, b graph.Value) bool {
return a.(string) == b.(string) return a.(string) == b.(string)
} }
func (qs *TripleStore) FixedIterator() graph.FixedIterator { func (qs *QuadStore) FixedIterator() graph.FixedIterator {
return iterator.NewFixedIteratorWithCompare(compareStrings) return iterator.NewFixedIteratorWithCompare(compareStrings)
} }
func (qs *TripleStore) Close() { func (qs *QuadStore) Close() {
qs.db.Session.Close() qs.db.Session.Close()
} }
func (qs *TripleStore) TripleDirection(in graph.Value, d quad.Direction) graph.Value { func (qs *QuadStore) QuadDirection(in graph.Value, d quad.Direction) graph.Value {
// Maybe do the trick here // Maybe do the trick here
var offset int var offset int
switch d { switch d {

8
graph/mongo/triplestore_iterator_optimize.go → graph/mongo/quadstore_iterator_optimize.go
View file

@ -19,16 +19,16 @@ import (
"github.com/google/cayley/graph/iterator" "github.com/google/cayley/graph/iterator"
) )
func (ts *TripleStore) OptimizeIterator(it graph.Iterator) (graph.Iterator, bool) { func (qs *QuadStore) OptimizeIterator(it graph.Iterator) (graph.Iterator, bool) {
switch it.Type() { switch it.Type() {
case graph.LinksTo: case graph.LinksTo:
return ts.optimizeLinksTo(it.(*iterator.LinksTo)) return qs.optimizeLinksTo(it.(*iterator.LinksTo))
} }
return it, false return it, false
} }
func (ts *TripleStore) optimizeLinksTo(it *iterator.LinksTo) (graph.Iterator, bool) { func (qs *QuadStore) optimizeLinksTo(it *iterator.LinksTo) (graph.Iterator, bool) {
subs := it.SubIterators() subs := it.SubIterators()
if len(subs) != 1 { if len(subs) != 1 {
return it, false return it, false
@ -41,7 +41,7 @@ func (ts *TripleStore) optimizeLinksTo(it *iterator.LinksTo) (graph.Iterator, bo
panic("unexpected size during optimize") panic("unexpected size during optimize")
} }
val := primary.Result() val := primary.Result()
newIt := ts.TripleIterator(it.Direction(), val) newIt := qs.QuadIterator(it.Direction(), val)
nt := newIt.Tagger() nt := newIt.Tagger()
nt.CopyFrom(it) nt.CopyFrom(it)
for _, tag := range primary.Tagger().Tags() { for _, tag := range primary.Tagger().Tags() {

62
graph/triplestore.go → graph/quadstore.go
View file

@ -14,12 +14,12 @@
package graph package graph
// Defines the TripleStore interface. Every backing store must implement at // Defines the QuadStore interface. Every backing store must implement at
// least this interface. // least this interface.
// //
// Most of these are pretty straightforward. As long as we can surface this // Most of these are pretty straightforward. As long as we can surface this
// interface, the rest of the stack will "just work" and we can connect to any // interface, the rest of the stack will "just work" and we can connect to any
// triple backing store we prefer. // quad backing store we prefer.
import ( import (
"errors" "errors"
@ -28,71 +28,73 @@ import (
"github.com/google/cayley/quad" "github.com/google/cayley/quad"
) )
// Value defines an opaque "triple store value" type. However the backend wishes // Value defines an opaque "quad store value" type. However the backend wishes
// to implement it, a Value is merely a token to a triple or a node that the // to implement it, a Value is merely a token to a quad or a node that the
// backing store itself understands, and the base iterators pass around. // backing store itself understands, and the base iterators pass around.
// //
// For example, in a very traditional, graphd-style graph, these are int64s // For example, in a very traditional, graphd-style graph, these are int64s
// (guids of the primitives). In a very direct sort of graph, these could be // (guids of the primitives). In a very direct sort of graph, these could be
// pointers to structs, or merely triples, or whatever works best for the // pointers to structs, or merely quads, or whatever works best for the
// backing store. // backing store.
// //
// These must be comparable, or implement a `Key() interface{}` function // These must be comparable, or implement a `Key() interface{}` function
// so that they may be stored in maps. // so that they may be stored in maps.
type Value interface{} type Value interface{}
type TripleStore interface { type QuadStore interface {
// The only way in is through building a transaction, which // The only way in is through building a transaction, which
// is done by a replication strategy. // is done by a replication strategy.
ApplyDeltas([]Delta) error ApplyDeltas([]Delta) error
// Given an opaque token, returns the triple for that token from the store. // Given an opaque token, returns the quad for that token from the store.
Quad(Value) quad.Quad Quad(Value) quad.Quad
// Given a direction and a token, creates an iterator of links which have // Given a direction and a token, creates an iterator of links which have
// that node token in that directional field. // that node token in that directional field.
TripleIterator(quad.Direction, Value) Iterator QuadIterator(quad.Direction, Value) Iterator
// Returns an iterator enumerating all nodes in the graph. // Returns an iterator enumerating all nodes in the graph.
NodesAllIterator() Iterator NodesAllIterator() Iterator
// Returns an iterator enumerating all links in the graph. // Returns an iterator enumerating all links in the graph.
TriplesAllIterator() Iterator QuadsAllIterator() Iterator
// Given a node ID, return the opaque token used by the TripleStore // Given a node ID, return the opaque token used by the QuadStore
// to represent that id. // to represent that id.
ValueOf(string) Value ValueOf(string) Value
// Given an opaque token, return the node that it represents. // Given an opaque token, return the node that it represents.
NameOf(Value) string NameOf(Value) string
// Returns the number of triples currently stored. // Returns the number of quads currently stored.
Size() int64 Size() int64
// The last replicated transaction ID that this triplestore has verified. // The last replicated transaction ID that this quadstore has verified.
Horizon() int64 Horizon() int64
// Creates a fixed iterator which can compare Values // Creates a fixed iterator which can compare Values
FixedIterator() FixedIterator FixedIterator() FixedIterator
// Optimize an iterator in the context of the triple store. // Optimize an iterator in the context of the quad store.
// Suppose we have a better index for the passed tree; this // Suppose we have a better index for the passed tree; this
// gives the TripleStore the opportunity to replace it // gives the QuadStore the opportunity to replace it
// with a more efficient iterator. // with a more efficient iterator.
OptimizeIterator(it Iterator) (Iterator, bool) OptimizeIterator(it Iterator) (Iterator, bool)
// Close the triple store and clean up. (Flush to disk, cleanly // Close the quad store and clean up. (Flush to disk, cleanly
// sever connections, etc) // sever connections, etc)
Close() Close()
// Convenience function for speed. Given a triple token and a direction // Convenience function for speed. Given a quad token and a direction
// return the node token for that direction. Sometimes, a TripleStore // return the node token for that direction. Sometimes, a QuadStore
// can do this without going all the way to the backing store, and // can do this without going all the way to the backing store, and
// gives the TripleStore the opportunity to make this optimization. // gives the QuadStore the opportunity to make this optimization.
// //
// Iterators will call this. At worst, a valid implementation is // Iterators will call this. At worst, a valid implementation is
// ts.IdFor(ts.quad.Quad(id).Get(dir)) //
TripleDirection(id Value, d quad.Direction) Value // qs.ValueOf(qs.Quad(id).Get(dir))
//
QuadDirection(id Value, d quad.Direction) Value
} }
type Options map[string]interface{} type Options map[string]interface{}
@ -133,16 +135,16 @@ func (d Options) BoolKey(key string) (bool, bool) {
return false, false return false, false
} }
var ErrCannotBulkLoad = errors.New("triplestore: cannot bulk load") var ErrCannotBulkLoad = errors.New("quadstore: cannot bulk load")
type BulkLoader interface { type BulkLoader interface {
// BulkLoad loads Quads from a quad.Unmarshaler in bulk to the TripleStore. // BulkLoad loads Quads from a quad.Unmarshaler in bulk to the QuadStore.
// It returns ErrCannotBulkLoad if bulk loading is not possible. For example if // It returns ErrCannotBulkLoad if bulk loading is not possible. For example if
// you cannot load in bulk to a non-empty database, and the db is non-empty. // you cannot load in bulk to a non-empty database, and the db is non-empty.
BulkLoad(quad.Unmarshaler) error BulkLoad(quad.Unmarshaler) error
} }
type NewStoreFunc func(string, Options) (TripleStore, error) type NewStoreFunc func(string, Options) (QuadStore, error)
type InitStoreFunc func(string, Options) error type InitStoreFunc func(string, Options) error
type register struct { type register struct {
@ -153,9 +155,9 @@ type register struct {
var storeRegistry = make(map[string]register) var storeRegistry = make(map[string]register)
func RegisterTripleStore(name string, persists bool, newFunc NewStoreFunc, initFunc InitStoreFunc) { func RegisterQuadStore(name string, persists bool, newFunc NewStoreFunc, initFunc InitStoreFunc) {
if _, found := storeRegistry[name]; found { if _, found := storeRegistry[name]; found {
panic("already registered TripleStore " + name) panic("already registered QuadStore " + name)
} }
storeRegistry[name] = register{ storeRegistry[name] = register{
newFunc: newFunc, newFunc: newFunc,
@ -164,27 +166,27 @@ func RegisterTripleStore(name string, persists bool, newFunc NewStoreFunc, initF
} }
} }
func NewTripleStore(name, dbpath string, opts Options) (TripleStore, error) { func NewQuadStore(name, dbpath string, opts Options) (QuadStore, error) {
r, registered := storeRegistry[name] r, registered := storeRegistry[name]
if !registered { if !registered {
return nil, errors.New("triplestore: name '" + name + "' is not registered") return nil, errors.New("quadstore: name '" + name + "' is not registered")
} }
return r.newFunc(dbpath, opts) return r.newFunc(dbpath, opts)
} }
func InitTripleStore(name, dbpath string, opts Options) error { func InitQuadStore(name, dbpath string, opts Options) error {
r, registered := storeRegistry[name] r, registered := storeRegistry[name]
if registered { if registered {
return r.initFunc(dbpath, opts) return r.initFunc(dbpath, opts)
} }
return errors.New("triplestore: name '" + name + "' is not registered") return errors.New("quadstore: name '" + name + "' is not registered")
} }
func IsPersistent(name string) bool { func IsPersistent(name string) bool {
return storeRegistry[name].isPersistent return storeRegistry[name].isPersistent
} }
func TripleStores() []string { func QuadStores() []string {
t := make([]string, 0, len(storeRegistry)) t := make([]string, 0, len(storeRegistry))
for n := range storeRegistry { for n := range storeRegistry {
t = append(t, n) t = append(t, n)

14
graph/quadwriter.go
View file

@ -16,9 +16,9 @@ package graph
// Defines the interface for consistent replication of a graph instance. // Defines the interface for consistent replication of a graph instance.
// //
// Separate from the backend, this dictates how individual triples get // Separate from the backend, this dictates how individual quads get
// identified and replicated consistently across (potentially) multiple // identified and replicated consistently across (potentially) multiple
// instances. The simplest case is to keep an append-only log of triple // instances. The simplest case is to keep an append-only log of quad
// changes. // changes.
import ( import (
@ -44,7 +44,7 @@ type Delta struct {
} }
type Handle struct { type Handle struct {
QuadStore TripleStore QuadStore QuadStore
QuadWriter QuadWriter QuadWriter QuadWriter
} }
@ -73,23 +73,23 @@ type QuadWriter interface {
Close() error Close() error
} }
type NewQuadWriterFunc func(TripleStore, Options) (QuadWriter, error) type NewQuadWriterFunc func(QuadStore, Options) (QuadWriter, error)
var writerRegistry = make(map[string]NewQuadWriterFunc) var writerRegistry = make(map[string]NewQuadWriterFunc)
func RegisterWriter(name string, newFunc NewQuadWriterFunc) { func RegisterWriter(name string, newFunc NewQuadWriterFunc) {
if _, found := writerRegistry[name]; found { if _, found := writerRegistry[name]; found {
panic("already registered TripleWriter " + name) panic("already registered QuadWriter " + name)
} }
writerRegistry[name] = newFunc writerRegistry[name] = newFunc
} }
func NewQuadWriter(name string, ts TripleStore, opts Options) (QuadWriter, error) { func NewQuadWriter(name string, qs QuadStore, opts Options) (QuadWriter, error) {
newFunc, hasNew := writerRegistry[name] newFunc, hasNew := writerRegistry[name]
if !hasNew { if !hasNew {
return nil, errors.New("replication: name '" + name + "' is not registered") return nil, errors.New("replication: name '" + name + "' is not registered")
} }
return newFunc(ts, opts) return newFunc(qs, opts)
} }
func WriterMethods() []string { func WriterMethods() []string {

4
http/http_test.go
View file

@ -56,13 +56,13 @@ var parseTests = []struct {
{"subject": "foo", "predicate": "bar"} {"subject": "foo", "predicate": "bar"}
]`, ]`,
expect: nil, expect: nil,
err: fmt.Errorf("Invalid triple at index %d. %v", 0, quad.Quad{"foo", "bar", "", ""}), err: fmt.Errorf("Invalid quad at index %d. %v", 0, quad.Quad{"foo", "bar", "", ""}),
}, },
} }
func TestParseJSON(t *testing.T) { func TestParseJSON(t *testing.T) {
for _, test := range parseTests { for _, test := range parseTests {
got, err := ParseJsonToTripleList([]byte(test.input)) got, err := ParseJsonToQuadList([]byte(test.input))
if fmt.Sprint(err) != fmt.Sprint(test.err) { if fmt.Sprint(err) != fmt.Sprint(test.err) {
t.Errorf("Failed to %v with unexpected error, got:%v expected %v", test.message, err, test.err) t.Errorf("Failed to %v with unexpected error, got:%v expected %v", test.message, err, test.err)
} }

38
http/write.go
View file

@ -29,18 +29,18 @@ import (
"github.com/google/cayley/quad/cquads" "github.com/google/cayley/quad/cquads"
) )
func ParseJsonToTripleList(jsonBody []byte) ([]quad.Quad, error) { func ParseJsonToQuadList(jsonBody []byte) ([]quad.Quad, error) {
var tripleList []quad.Quad var quads []quad.Quad
err := json.Unmarshal(jsonBody, &tripleList) err := json.Unmarshal(jsonBody, &quads)
if err != nil { if err != nil {
return nil, err return nil, err
} }
for i, t := range tripleList { for i, q := range quads {
if !t.IsValid() { if !q.IsValid() {
return nil, fmt.Errorf("Invalid triple at index %d. %s", i, t) return nil, fmt.Errorf("Invalid quad at index %d. %s", i, q)
} }
} }
return tripleList, nil return quads, nil
} }
func (api *Api) ServeV1Write(w http.ResponseWriter, r *http.Request, _ httprouter.Params) int { func (api *Api) ServeV1Write(w http.ResponseWriter, r *http.Request, _ httprouter.Params) int {
@ -51,12 +51,12 @@ func (api *Api) ServeV1Write(w http.ResponseWriter, r *http.Request, _ httproute
if err != nil { if err != nil {
return FormatJson400(w, err) return FormatJson400(w, err)
} }
tripleList, terr := ParseJsonToTripleList(bodyBytes) quads, err := ParseJsonToQuadList(bodyBytes)
if terr != nil { if err != nil {
return FormatJson400(w, terr) return FormatJson400(w, err)
} }
api.handle.QuadWriter.AddQuadSet(tripleList) api.handle.QuadWriter.AddQuadSet(quads)
fmt.Fprintf(w, "{\"result\": \"Successfully wrote %d triples.\"}", len(tripleList)) fmt.Fprintf(w, "{\"result\": \"Successfully wrote %d quads.\"}", len(quads))
return 200 return 200
} }
@ -103,7 +103,7 @@ func (api *Api) ServeV1WriteNQuad(w http.ResponseWriter, r *http.Request, params
} }
api.handle.QuadWriter.AddQuadSet(block) api.handle.QuadWriter.AddQuadSet(block)
fmt.Fprintf(w, "{\"result\": \"Successfully wrote %d triples.\"}", n) fmt.Fprintf(w, "{\"result\": \"Successfully wrote %d quads.\"}", n)
return 200 return 200
} }
@ -116,15 +116,15 @@ func (api *Api) ServeV1Delete(w http.ResponseWriter, r *http.Request, params htt
if err != nil { if err != nil {
return FormatJson400(w, err) return FormatJson400(w, err)
} }
tripleList, terr := ParseJsonToTripleList(bodyBytes) quads, err := ParseJsonToQuadList(bodyBytes)
if terr != nil { if err != nil {
return FormatJson400(w, terr) return FormatJson400(w, err)
} }
count := 0 count := 0
for _, triple := range tripleList { for _, q := range quads {
api.handle.QuadWriter.RemoveQuad(triple) api.handle.QuadWriter.RemoveQuad(q)
count++ count++
} }
fmt.Fprintf(w, "{\"result\": \"Successfully deleted %d triples.\"}", count) fmt.Fprintf(w, "{\"result\": \"Successfully deleted %d quads.\"}", count)
return 200 return 200
} }

6
quad/cquads/cquads.go
View file

@ -65,14 +65,14 @@ func (dec *Decoder) Unmarshal() (quad.Quad, error) {
} }
dec.line = dec.line[:0] dec.line = dec.line[:0]
} }
triple, err := Parse(string(line)) q, err := Parse(string(line))
if err != nil { if err != nil {
return quad.Quad{}, fmt.Errorf("failed to parse %q: %v", dec.line, err) return quad.Quad{}, fmt.Errorf("failed to parse %q: %v", dec.line, err)
} }
if !triple.IsValid() { if !q.IsValid() {
return dec.Unmarshal() return dec.Unmarshal()
} }
return triple, nil return q, nil
} }
func unEscape(r []rune, isQuoted, isEscaped bool) string { func unEscape(r []rune, isQuoted, isEscaped bool) string {

12
quad/cquads/cquads_test.go
View file

@ -28,7 +28,7 @@ import (
"github.com/google/cayley/quad" "github.com/google/cayley/quad"
) )
var testNTriples = []struct { var testNQuads = []struct {
message string message string
input string input string
expect quad.Quad expect quad.Quad
@ -596,7 +596,7 @@ var testNTriples = []struct {
} }
func TestParse(t *testing.T) { func TestParse(t *testing.T) {
for _, test := range testNTriples { for _, test := range testNQuads {
got, err := Parse(test.input) got, err := Parse(test.input)
_ = err _ = err
if err != test.err && (err != nil && err.Error() != test.err.Error()) { if err != test.err && (err != nil && err.Error() != test.err.Error()) {
@ -641,20 +641,20 @@ func TestDecoder(t *testing.T) {
dec := NewDecoder(strings.NewReader(document)) dec := NewDecoder(strings.NewReader(document))
var n int var n int
for { for {
triple, err := dec.Unmarshal() q, err := dec.Unmarshal()
if err != nil { if err != nil {
if err != io.EOF { if err != io.EOF {
t.Fatalf("Failed to read document:", err) t.Fatalf("Failed to read document:", err)
} }
break break
} }
if triple.Subject == "" || triple.Predicate == "" || triple.Object == "" { if q.Subject == "" || q.Predicate == "" || q.Object == "" {
t.Errorf("Unexpected triple, got:%v", triple) t.Errorf("Unexpected quad, got:%v", q)
} }
n++ n++
} }
if n != 20 { if n != 20 {
t.Errorf("Unexpected number of triples read, got:%d expect:20", n) t.Errorf("Unexpected number of quads read, got:%d expect:20", n)
} }
} }

6
quad/nquads/nquads.go
View file

@ -65,14 +65,14 @@ func (dec *Decoder) Unmarshal() (quad.Quad, error) {
} }
dec.line = dec.line[:0] dec.line = dec.line[:0]
} }
triple, err := Parse(string(line)) q, err := Parse(string(line))
if err != nil { if err != nil {
return quad.Quad{}, fmt.Errorf("failed to parse %q: %v", dec.line, err) return quad.Quad{}, fmt.Errorf("failed to parse %q: %v", dec.line, err)
} }
if !triple.IsValid() { if !q.IsValid() {
return dec.Unmarshal() return dec.Unmarshal()
} }
return triple, nil return q, nil
} }
func unEscape(r []rune, isEscaped bool) string { func unEscape(r []rune, isEscaped bool) string {

12
quad/nquads/nquads_test.go
View file

@ -28,7 +28,7 @@ import (
"github.com/google/cayley/quad" "github.com/google/cayley/quad"
) )
var testNTriples = []struct { var testNQuads = []struct {
message string message string
input string input string
expect quad.Quad expect quad.Quad
@ -436,7 +436,7 @@ var testNTriples = []struct {
} }
func TestParse(t *testing.T) { func TestParse(t *testing.T) {
for _, test := range testNTriples { for _, test := range testNQuads {
got, err := Parse(test.input) got, err := Parse(test.input)
if err != test.err && (err != nil && err.Error() != test.err.Error()) { if err != test.err && (err != nil && err.Error() != test.err.Error()) {
t.Errorf("Unexpected error when %s: got:%v expect:%v", test.message, err, test.err) t.Errorf("Unexpected error when %s: got:%v expect:%v", test.message, err, test.err)
@ -480,20 +480,20 @@ func TestDecoder(t *testing.T) {
dec := NewDecoder(strings.NewReader(document)) dec := NewDecoder(strings.NewReader(document))
var n int var n int
for { for {
triple, err := dec.Unmarshal() q, err := dec.Unmarshal()
if err != nil { if err != nil {
if err != io.EOF { if err != io.EOF {
t.Fatalf("Failed to read document:", err) t.Fatalf("Failed to read document:", err)
} }
break break
} }
if triple.Subject == "" || triple.Predicate == "" || triple.Object == "" { if q.Subject == "" || q.Predicate == "" || q.Object == "" {
t.Errorf("Unexpected triple, got:%v", triple) t.Errorf("Unexpected quad, got:%v", q)
} }
n++ n++
} }
if n != 20 { if n != 20 {
t.Errorf("Unexpected number of triples read, got:%d expect:20", n) t.Errorf("Unexpected number of quads read, got:%d expect:20", n)
} }
} }

226
quad/nquads/quadfix.go
View file

@ -1,226 +0,0 @@
// +build ignore
package main
import (
"bufio"
"errors"
"fmt"
"io"
"log"
"os"
"strings"
"github.com/google/cayley/graph"
)
func main() {
dec := NewDecoder(os.Stdin)
for {
t, err := dec.Unmarshal()
if err != nil {
if err == io.EOF {
return
}
log.Println(err)
}
if t.Subject[0] == ':' && t.Subject[1] == '/' {
t.Subject = "<" + t.Subject[1:] + ">"
} else {
t.Subject = "_" + t.Subject
}
if t.Object[0] == ':' {
if t.Object[1] == '/' {
t.Object = "<" + t.Object[1:] + ">"
} else {
t.Object = "_" + t.Object
}
} else if t.Object[0] == '/' {
t.Object = "<" + t.Object + ">"
} else {
t.Object = fmt.Sprintf(`%q`, t.Object)
}
fmt.Printf("%s <%s> %s .\n", t.Subject, t.Predicate, t.Object)
}
}
// Historical N-Quads parser code.
// -------------------------------
var (
ErrAbsentSubject = errors.New("nqauds: absent subject")
ErrAbsentPredicate = errors.New("nqauds: absent predicate")
ErrAbsentObject = errors.New("nqauds: absent object")
ErrUnterminated = errors.New("nqauds: unterminated quad")
)
func Parse(str string) (*graph.Triple, error) {
// Skip leading whitespace.
str = trimSpace(str)
// Check for a comment
if str != "" && str[0] == '#' {
return nil, nil
}
sub, remainder := getTripleComponent(str)
if sub == "" {
return nil, ErrAbsentSubject
}
str = trimSpace(remainder)
pred, remainder := getTripleComponent(str)
if pred == "" {
return nil, ErrAbsentPredicate
}
str = trimSpace(remainder)
obj, remainder := getTripleComponent(str)
if obj == "" {
return nil, ErrAbsentObject
}
str = trimSpace(remainder)
prov, remainder := getTripleComponent(str)
str = trimSpace(remainder)
if str != "" && str[0] == '.' {
return &graph.Triple{sub, pred, obj, prov}, nil
}
return nil, ErrUnterminated
}
func isSpace(s uint8) bool {
return s == ' ' || s == '\t' || s == '\r'
}
func trimSpace(str string) string {
i := 0
for i < len(str) && isSpace(str[i]) {
i += 1
}
return str[i:]
}
func getTripleComponent(str string) (head, tail string) {
if len(str) == 0 {
return "", str
}
if str[0] == '<' {
return getUriPart(str[1:])
} else if str[0] == '"' {
return getQuotedPart(str[1:])
} else if str[0] == '.' {
return "", str
} else {
// Technically not part of the spec. But we do it anyway for convenience.
return getUnquotedPart(str)
}
}
func getUriPart(str string) (head, tail string) {
i := 0
for i < len(str) && str[i] != '>' {
i += 1
}
if i == len(str) {
return "", str
}
head = str[0:i]
return head, str[i+1:]
}
func getQuotedPart(str string) (head, tail string) {
var (
i int
start int
)
for i < len(str) && str[i] != '"' {
if str[i] == '\\' {
head += str[start:i]
switch str[i+1] {
case '\\':
head += "\\"
case 'r':
head += "\r"
case 'n':
head += "\n"
case 't':
head += "\t"
case '"':
head += "\""
default:
return "", str
}
i += 2
start = i
continue
}
i += 1
}
if i == len(str) {
return "", str
}
head += str[start:i]
i += 1
switch {
case strings.HasPrefix(str[i:], "^^<"):
// Ignore type, for now
_, tail = getUriPart(str[i+3:])
case str[i] == '@':
_, tail = getUnquotedPart(str[i+1:])
default:
tail = str[i:]
}
return head, tail
}
func getUnquotedPart(str string) (head, tail string) {
var (
i int
initStr = str
start int
)
for i < len(str) && !isSpace(str[i]) {
if str[i] == '"' {
part, remainder := getQuotedPart(str[i+1:])
if part == "" {
return part, initStr
}
head += str[start:i]
str = remainder
i = 0
start = 0
head += part
}
i += 1
}
head += str[start:i]
return head, str[i:]
}
type Decoder struct {
r *bufio.Reader
line []byte
}
func NewDecoder(r io.Reader) *Decoder {
return &Decoder{r: bufio.NewReader(r)}
}
func (dec *Decoder) Unmarshal() (*graph.Triple, error) {
dec.line = dec.line[:0]
for {
l, pre, err := dec.r.ReadLine()
if err != nil {
return nil, err
}
dec.line = append(dec.line, l...)
if !pre {
break
}
}
triple, err := Parse(string(dec.line))
if err != nil {
return nil, fmt.Errorf("failed to parse %q: %v", dec.line, err)
}
if triple == nil {
return dec.Unmarshal()
}
return triple, nil
}

22
quad/quad.go
View file

@ -15,17 +15,17 @@
// Package quad defines quad and triple handling. // Package quad defines quad and triple handling.
package quad package quad
// Defines the struct which makes the TripleStore possible -- the triple. // Defines the struct which makes the QuadStore possible -- the quad.
// //
// At its heart, it consists of three fields -- Subject, Predicate, and Object. // At its heart, it consists of three fields -- Subject, Predicate, and Object.
// Three IDs that relate to each other. That's all there is to it. The triples // Three IDs that relate to each other. That's all there is to it. The quads
// are the links in the graph, and the existence of node IDs is defined by the // are the links in the graph, and the existence of node IDs is defined by the
// fact that some triple in the graph mentions them. // fact that some quad in the graph mentions them.
// //
// This means that a complete representation of the graph is equivalent to a // This means that a complete representation of the graph is equivalent to a
// list of triples. The rest is just indexing for speed. // list of quads. The rest is just indexing for speed.
// //
// Adding fields to the triple is not to be taken lightly. You'll see I mention // Adding fields to the quad is not to be taken lightly. You'll see I mention
// label, but don't as yet use it in any backing store. In general, there // label, but don't as yet use it in any backing store. In general, there
// can be features that can be turned on or off for any store, but I haven't // can be features that can be turned on or off for any store, but I haven't
// decided how to allow/disallow them yet. Another such example would be to add // decided how to allow/disallow them yet. Another such example would be to add
@ -46,7 +46,7 @@ var (
ErrIncomplete = errors.New("incomplete N-Quad") ErrIncomplete = errors.New("incomplete N-Quad")
) )
// Our triple struct, used throughout. // Our quad struct, used throughout.
type Quad struct { type Quad struct {
Subject string `json:"subject"` Subject string `json:"subject"`
Predicate string `json:"predicate"` Predicate string `json:"predicate"`
@ -57,7 +57,7 @@ type Quad struct {
// Direction specifies an edge's type. // Direction specifies an edge's type.
type Direction byte type Direction byte
// List of the valid directions of a triple. // List of the valid directions of a quad.
const ( const (
Any Direction = iota Any Direction = iota
Subject Subject
@ -100,7 +100,7 @@ func (d Direction) String() string {
} }
} }
// Per-field accessor for triples // Per-field accessor for quads.
func (q Quad) Get(d Direction) string { func (q Quad) Get(d Direction) string {
switch d { switch d {
case Subject: case Subject:
@ -116,7 +116,7 @@ func (q Quad) Get(d Direction) string {
} }
} }
// Pretty-prints a triple. // Pretty-prints a quad.
func (q Quad) String() string { func (q Quad) String() string {
return fmt.Sprintf("%s -- %s -> %s", q.Subject, q.Predicate, q.Object) return fmt.Sprintf("%s -- %s -> %s", q.Subject, q.Predicate, q.Object)
} }
@ -125,8 +125,8 @@ func (q Quad) IsValid() bool {
return q.Subject != "" && q.Predicate != "" && q.Object != "" return q.Subject != "" && q.Predicate != "" && q.Object != ""
} }
// Prints a triple in N-Quad format. // Prints a quad in N-Quad format.
func (q Quad) NTriple() string { func (q Quad) NQuad() string {
if q.Label == "" { if q.Label == "" {
//TODO(barakmich): Proper escaping. //TODO(barakmich): Proper escaping.
return fmt.Sprintf("%s %s %s .", q.Subject, q.Predicate, q.Object) return fmt.Sprintf("%s %s %s .", q.Subject, q.Predicate, q.Object)

104
query/gremlin/build_iterator.go
View file

@ -34,11 +34,11 @@ func propertiesOf(obj *otto.Object, name string) []string {
return export.([]string) return export.([]string)
} }
func buildIteratorTree(obj *otto.Object, ts graph.TripleStore) graph.Iterator { func buildIteratorTree(obj *otto.Object, qs graph.QuadStore) graph.Iterator {
if !isVertexChain(obj) { if !isVertexChain(obj) {
return iterator.NewNull() return iterator.NewNull()
} }
return buildIteratorTreeHelper(obj, ts, iterator.NewNull()) return buildIteratorTreeHelper(obj, qs, iterator.NewNull())
} }
func stringsFrom(obj *otto.Object) []string { func stringsFrom(obj *otto.Object) []string {
@ -57,16 +57,16 @@ func stringsFrom(obj *otto.Object) []string {
return output return output
} }
func buildIteratorFromValue(val otto.Value, ts graph.TripleStore) graph.Iterator { func buildIteratorFromValue(val otto.Value, qs graph.QuadStore) graph.Iterator {
if val.IsNull() || val.IsUndefined() { if val.IsNull() || val.IsUndefined() {
return ts.NodesAllIterator() return qs.NodesAllIterator()
} }
if val.IsPrimitive() { if val.IsPrimitive() {
thing, _ := val.Export() thing, _ := val.Export()
switch v := thing.(type) { switch v := thing.(type) {
case string: case string:
it := ts.FixedIterator() it := qs.FixedIterator()
it.Add(ts.ValueOf(v)) it.Add(qs.ValueOf(v))
return it return it
default: default:
glog.Errorln("Trying to build unknown primitive value.") glog.Errorln("Trying to build unknown primitive value.")
@ -74,13 +74,13 @@ func buildIteratorFromValue(val otto.Value, ts graph.TripleStore) graph.Iterator
} }
switch val.Class() { switch val.Class() {
case "Object": case "Object":
return buildIteratorTree(val.Object(), ts) return buildIteratorTree(val.Object(), qs)
case "Array": case "Array":
// Had better be an array of strings // Had better be an array of strings
strings := stringsFrom(val.Object()) strings := stringsFrom(val.Object())
it := ts.FixedIterator() it := qs.FixedIterator()
for _, x := range strings { for _, x := range strings {
it.Add(ts.ValueOf(x)) it.Add(qs.ValueOf(x))
} }
return it return it
case "Number": case "Number":
@ -90,8 +90,8 @@ func buildIteratorFromValue(val otto.Value, ts graph.TripleStore) graph.Iterator
case "Date": case "Date":
fallthrough fallthrough
case "String": case "String":
it := ts.FixedIterator() it := qs.FixedIterator()
it.Add(ts.ValueOf(val.String())) it.Add(qs.ValueOf(val.String()))
return it return it
default: default:
glog.Errorln("Trying to handle unsupported Javascript value.") glog.Errorln("Trying to handle unsupported Javascript value.")
@ -99,7 +99,7 @@ func buildIteratorFromValue(val otto.Value, ts graph.TripleStore) graph.Iterator
} }
} }
func buildInOutIterator(obj *otto.Object, ts graph.TripleStore, base graph.Iterator, isReverse bool) graph.Iterator { func buildInOutIterator(obj *otto.Object, qs graph.QuadStore, base graph.Iterator, isReverse bool) graph.Iterator {
argList, _ := obj.Get("_gremlin_values") argList, _ := obj.Get("_gremlin_values")
if argList.Class() != "GoArray" { if argList.Class() != "GoArray" {
glog.Errorln("How is arglist not an array? Return nothing.", argList.Class()) glog.Errorln("How is arglist not an array? Return nothing.", argList.Class())
@ -110,10 +110,10 @@ func buildInOutIterator(obj *otto.Object, ts graph.TripleStore, base graph.Itera
length, _ := lengthVal.ToInteger() length, _ := lengthVal.ToInteger()
var predicateNodeIterator graph.Iterator var predicateNodeIterator graph.Iterator
if length == 0 { if length == 0 {
predicateNodeIterator = ts.NodesAllIterator() predicateNodeIterator = qs.NodesAllIterator()
} else { } else {
zero, _ := argArray.Get("0") zero, _ := argArray.Get("0")
predicateNodeIterator = buildIteratorFromValue(zero, ts) predicateNodeIterator = buildIteratorFromValue(zero, qs)
} }
if length >= 2 { if length >= 2 {
var tags []string var tags []string
@ -132,14 +132,14 @@ func buildInOutIterator(obj *otto.Object, ts graph.TripleStore, base graph.Itera
if isReverse { if isReverse {
in, out = out, in in, out = out, in
} }
lto := iterator.NewLinksTo(ts, base, in) lto := iterator.NewLinksTo(qs, base, in)
and := iterator.NewAnd() and := iterator.NewAnd()
and.AddSubIterator(iterator.NewLinksTo(ts, predicateNodeIterator, quad.Predicate)) and.AddSubIterator(iterator.NewLinksTo(qs, predicateNodeIterator, quad.Predicate))
and.AddSubIterator(lto) and.AddSubIterator(lto)
return iterator.NewHasA(ts, and, out) return iterator.NewHasA(qs, and, out)
} }
func buildIteratorTreeHelper(obj *otto.Object, ts graph.TripleStore, base graph.Iterator) graph.Iterator { func buildIteratorTreeHelper(obj *otto.Object, qs graph.QuadStore, base graph.Iterator) graph.Iterator {
var it graph.Iterator = base var it graph.Iterator = base
// TODO: Better error handling // TODO: Better error handling
@ -147,7 +147,7 @@ func buildIteratorTreeHelper(obj *otto.Object, ts graph.TripleStore, base graph.
if prev, _ := obj.Get("_gremlin_prev"); !prev.IsObject() { if prev, _ := obj.Get("_gremlin_prev"); !prev.IsObject() {
subIt = base subIt = base
} else { } else {
subIt = buildIteratorTreeHelper(prev.Object(), ts, base) subIt = buildIteratorTreeHelper(prev.Object(), qs, base)
} }
stringArgs := propertiesOf(obj, "string_args") stringArgs := propertiesOf(obj, "string_args")
@ -155,11 +155,11 @@ func buildIteratorTreeHelper(obj *otto.Object, ts graph.TripleStore, base graph.
switch val.String() { switch val.String() {
case "vertex": case "vertex":
if len(stringArgs) == 0 { if len(stringArgs) == 0 {
it = ts.NodesAllIterator() it = qs.NodesAllIterator()
} else { } else {
fixed := ts.FixedIterator() fixed := qs.FixedIterator()
for _, name := range stringArgs { for _, name := range stringArgs {
fixed.Add(ts.ValueOf(name)) fixed.Add(qs.ValueOf(name))
} }
it = fixed it = fixed
} }
@ -169,7 +169,7 @@ func buildIteratorTreeHelper(obj *otto.Object, ts graph.TripleStore, base graph.
it.Tagger().Add(tag) it.Tagger().Add(tag)
} }
case "save": case "save":
all := ts.NodesAllIterator() all := qs.NodesAllIterator()
if len(stringArgs) > 2 || len(stringArgs) == 0 { if len(stringArgs) > 2 || len(stringArgs) == 0 {
return iterator.NewNull() return iterator.NewNull()
} }
@ -178,18 +178,18 @@ func buildIteratorTreeHelper(obj *otto.Object, ts graph.TripleStore, base graph.
} else { } else {
all.Tagger().Add(stringArgs[0]) all.Tagger().Add(stringArgs[0])
} }
predFixed := ts.FixedIterator() predFixed := qs.FixedIterator()
predFixed.Add(ts.ValueOf(stringArgs[0])) predFixed.Add(qs.ValueOf(stringArgs[0]))
subAnd := iterator.NewAnd() subAnd := iterator.NewAnd()
subAnd.AddSubIterator(iterator.NewLinksTo(ts, predFixed, quad.Predicate)) subAnd.AddSubIterator(iterator.NewLinksTo(qs, predFixed, quad.Predicate))
subAnd.AddSubIterator(iterator.NewLinksTo(ts, all, quad.Object)) subAnd.AddSubIterator(iterator.NewLinksTo(qs, all, quad.Object))
hasa := iterator.NewHasA(ts, subAnd, quad.Subject) hasa := iterator.NewHasA(qs, subAnd, quad.Subject)
and := iterator.NewAnd() and := iterator.NewAnd()
and.AddSubIterator(hasa) and.AddSubIterator(hasa)
and.AddSubIterator(subIt) and.AddSubIterator(subIt)
it = and it = and
case "saver": case "saver":
all := ts.NodesAllIterator() all := qs.NodesAllIterator()
if len(stringArgs) > 2 || len(stringArgs) == 0 { if len(stringArgs) > 2 || len(stringArgs) == 0 {
return iterator.NewNull() return iterator.NewNull()
} }
@ -198,30 +198,30 @@ func buildIteratorTreeHelper(obj *otto.Object, ts graph.TripleStore, base graph.
} else { } else {
all.Tagger().Add(stringArgs[0]) all.Tagger().Add(stringArgs[0])
} }
predFixed := ts.FixedIterator() predFixed := qs.FixedIterator()
predFixed.Add(ts.ValueOf(stringArgs[0])) predFixed.Add(qs.ValueOf(stringArgs[0]))
subAnd := iterator.NewAnd() subAnd := iterator.NewAnd()
subAnd.AddSubIterator(iterator.NewLinksTo(ts, predFixed, quad.Predicate)) subAnd.AddSubIterator(iterator.NewLinksTo(qs, predFixed, quad.Predicate))
subAnd.AddSubIterator(iterator.NewLinksTo(ts, all, quad.Subject)) subAnd.AddSubIterator(iterator.NewLinksTo(qs, all, quad.Subject))
hasa := iterator.NewHasA(ts, subAnd, quad.Object) hasa := iterator.NewHasA(qs, subAnd, quad.Object)
and := iterator.NewAnd() and := iterator.NewAnd()
and.AddSubIterator(hasa) and.AddSubIterator(hasa)
and.AddSubIterator(subIt) and.AddSubIterator(subIt)
it = and it = and
case "has": case "has":
fixed := ts.FixedIterator() fixed := qs.FixedIterator()
if len(stringArgs) < 2 { if len(stringArgs) < 2 {
return iterator.NewNull() return iterator.NewNull()
} }
for _, name := range stringArgs[1:] { for _, name := range stringArgs[1:] {
fixed.Add(ts.ValueOf(name)) fixed.Add(qs.ValueOf(name))
} }
predFixed := ts.FixedIterator() predFixed := qs.FixedIterator()
predFixed.Add(ts.ValueOf(stringArgs[0])) predFixed.Add(qs.ValueOf(stringArgs[0]))
subAnd := iterator.NewAnd() subAnd := iterator.NewAnd()
subAnd.AddSubIterator(iterator.NewLinksTo(ts, predFixed, quad.Predicate)) subAnd.AddSubIterator(iterator.NewLinksTo(qs, predFixed, quad.Predicate))
subAnd.AddSubIterator(iterator.NewLinksTo(ts, fixed, quad.Object)) subAnd.AddSubIterator(iterator.NewLinksTo(qs, fixed, quad.Object))
hasa := iterator.NewHasA(ts, subAnd, quad.Subject) hasa := iterator.NewHasA(qs, subAnd, quad.Subject)
and := iterator.NewAnd() and := iterator.NewAnd()
and.AddSubIterator(hasa) and.AddSubIterator(hasa)
and.AddSubIterator(subIt) and.AddSubIterator(subIt)
@ -234,7 +234,7 @@ func buildIteratorTreeHelper(obj *otto.Object, ts graph.TripleStore, base graph.
if !isVertexChain(firstArg.Object()) { if !isVertexChain(firstArg.Object()) {
return iterator.NewNull() return iterator.NewNull()
} }
argIt := buildIteratorTree(firstArg.Object(), ts) argIt := buildIteratorTree(firstArg.Object(), qs)
and := iterator.NewAnd() and := iterator.NewAnd()
and.AddSubIterator(subIt) and.AddSubIterator(subIt)
@ -242,15 +242,15 @@ func buildIteratorTreeHelper(obj *otto.Object, ts graph.TripleStore, base graph.
it = and it = and
case "back": case "back":
arg, _ := obj.Get("_gremlin_back_chain") arg, _ := obj.Get("_gremlin_back_chain")
argIt := buildIteratorTree(arg.Object(), ts) argIt := buildIteratorTree(arg.Object(), qs)
and := iterator.NewAnd() and := iterator.NewAnd()
and.AddSubIterator(subIt) and.AddSubIterator(subIt)
and.AddSubIterator(argIt) and.AddSubIterator(argIt)
it = and it = and
case "is": case "is":
fixed := ts.FixedIterator() fixed := qs.FixedIterator()
for _, name := range stringArgs { for _, name := range stringArgs {
fixed.Add(ts.ValueOf(name)) fixed.Add(qs.ValueOf(name))
} }
and := iterator.NewAnd() and := iterator.NewAnd()
and.AddSubIterator(fixed) and.AddSubIterator(fixed)
@ -262,7 +262,7 @@ func buildIteratorTreeHelper(obj *otto.Object, ts graph.TripleStore, base graph.
if !isVertexChain(firstArg.Object()) { if !isVertexChain(firstArg.Object()) {
return iterator.NewNull() return iterator.NewNull()
} }
argIt := buildIteratorTree(firstArg.Object(), ts) argIt := buildIteratorTree(firstArg.Object(), qs)
or := iterator.NewOr() or := iterator.NewOr()
or.AddSubIterator(subIt) or.AddSubIterator(subIt)
@ -272,15 +272,15 @@ func buildIteratorTreeHelper(obj *otto.Object, ts graph.TripleStore, base graph.
// Hardly the most efficient pattern, but the most general. // Hardly the most efficient pattern, but the most general.
// Worth looking into an Optimize() optimization here. // Worth looking into an Optimize() optimization here.
clone := subIt.Clone() clone := subIt.Clone()
it1 := buildInOutIterator(obj, ts, subIt, false) it1 := buildInOutIterator(obj, qs, subIt, false)
it2 := buildInOutIterator(obj, ts, clone, true) it2 := buildInOutIterator(obj, qs, clone, true)
or := iterator.NewOr() or := iterator.NewOr()
or.AddSubIterator(it1) or.AddSubIterator(it1)
or.AddSubIterator(it2) or.AddSubIterator(it2)
it = or it = or
case "out": case "out":
it = buildInOutIterator(obj, ts, subIt, false) it = buildInOutIterator(obj, qs, subIt, false)
case "follow": case "follow":
// Follow a morphism // Follow a morphism
arg, _ := obj.Get("_gremlin_values") arg, _ := obj.Get("_gremlin_values")
@ -288,16 +288,16 @@ func buildIteratorTreeHelper(obj *otto.Object, ts graph.TripleStore, base graph.
if isVertexChain(firstArg.Object()) { if isVertexChain(firstArg.Object()) {
return iterator.NewNull() return iterator.NewNull()
} }
it = buildIteratorTreeHelper(firstArg.Object(), ts, subIt) it = buildIteratorTreeHelper(firstArg.Object(), qs, subIt)
case "followr": case "followr":
// Follow a morphism // Follow a morphism
arg, _ := obj.Get("_gremlin_followr") arg, _ := obj.Get("_gremlin_followr")
if isVertexChain(arg.Object()) { if isVertexChain(arg.Object()) {
return iterator.NewNull() return iterator.NewNull()
} }
it = buildIteratorTreeHelper(arg.Object(), ts, subIt) it = buildIteratorTreeHelper(arg.Object(), qs, subIt)
case "in": case "in":
it = buildInOutIterator(obj, ts, subIt, true) it = buildInOutIterator(obj, qs, subIt, true)
} }
return it return it
} }

6
query/gremlin/environ.go
View file

@ -26,7 +26,7 @@ import (
) )
type worker struct { type worker struct {
ts graph.TripleStore qs graph.QuadStore
env *otto.Otto env *otto.Otto
sync.Mutex sync.Mutex
@ -39,10 +39,10 @@ type worker struct {
kill <-chan struct{} kill <-chan struct{}
} }
func newWorker(ts graph.TripleStore) *worker { func newWorker(qs graph.QuadStore) *worker {
env := otto.New() env := otto.New()
wk := &worker{ wk := &worker{
ts: ts, qs: qs,
env: env, env: env,
limit: -1, limit: -1,
} }

16
query/gremlin/finals.go
View file

@ -39,7 +39,7 @@ func (wk *worker) embedFinals(env *otto.Otto, obj *otto.Object) {
func (wk *worker) allFunc(env *otto.Otto, obj *otto.Object) func(otto.FunctionCall) otto.Value { func (wk *worker) allFunc(env *otto.Otto, obj *otto.Object) func(otto.FunctionCall) otto.Value {
return func(call otto.FunctionCall) otto.Value { return func(call otto.FunctionCall) otto.Value {
it := buildIteratorTree(obj, wk.ts) it := buildIteratorTree(obj, wk.qs)
it.Tagger().Add(TopResultTag) it.Tagger().Add(TopResultTag)
wk.limit = -1 wk.limit = -1
wk.count = 0 wk.count = 0
@ -52,7 +52,7 @@ func (wk *worker) limitFunc(env *otto.Otto, obj *otto.Object) func(otto.Function
return func(call otto.FunctionCall) otto.Value { return func(call otto.FunctionCall) otto.Value {
if len(call.ArgumentList) > 0 { if len(call.ArgumentList) > 0 {
limitVal, _ := call.Argument(0).ToInteger() limitVal, _ := call.Argument(0).ToInteger()
it := buildIteratorTree(obj, wk.ts) it := buildIteratorTree(obj, wk.qs)
it.Tagger().Add(TopResultTag) it.Tagger().Add(TopResultTag)
wk.limit = int(limitVal) wk.limit = int(limitVal)
wk.count = 0 wk.count = 0
@ -64,7 +64,7 @@ func (wk *worker) limitFunc(env *otto.Otto, obj *otto.Object) func(otto.Function
func (wk *worker) toArrayFunc(env *otto.Otto, obj *otto.Object, withTags bool) func(otto.FunctionCall) otto.Value { func (wk *worker) toArrayFunc(env *otto.Otto, obj *otto.Object, withTags bool) func(otto.FunctionCall) otto.Value {
return func(call otto.FunctionCall) otto.Value { return func(call otto.FunctionCall) otto.Value {
it := buildIteratorTree(obj, wk.ts) it := buildIteratorTree(obj, wk.qs)
it.Tagger().Add(TopResultTag) it.Tagger().Add(TopResultTag)
limit := -1 limit := -1
if len(call.ArgumentList) > 0 { if len(call.ArgumentList) > 0 {
@ -91,7 +91,7 @@ func (wk *worker) toArrayFunc(env *otto.Otto, obj *otto.Object, withTags bool) f
func (wk *worker) toValueFunc(env *otto.Otto, obj *otto.Object, withTags bool) func(otto.FunctionCall) otto.Value { func (wk *worker) toValueFunc(env *otto.Otto, obj *otto.Object, withTags bool) func(otto.FunctionCall) otto.Value {
return func(call otto.FunctionCall) otto.Value { return func(call otto.FunctionCall) otto.Value {
it := buildIteratorTree(obj, wk.ts) it := buildIteratorTree(obj, wk.qs)
it.Tagger().Add(TopResultTag) it.Tagger().Add(TopResultTag)
limit := 1 limit := 1
var val otto.Value var val otto.Value
@ -120,7 +120,7 @@ func (wk *worker) toValueFunc(env *otto.Otto, obj *otto.Object, withTags bool) f
func (wk *worker) mapFunc(env *otto.Otto, obj *otto.Object) func(otto.FunctionCall) otto.Value { func (wk *worker) mapFunc(env *otto.Otto, obj *otto.Object) func(otto.FunctionCall) otto.Value {
return func(call otto.FunctionCall) otto.Value { return func(call otto.FunctionCall) otto.Value {
it := buildIteratorTree(obj, wk.ts) it := buildIteratorTree(obj, wk.qs)
it.Tagger().Add(TopResultTag) it.Tagger().Add(TopResultTag)
limit := -1 limit := -1
if len(call.ArgumentList) == 0 { if len(call.ArgumentList) == 0 {
@ -139,7 +139,7 @@ func (wk *worker) mapFunc(env *otto.Otto, obj *otto.Object) func(otto.FunctionCa
func (wk *worker) tagsToValueMap(m map[string]graph.Value) map[string]string { func (wk *worker) tagsToValueMap(m map[string]graph.Value) map[string]string {
outputMap := make(map[string]string) outputMap := make(map[string]string)
for k, v := range m { for k, v := range m {
outputMap[k] = wk.ts.NameOf(v) outputMap[k] = wk.qs.NameOf(v)
} }
return outputMap return outputMap
} }
@ -196,7 +196,7 @@ func (wk *worker) runIteratorToArrayNoTags(it graph.Iterator, limit int) []strin
if !graph.Next(it) { if !graph.Next(it) {
break break
} }
output = append(output, wk.ts.NameOf(it.Result())) output = append(output, wk.qs.NameOf(it.Result()))
n++ n++
if limit >= 0 && n >= limit { if limit >= 0 && n >= limit {
break break
@ -269,7 +269,7 @@ func (wk *worker) send(r *Result) bool {
func (wk *worker) runIterator(it graph.Iterator) { func (wk *worker) runIterator(it graph.Iterator) {
if wk.wantShape() { if wk.wantShape() {
iterator.OutputQueryShapeForIterator(it, wk.ts, wk.shape) iterator.OutputQueryShapeForIterator(it, wk.qs, wk.shape)
return return
} }
it, _ = it.Optimize() it, _ = it.Optimize()

8
query/gremlin/gremlin_test.go
View file

@ -54,12 +54,12 @@ var simpleGraph = []quad.Quad{
} }
func makeTestSession(data []quad.Quad) *Session { func makeTestSession(data []quad.Quad) *Session {
ts, _ := graph.NewTripleStore("memstore", "", nil) qs, _ := graph.NewQuadStore("memstore", "", nil)
w, _ := graph.NewQuadWriter("single", ts, nil) w, _ := graph.NewQuadWriter("single", qs, nil)
for _, t := range data { for _, t := range data {
w.AddQuad(t) w.AddQuad(t)
} }
return NewSession(ts, -1, false) return NewSession(qs, -1, false)
} }
var testQueries = []struct { var testQueries = []struct {
@ -259,7 +259,7 @@ func runQueryGetTag(g []quad.Quad, query string, tag string) []string {
if data.val == nil { if data.val == nil {
val := data.actualResults[tag] val := data.actualResults[tag]
if val != nil { if val != nil {
results = append(results, js.ts.NameOf(val)) results = append(results, js.qs.NameOf(val))
} }
} }
} }

12
query/gremlin/session.go
View file

@ -30,7 +30,7 @@ import (
var ErrKillTimeout = errors.New("query timed out") var ErrKillTimeout = errors.New("query timed out")
type Session struct { type Session struct {
ts graph.TripleStore qs graph.QuadStore
wk *worker wk *worker
script *otto.Script script *otto.Script
@ -45,10 +45,10 @@ type Session struct {
err error err error
} }
func NewSession(ts graph.TripleStore, timeout time.Duration, persist bool) *Session { func NewSession(qs graph.QuadStore, timeout time.Duration, persist bool) *Session {
g := Session{ g := Session{
ts: ts, qs: qs,
wk: newWorker(ts), wk: newWorker(qs),
timeout: timeout, timeout: timeout,
} }
if persist { if persist {
@ -185,7 +185,7 @@ func (s *Session) ToText(result interface{}) string {
if k == "$_" { if k == "$_" {
continue continue
} }
out += fmt.Sprintf("%s : %s\n", k, s.ts.NameOf(tags[k])) out += fmt.Sprintf("%s : %s\n", k, s.qs.NameOf(tags[k]))
} }
} else { } else {
if data.val.IsObject() { if data.val.IsObject() {
@ -217,7 +217,7 @@ func (s *Session) BuildJson(result interface{}) {
} }
sort.Strings(tagKeys) sort.Strings(tagKeys)
for _, k := range tagKeys { for _, k := range tagKeys {
obj[k] = s.ts.NameOf(tags[k]) obj[k] = s.qs.NameOf(tags[k])
} }
s.dataOutput = append(s.dataOutput, obj) s.dataOutput = append(s.dataOutput, obj)
} else { } else {

22
query/mql/build_iterator.go
View file

@ -27,13 +27,13 @@ import (
) )
func (q *Query) buildFixed(s string) graph.Iterator { func (q *Query) buildFixed(s string) graph.Iterator {
f := q.ses.ts.FixedIterator() f := q.ses.qs.FixedIterator()
f.Add(q.ses.ts.ValueOf(s)) f.Add(q.ses.qs.ValueOf(s))
return f return f
} }
func (q *Query) buildResultIterator(path Path) graph.Iterator { func (q *Query) buildResultIterator(path Path) graph.Iterator {
all := q.ses.ts.NodesAllIterator() all := q.ses.qs.NodesAllIterator()
all.Tagger().Add(string(path)) all.Tagger().Add(string(path))
return all return all
} }
@ -104,7 +104,7 @@ func (q *Query) buildIteratorTreeInternal(query interface{}, path Path) (it grap
func (q *Query) buildIteratorTreeMapInternal(query map[string]interface{}, path Path) (graph.Iterator, error) { func (q *Query) buildIteratorTreeMapInternal(query map[string]interface{}, path Path) (graph.Iterator, error) {
it := iterator.NewAnd() it := iterator.NewAnd()
it.AddSubIterator(q.ses.ts.NodesAllIterator()) it.AddSubIterator(q.ses.qs.NodesAllIterator())
var err error var err error
err = nil err = nil
outputStructure := make(map[string]interface{}) outputStructure := make(map[string]interface{})
@ -138,18 +138,18 @@ func (q *Query) buildIteratorTreeMapInternal(query map[string]interface{}, path
return nil, err return nil, err
} }
subAnd := iterator.NewAnd() subAnd := iterator.NewAnd()
predFixed := q.ses.ts.FixedIterator() predFixed := q.ses.qs.FixedIterator()
predFixed.Add(q.ses.ts.ValueOf(pred)) predFixed.Add(q.ses.qs.ValueOf(pred))
subAnd.AddSubIterator(iterator.NewLinksTo(q.ses.ts, predFixed, quad.Predicate)) subAnd.AddSubIterator(iterator.NewLinksTo(q.ses.qs, predFixed, quad.Predicate))
if reverse { if reverse {
lto := iterator.NewLinksTo(q.ses.ts, builtIt, quad.Subject) lto := iterator.NewLinksTo(q.ses.qs, builtIt, quad.Subject)
subAnd.AddSubIterator(lto) subAnd.AddSubIterator(lto)
hasa := iterator.NewHasA(q.ses.ts, subAnd, quad.Object) hasa := iterator.NewHasA(q.ses.qs, subAnd, quad.Object)
subit = hasa subit = hasa
} else { } else {
lto := iterator.NewLinksTo(q.ses.ts, builtIt, quad.Object) lto := iterator.NewLinksTo(q.ses.qs, builtIt, quad.Object)
subAnd.AddSubIterator(lto) subAnd.AddSubIterator(lto)
hasa := iterator.NewHasA(q.ses.ts, subAnd, quad.Subject) hasa := iterator.NewHasA(q.ses.qs, subAnd, quad.Subject)
subit = hasa subit = hasa
} }
} }

2
query/mql/fill.go
View file

@ -27,7 +27,7 @@ func (q *Query) treeifyResult(tags map[string]graph.Value) map[ResultPath]string
if v == nil { if v == nil {
continue continue
} }
results[Path(k)] = q.ses.ts.NameOf(v) results[Path(k)] = q.ses.qs.NameOf(v)
} }
resultPaths := make(map[ResultPath]string) resultPaths := make(map[ResultPath]string)
for k, v := range results { for k, v := range results {

6
query/mql/mql_test.go
View file

@ -53,12 +53,12 @@ var simpleGraph = []quad.Quad{
} }
func makeTestSession(data []quad.Quad) *Session { func makeTestSession(data []quad.Quad) *Session {
ts, _ := graph.NewTripleStore("memstore", "", nil) qs, _ := graph.NewQuadStore("memstore", "", nil)
w, _ := graph.NewQuadWriter("single", ts, nil) w, _ := graph.NewQuadWriter("single", qs, nil)
for _, t := range data { for _, t := range data {
w.AddQuad(t) w.AddQuad(t)
} }
return NewSession(ts) return NewSession(qs)
} }
var testQueries = []struct { var testQueries = []struct {

10
query/mql/session.go
View file

@ -27,14 +27,14 @@ import (
) )
type Session struct { type Session struct {
ts graph.TripleStore qs graph.QuadStore
currentQuery *Query currentQuery *Query
debug bool debug bool
} }
func NewSession(ts graph.TripleStore) *Session { func NewSession(qs graph.QuadStore) *Session {
var m Session var m Session
m.ts = ts m.qs = qs
return &m return &m
} }
@ -52,7 +52,7 @@ func (m *Session) GetQuery(input string, output_struct chan map[string]interface
m.currentQuery = NewQuery(m) m.currentQuery = NewQuery(m)
m.currentQuery.BuildIteratorTree(mqlQuery) m.currentQuery.BuildIteratorTree(mqlQuery)
output := make(map[string]interface{}) output := make(map[string]interface{})
iterator.OutputQueryShapeForIterator(m.currentQuery.it, m.ts, output) iterator.OutputQueryShapeForIterator(m.currentQuery.it, m.qs, output)
nodes := output["nodes"].([]iterator.Node) nodes := output["nodes"].([]iterator.Node)
new_nodes := make([]iterator.Node, 0) new_nodes := make([]iterator.Node, 0)
for _, n := range nodes { for _, n := range nodes {
@ -118,7 +118,7 @@ func (s *Session) ToText(result interface{}) string {
if k == "$_" { if k == "$_" {
continue continue
} }
out += fmt.Sprintf("%s : %s\n", k, s.ts.NameOf(tags[k])) out += fmt.Sprintf("%s : %s\n", k, s.qs.NameOf(tags[k]))
} }
return out return out
} }

28
query/sexp/parser.go
View file

@ -22,9 +22,9 @@ import (
"github.com/google/cayley/quad" "github.com/google/cayley/quad"
) )
func BuildIteratorTreeForQuery(ts graph.TripleStore, query string) graph.Iterator { func BuildIteratorTreeForQuery(qs graph.QuadStore, query string) graph.Iterator {
tree := parseQuery(query) tree := parseQuery(query)
return buildIteratorTree(tree, ts) return buildIteratorTree(tree, qs)
} }
func ParseString(input string) string { func ParseString(input string) string {
@ -181,15 +181,15 @@ func getIdentString(tree *peg.ExpressionTree) string {
return out return out
} }
func buildIteratorTree(tree *peg.ExpressionTree, ts graph.TripleStore) graph.Iterator { func buildIteratorTree(tree *peg.ExpressionTree, qs graph.QuadStore) graph.Iterator {
switch tree.Name { switch tree.Name {
case "Start": case "Start":
return buildIteratorTree(tree.Children[0], ts) return buildIteratorTree(tree.Children[0], qs)
case "NodeIdentifier": case "NodeIdentifier":
var out graph.Iterator var out graph.Iterator
nodeID := getIdentString(tree) nodeID := getIdentString(tree)
if tree.Children[0].Name == "Variable" { if tree.Children[0].Name == "Variable" {
allIt := ts.NodesAllIterator() allIt := qs.NodesAllIterator()
allIt.Tagger().Add(nodeID) allIt.Tagger().Add(nodeID)
out = allIt out = allIt
} else { } else {
@ -197,8 +197,8 @@ func buildIteratorTree(tree *peg.ExpressionTree, ts graph.TripleStore) graph.Ite
if tree.Children[0].Children[0].Name == "ColonIdentifier" { if tree.Children[0].Children[0].Name == "ColonIdentifier" {
n = nodeID[1:] n = nodeID[1:]
} }
fixed := ts.FixedIterator() fixed := qs.FixedIterator()
fixed.Add(ts.ValueOf(n)) fixed.Add(qs.ValueOf(n))
out = fixed out = fixed
} }
return out return out
@ -208,8 +208,8 @@ func buildIteratorTree(tree *peg.ExpressionTree, ts graph.TripleStore) graph.Ite
//Taken care of below //Taken care of below
i++ i++
} }
it := buildIteratorTree(tree.Children[i], ts) it := buildIteratorTree(tree.Children[i], qs)
lto := iterator.NewLinksTo(ts, it, quad.Predicate) lto := iterator.NewLinksTo(qs, it, quad.Predicate)
return lto return lto
case "RootConstraint": case "RootConstraint":
constraintCount := 0 constraintCount := 0
@ -219,7 +219,7 @@ func buildIteratorTree(tree *peg.ExpressionTree, ts graph.TripleStore) graph.Ite
case "NodeIdentifier": case "NodeIdentifier":
fallthrough fallthrough
case "Constraint": case "Constraint":
it := buildIteratorTree(c, ts) it := buildIteratorTree(c, qs)
and.AddSubIterator(it) and.AddSubIterator(it)
constraintCount++ constraintCount++
continue continue
@ -241,7 +241,7 @@ func buildIteratorTree(tree *peg.ExpressionTree, ts graph.TripleStore) graph.Ite
topLevelDir = quad.Object topLevelDir = quad.Object
subItDir = quad.Subject subItDir = quad.Subject
} }
it := buildIteratorTree(c, ts) it := buildIteratorTree(c, qs)
subAnd.AddSubIterator(it) subAnd.AddSubIterator(it)
continue continue
case "PredicateKeyword": case "PredicateKeyword":
@ -252,15 +252,15 @@ func buildIteratorTree(tree *peg.ExpressionTree, ts graph.TripleStore) graph.Ite
case "NodeIdentifier": case "NodeIdentifier":
fallthrough fallthrough
case "RootConstraint": case "RootConstraint":
it := buildIteratorTree(c, ts) it := buildIteratorTree(c, qs)
l := iterator.NewLinksTo(ts, it, subItDir) l := iterator.NewLinksTo(qs, it, subItDir)
subAnd.AddSubIterator(l) subAnd.AddSubIterator(l)
continue continue
default: default:
continue continue
} }
} }
hasa = iterator.NewHasA(ts, subAnd, topLevelDir) hasa = iterator.NewHasA(qs, subAnd, topLevelDir)
if isOptional { if isOptional {
optional := iterator.NewOptional(hasa) optional := iterator.NewOptional(hasa)
return optional return optional

44
query/sexp/parser_test.go
View file

@ -39,14 +39,14 @@ var testQueries = []struct {
expect string expect string
}{ }{
{ {
message: "get a single triple linkage", message: "get a single quad linkage",
add: quad.Quad{"i", "can", "win", ""}, add: quad.Quad{"i", "can", "win", ""},
query: "($a (:can \"win\"))", query: "($a (:can \"win\"))",
typ: graph.And, typ: graph.And,
expect: "i", expect: "i",
}, },
{ {
message: "get a single triple linkage", message: "get a single quad linkage",
add: quad.Quad{"i", "can", "win", ""}, add: quad.Quad{"i", "can", "win", ""},
query: "(\"i\" (:can $a))", query: "(\"i\" (:can $a))",
typ: graph.And, typ: graph.And,
@ -55,9 +55,9 @@ var testQueries = []struct {
} }
func TestMemstoreBackedSexp(t *testing.T) { func TestMemstoreBackedSexp(t *testing.T) {
ts, _ := graph.NewTripleStore("memstore", "", nil) qs, _ := graph.NewQuadStore("memstore", "", nil)
w, _ := graph.NewQuadWriter("single", ts, nil) w, _ := graph.NewQuadWriter("single", qs, nil)
it := BuildIteratorTreeForQuery(ts, "()") it := BuildIteratorTreeForQuery(qs, "()")
if it.Type() != graph.Null { if it.Type() != graph.Null {
t.Errorf(`Incorrect type for empty query, got:%q expect: "null"`, it.Type()) t.Errorf(`Incorrect type for empty query, got:%q expect: "null"`, it.Type())
} }
@ -65,7 +65,7 @@ func TestMemstoreBackedSexp(t *testing.T) {
if test.add.IsValid() { if test.add.IsValid() {
w.AddQuad(test.add) w.AddQuad(test.add)
} }
it := BuildIteratorTreeForQuery(ts, test.query) it := BuildIteratorTreeForQuery(qs, test.query)
if it.Type() != test.typ { if it.Type() != test.typ {
t.Errorf("Incorrect type for %s, got:%q expect %q", test.message, it.Type(), test.expect) t.Errorf("Incorrect type for %s, got:%q expect %q", test.message, it.Type(), test.expect)
} }
@ -73,21 +73,21 @@ func TestMemstoreBackedSexp(t *testing.T) {
t.Errorf("Failed to %s", test.message) t.Errorf("Failed to %s", test.message)
} }
got := it.Result() got := it.Result()
if expect := ts.ValueOf(test.expect); got != expect { if expect := qs.ValueOf(test.expect); got != expect {
t.Errorf("Incorrect result for %s, got:%v expect %v", test.message, got, expect) t.Errorf("Incorrect result for %s, got:%v expect %v", test.message, got, expect)
} }
} }
} }
func TestTreeConstraintParse(t *testing.T) { func TestTreeConstraintParse(t *testing.T) {
ts, _ := graph.NewTripleStore("memstore", "", nil) qs, _ := graph.NewQuadStore("memstore", "", nil)
w, _ := graph.NewQuadWriter("single", ts, nil) w, _ := graph.NewQuadWriter("single", qs, nil)
w.AddQuad(quad.Quad{"i", "like", "food", ""}) w.AddQuad(quad.Quad{"i", "like", "food", ""})
w.AddQuad(quad.Quad{"food", "is", "good", ""}) w.AddQuad(quad.Quad{"food", "is", "good", ""})
query := "(\"i\"\n" + query := "(\"i\"\n" +
"(:like\n" + "(:like\n" +
"($a (:is :good))))" "($a (:is :good))))"
it := BuildIteratorTreeForQuery(ts, query) it := BuildIteratorTreeForQuery(qs, query)
if it.Type() != graph.And { if it.Type() != graph.And {
t.Error("Odd iterator tree. Got: %s", it.DebugString(0)) t.Error("Odd iterator tree. Got: %s", it.DebugString(0))
} }
@ -95,34 +95,34 @@ func TestTreeConstraintParse(t *testing.T) {
t.Error("Got no results") t.Error("Got no results")
} }
out := it.Result() out := it.Result()
if out != ts.ValueOf("i") { if out != qs.ValueOf("i") {
t.Errorf("Got %d, expected %d", out, ts.ValueOf("i")) t.Errorf("Got %d, expected %d", out, qs.ValueOf("i"))
} }
} }
func TestTreeConstraintTagParse(t *testing.T) { func TestTreeConstraintTagParse(t *testing.T) {
ts, _ := graph.NewTripleStore("memstore", "", nil) qs, _ := graph.NewQuadStore("memstore", "", nil)
w, _ := graph.NewQuadWriter("single", ts, nil) w, _ := graph.NewQuadWriter("single", qs, nil)
w.AddQuad(quad.Quad{"i", "like", "food", ""}) w.AddQuad(quad.Quad{"i", "like", "food", ""})
w.AddQuad(quad.Quad{"food", "is", "good", ""}) w.AddQuad(quad.Quad{"food", "is", "good", ""})
query := "(\"i\"\n" + query := "(\"i\"\n" +
"(:like\n" + "(:like\n" +
"($a (:is :good))))" "($a (:is :good))))"
it := BuildIteratorTreeForQuery(ts, query) it := BuildIteratorTreeForQuery(qs, query)
if !graph.Next(it) { if !graph.Next(it) {
t.Error("Got no results") t.Error("Got no results")
} }
tags := make(map[string]graph.Value) tags := make(map[string]graph.Value)
it.TagResults(tags) it.TagResults(tags)
if ts.NameOf(tags["$a"]) != "food" { if qs.NameOf(tags["$a"]) != "food" {
t.Errorf("Got %s, expected food", ts.NameOf(tags["$a"])) t.Errorf("Got %s, expected food", qs.NameOf(tags["$a"]))
} }
} }
func TestMultipleConstraintParse(t *testing.T) { func TestMultipleConstraintParse(t *testing.T) {
ts, _ := graph.NewTripleStore("memstore", "", nil) qs, _ := graph.NewQuadStore("memstore", "", nil)
w, _ := graph.NewQuadWriter("single", ts, nil) w, _ := graph.NewQuadWriter("single", qs, nil)
for _, tv := range []quad.Quad{ for _, tv := range []quad.Quad{
{"i", "like", "food", ""}, {"i", "like", "food", ""},
{"i", "like", "beer", ""}, {"i", "like", "beer", ""},
@ -135,7 +135,7 @@ func TestMultipleConstraintParse(t *testing.T) {
(:like :beer) (:like :beer)
(:like "food") (:like "food")
)` )`
it := BuildIteratorTreeForQuery(ts, query) it := BuildIteratorTreeForQuery(qs, query)
if it.Type() != graph.And { if it.Type() != graph.And {
t.Error("Odd iterator tree. Got: %s", it.DebugString(0)) t.Error("Odd iterator tree. Got: %s", it.DebugString(0))
} }
@ -143,8 +143,8 @@ func TestMultipleConstraintParse(t *testing.T) {
t.Error("Got no results") t.Error("Got no results")
} }
out := it.Result() out := it.Result()
if out != ts.ValueOf("i") { if out != qs.ValueOf("i") {
t.Errorf("Got %d, expected %d", out, ts.ValueOf("i")) t.Errorf("Got %d, expected %d", out, qs.ValueOf("i"))
} }
if graph.Next(it) { if graph.Next(it) {
t.Error("Too many results") t.Error("Too many results")

10
query/sexp/session.go
View file

@ -26,13 +26,13 @@ import (
) )
type Session struct { type Session struct {
ts graph.TripleStore qs graph.QuadStore
debug bool debug bool
} }
func NewSession(inputTripleStore graph.TripleStore) *Session { func NewSession(qs graph.QuadStore) *Session {
var s Session var s Session
s.ts = inputTripleStore s.qs = qs
return &s return &s
} }
@ -67,7 +67,7 @@ func (s *Session) InputParses(input string) (query.ParseResult, error) {
} }
func (s *Session) ExecInput(input string, out chan interface{}, limit int) { func (s *Session) ExecInput(input string, out chan interface{}, limit int) {
it := BuildIteratorTreeForQuery(s.ts, input) it := BuildIteratorTreeForQuery(s.qs, input)
newIt, changed := it.Optimize() newIt, changed := it.Optimize()
if changed { if changed {
it = newIt it = newIt
@ -112,7 +112,7 @@ func (s *Session) ToText(result interface{}) string {
if k == "$_" { if k == "$_" {
continue continue
} }
out += fmt.Sprintf("%s : %s\n", k, s.ts.NameOf(tags[k])) out += fmt.Sprintf("%s : %s\n", k, s.qs.NameOf(tags[k]))
} }
return out return out
} }

22
static/js/cayley_write.js
View file

@ -25,7 +25,7 @@ $(function() {
$("#alertBox").addClass("alert-success").fadeIn(300).delay(2000).fadeOut(300).queue(function(){ $(this).removeClass("alert-success")}); $("#alertBox").addClass("alert-success").fadeIn(300).delay(2000).fadeOut(300).queue(function(){ $(this).removeClass("alert-success")});
} }
var checkTriple = function(t) { var checkQuad = function(t) {
if (t.subject == "") { if (t.subject == "") {
alertFail("Need a subject") alertFail("Need a subject")
return false return false
@ -43,19 +43,19 @@ $(function() {
$("#sbWrite").addClass("active"); $("#sbWrite").addClass("active");
$("#add_triple").click(function() { $("#add_quad").click(function() {
var triple = { var quad = {
subject: $("#subject").val(), subject: $("#subject").val(),
predicate: $("#predicate").val(), predicate: $("#predicate").val(),
object: $("#object").val(), object: $("#object").val(),
label: $("#label").val() label: $("#label").val()
} }
if (!checkTriple(triple)) { if (!checkQuad(quad)) {
return return
} }
$.post("/api/v1/write", JSON.stringify([triple])) $.post("/api/v1/write", JSON.stringify([quad]))
.done(function(return_data){ .done(function(return_data){
alertSucceed("Wrote a triple!") alertSucceed("Wrote a quad!")
}) })
.fail(function(jqxhr) { .fail(function(jqxhr) {
var data = $.parseJSON(jqxhr.responseText) var data = $.parseJSON(jqxhr.responseText)
@ -63,19 +63,19 @@ $(function() {
}) })
}) })
$("#delete_triple").click(function() { $("#delete_quad").click(function() {
var triple = { var quad = {
subject: $("#rsubject").val(), subject: $("#rsubject").val(),
predicate: $("#rpredicate").val(), predicate: $("#rpredicate").val(),
object: $("#robject").val(), object: $("#robject").val(),
label: $("#rlabel").val() label: $("#rlabel").val()
} }
if (!checkTriple(triple)) { if (!checkQuad(quad)) {
return return
} }
$.post("/api/v1/delete", JSON.stringify([triple])) $.post("/api/v1/delete", JSON.stringify([quad]))
.done(function(return_data){ .done(function(return_data){
alertSucceed("Deleted a triple!") alertSucceed("Deleted a quad!")
}) })
.fail(function(jqxhr) { .fail(function(jqxhr) {
var data = $.parseJSON(jqxhr.responseText) var data = $.parseJSON(jqxhr.responseText)

2
templates/top_navbar.tmpl
View file

@ -17,7 +17,7 @@
<!-- top navbar --> <!-- top navbar -->
<div class="navbar navbar-fixed-top" role="navigation"> <div class="navbar navbar-fixed-top" role="navigation">
<div id="alertBox" class="alert" > <div id="alertBox" class="alert" >
Wrote a triple! Wrote a quad!
</div> </div>
<div class="container-fluid"> <div class="container-fluid">
<div class="navbar-header"> <div class="navbar-header">

4
templates/write.html
View file

@ -50,7 +50,7 @@
</div> </div>
<div class="row button-row"> <div class="row button-row">
<div class="col-sm-12 col-xs-12"> <div class="col-sm-12 col-xs-12">
<button id="add_triple" type="button" class="btn">Add Triple</button> <button id="add_quad" type="button" class="btn">Add Quad</button>
</div> </div>
</div> </div>
<div class="row"> <div class="row">
@ -64,7 +64,7 @@
</div><!-- /.col-xs-12 main --> </div><!-- /.col-xs-12 main -->
<div class="row button-row"> <div class="row button-row">
<div class="col-sm-12 col-xs-12"> <div class="col-sm-12 col-xs-12">
<button id="delete_triple" type="button" class="btn">Delete Triple</button> <button id="delete_quad" type="button" class="btn">Delete Quad</button>
</div> </div>
</div> </div>
</div> </div>

14
writer/single.go
View file

@ -28,13 +28,13 @@ func init() {
type Single struct { type Single struct {
nextID int64 nextID int64
ts graph.TripleStore qs graph.QuadStore
mut sync.Mutex mut sync.Mutex
} }
func NewSingleReplication(ts graph.TripleStore, opts graph.Options) (graph.QuadWriter, error) { func NewSingleReplication(qs graph.QuadStore, opts graph.Options) (graph.QuadWriter, error) {
horizon := ts.Horizon() horizon := qs.Horizon()
rep := &Single{nextID: horizon + 1, ts: ts} rep := &Single{nextID: horizon + 1, qs: qs}
if horizon <= 0 { if horizon <= 0 {
rep.nextID = 1 rep.nextID = 1
} }
@ -57,7 +57,7 @@ func (s *Single) AddQuad(q quad.Quad) error {
Action: graph.Add, Action: graph.Add,
Timestamp: time.Now(), Timestamp: time.Now(),
} }
return s.ts.ApplyDeltas(deltas) return s.qs.ApplyDeltas(deltas)
} }
func (s *Single) AddQuadSet(set []quad.Quad) error { func (s *Single) AddQuadSet(set []quad.Quad) error {
@ -70,7 +70,7 @@ func (s *Single) AddQuadSet(set []quad.Quad) error {
Timestamp: time.Now(), Timestamp: time.Now(),
} }
} }
s.ts.ApplyDeltas(deltas) s.qs.ApplyDeltas(deltas)
return nil return nil
} }
@ -82,7 +82,7 @@ func (s *Single) RemoveQuad(q quad.Quad) error {
Action: graph.Delete, Action: graph.Delete,
Timestamp: time.Now(), Timestamp: time.Now(),
} }
return s.ts.ApplyDeltas(deltas) return s.qs.ApplyDeltas(deltas)
} }
func (s *Single) Close() error { func (s *Single) Close() error {