Commit graph

242 commits

Author SHA1 Message Date
kortschak
d98ca99974 Merge branch 'master' into b
$ benchcmp gollrb.bench b-gen.bench
benchmark                                   old ns/op       new ns/op	delta
BenchmarkNamePredicate                      1369329         1444990	+5.53%
BenchmarkLargeSetsNoIntersection            72329029        64975716	-10.17%
BenchmarkVeryLargeSetsSmallIntersection     890824761       408784476	-54.11%
BenchmarkHelplessContainsChecker            35314797618     30673240485	-13.14%
BenchmarkNetAndSpeed                        19694146        19486797	-1.05%
BenchmarkKeanuAndNet                        15340756        15317415	-0.15%
BenchmarkKeanuAndSpeed                      17902709        18042030	+0.78%
BenchmarkKeanuOther                         53452058        50984817	-4.62%
BenchmarkKeanuBullockOther                  90827780        86536510	-4.72%

benchmark                                   old allocs     new allocs	delta
BenchmarkNamePredicate                      1339           1339		+0.00%
BenchmarkLargeSetsNoIntersection            22603          22674	+0.31%
BenchmarkVeryLargeSetsSmallIntersection     65787          65860	+0.11%
BenchmarkHelplessContainsChecker            1713541        1713669	+0.01%
BenchmarkNetAndSpeed                        17135          17146	+0.06%
BenchmarkKeanuAndNet                        15802          15802	+0.00%
BenchmarkKeanuAndSpeed                      16397          16396	-0.01%
BenchmarkKeanuOther                         30148          30149	+0.00%
BenchmarkKeanuBullockOther                  35542          35544	+0.01%

benchmark                                   old bytes     new bytes	delta
BenchmarkNamePredicate                      96226         95842		-0.40%
BenchmarkLargeSetsNoIntersection            1165914       119725	+2.69%
BenchmarkVeryLargeSetsSmallIntersection     2760072       2777798	+0.64%
BenchmarkHelplessContainsChecker            84388448      84351168	-0.04%
BenchmarkNetAndSpeed                        1414837       1425752	+0.77%
BenchmarkKeanuAndNet                        1247249       1247453	+0.02%
BenchmarkKeanuAndSpeed                      1275522       1275243	-0.02%
BenchmarkKeanuOther                         2021107       2021497	+0.02%
BenchmarkKeanuBullockOther                  2682243       2683250	+0.04%
2014-08-13 16:01:32 +09:30
1b22203e7a Merge pull request #109 from kortschak/empty
Allow memstore instances to be started empty
2014-08-12 19:42:46 -04:00
ee11a26bc2 Merge pull request #116 from barakmich/issue114
Fix Issue 114 -- And subiterators may move forward on Contains, even if the And ultimately doesn't
2014-08-12 19:32:30 -04:00
50efe31fdd fix the test, remove -short from .travis.yml 2014-08-12 19:26:02 -04:00
b6d966e9b5 fix and iterator, add some logging, and reenable 2014-08-12 16:38:10 -04:00
kortschak
99d44e3228 Add test and make it pass 2014-08-12 10:20:26 +09:30
kortschak
2540ea8f87 Use cznic/b for index store
$ benchcmp gollrb.bench b-gen.bench
benchmark                                   old ns/op       new ns/op	delta
BenchmarkNamePredicate                      1731218         1693373		-2.19%
BenchmarkLargeSetsNoIntersection            81290360        70205277	-13.64%
BenchmarkVeryLargeSetsSmallIntersection     768135620       442906243	-42.34%
BenchmarkHelplessContainsChecker            39477086024     35260603748	-10.68%
BenchmarkNetAndSpeed                        22510637        21587975	-4.10%
BenchmarkKeanuAndNet                        18018886        17795328	-1.24%
BenchmarkKeanuAndSpeed                      20336586        20560228	+1.10%
BenchmarkKeanuOther                         85495040        80718152	-5.59%
BenchmarkKeanuBullockOther                  95457792        83868434	-12.14%

Code gen from $GOPATH/src/github.com/cznic/b:

  make generic \
| sed -e 's/KEY/int64/g' -e 's/VALUE/struct{}/g' \
> $GOPATH/src/github.com/google/cayley/graph/memstore/b/keys.go

key_test.go manually edited.
2014-08-11 20:45:55 +09:30
aad21b0585 Fix aborted materialization 2014-08-10 21:49:57 -04:00
kortschak
0c79c11606 Allow memstore instances to be started empty
This allows easier debugging of web UI problem.
2014-08-09 18:13:44 +09:30
11c3cd17c6 Fix cayley load command 2014-08-09 01:23:23 -04:00
614184bf87 Merge pull request #108 from kortschak/issue106
Use cquads in the web UI data loader
2014-08-09 01:22:16 -04:00
kortschak
9263b05d75 Use cquads in the web UI data loader
Fixes issue #106.
2014-08-09 10:09:43 +09:30
166ebb1a3a Merge pull request #105 from kortschak/persistence
Handle persistence and quad format in cayley
2014-08-08 17:00:28 -04:00
607d0040e2 Merge pull request #104 from kortschak/repl
Use github.com/peterh/liner for REPL lines
2014-08-08 16:44:59 -04:00
kortschak
86bf7e9e6a Remove now-irrelevant failing test 2014-08-08 22:21:42 +09:30
kortschak
4844ef3e58 Add support for remote data sources
Now you can, for example:

cayley http --dbpath="https://github.com/google/cayley/blob/master/30kmoviedata.nq.gz?raw=true"
2014-08-08 22:19:53 +09:30
kortschak
844927ff1f Make decompressor conditional on reader interface 2014-08-08 21:27:17 +09:30
kortschak
088e73a163 Merge {init,open,load}.go into db.go
The functions are too small to justify a single file each. The repl file
is left out as semantically unrelated.
2014-08-08 10:32:58 +09:30
kortschak
979a0c4aee Allow optional strict N-Quad parsing
This puts more of the logic in cayley, but other approaches require that
db knows about quad formats.
2014-08-08 10:28:18 +09:30
kortschak
4c3f5109eb Separate db.Open and db.Load
Updates #82.
2014-08-08 09:52:35 +09:30
kortschak
2d4c07b56d Make db name-literal agnostic
Move the persistence characteristic of the store into the graph registry
and provide an API hook to get that information.

Add error return for init on a non-persistent store.

Updates #35.
2014-08-08 08:36:49 +09:30
kortschak
7265e1d7a1 Use github.com/peterh/liner for REPL lines
This gives us history and line conveniences.
2014-08-07 15:02:30 +09:30
7cd740aa7b Merge pull request #93 from kortschak/nexter
Simplify the Nexter interface
2014-08-06 23:48:46 -04:00
kortschak
191244c40e Sync iteration calls to Nexter interface changes 2014-08-07 07:07:45 +09:30
kortschak
62785d25c2 Merge branch 'master' into nexter
Conflicts:
	graph/leveldb/all_iterator.go
	graph/leveldb/iterator.go
	graph/memstore/triplestore.go
	query/gremlin/finals.go
2014-08-07 06:57:45 +09:30
b52f9726ea update readme 2014-08-06 17:18:12 -04:00
b74cb142f0 Key/Keyer 2014-08-06 16:59:46 -04:00
408103b406 Merge branch 'materializer_iterator' 2014-08-06 15:59:24 -04:00
2b43185117 rename Hashable/Hasher 2014-08-06 15:58:37 -04:00
272829f3e9 Merge pull request #103 from barakmich/materializer_iterator
Fix Materializer iterator
2014-08-06 15:52:36 -04:00
4d9dd42dd8 comment 2014-08-06 15:51:56 -04:00
840c341274 Fix ID Lookup 2014-08-06 15:39:55 -04:00
6f1e46c2d8 Fix hashability for graph.Value 2014-08-06 15:34:30 -04:00
7e7371c3c8 Merge pull request #101 from barakmich/materializer_iterator
Add Materialize Iterator
2014-08-06 14:44:22 -04:00
2bec255b52 Copy refs and comment on Value 2014-08-06 14:37:37 -04:00
cedaa9cdfa Merge branch 'master' into materializer_iterator 2014-08-06 14:31:10 -04:00
9fceb9d4b0 Merge pull request #100 from barakmich/fix_integration
Fix the integration test from crashing
2014-08-06 14:18:36 -04:00
11891f02a9 remove lead-in parsing test 2014-08-06 05:19:46 -04:00
8a08b59cc2 Merge branch 'fix_integration' into materializer_iterator 2014-08-06 05:06:10 -04:00
0c3e0381f3 Fix the integration test from crashing
Sessions are expected to only have one running query (perhaps this is a
bug). So we need to make a new session for each of the benchmark runs,
timing only the running part.
2014-08-06 05:03:20 -04:00
76efc2fcb7 redo data structure for sensibility 2014-08-06 04:07:30 -04:00
d102394836 bounds errors 2014-08-06 03:49:55 -04:00
24f57df859 fix overshoot and optimize better 2014-08-06 03:49:55 -04:00
09244ddd38 materialize implementation and and optimization 2014-08-06 03:49:55 -04:00
f441fc4bbf Merge pull request #99 from kortschak/concrete
Use concrete value for quad.Quad use
2014-08-06 01:38:00 -04:00
a43531b4ce Merge pull request #98 from iaincollins/master
Fixed typo in quick setup command
2014-08-05 16:05:08 -04:00
Iain Collins
6d676094ee Fixed typo in quick setup command 2014-08-05 17:08:09 +01:00
kortschak
2c6f0f7345 Clean up residual changes
Remove done TODOs and unnecessary helpers.
2014-08-05 23:27:13 +09:30
kortschak
6acfdcc5d6 Use concrete value for quad.Quad
Comparison of -short benchmarks in cayley.

$ benchcmp pointer.bench concrete.bench
benchmark                                   old ns/op     new ns/op	delta
BenchmarkNamePredicate                      1673276       1655093	-1.09%
BenchmarkLargeSetsNoIntersection            318985907     261499984	-18.02%
BenchmarkNetAndSpeed                        104403743     41516981	-60.23%
BenchmarkKeanuAndNet                        17309258      16857513	-2.61%
BenchmarkKeanuAndSpeed                      20159161      19282833	-4.35%

Comparison of pathological cases are not so happy.

benchmark                                   old ns/op       new ns/op		delta
BenchmarkVeryLargeSetsSmallIntersection     55269775527     246084606672	+345.24%
BenchmarkHelplessContainsChecker            23436501319     24308906949		+3.72%

Profiling the worst case:

Pointer:
Total: 6121 samples
    1973  32.2%  32.2%     1973  32.2% runtime.findfunc
     773  12.6%  44.9%      773  12.6% readvarint
     510   8.3%  53.2%      511   8.3% step
     409   6.7%  59.9%      410   6.7% runtime.gentraceback
     390   6.4%  66.2%      391   6.4% pcvalue
     215   3.5%  69.8%      215   3.5% runtime.funcdata
     181   3.0%  72.7%      181   3.0% checkframecopy
     118   1.9%  74.6%      119   1.9% runtime.funcspdelta
      96   1.6%  76.2%       96   1.6% runtime.topofstack
      76   1.2%  77.5%       76   1.2% scanblock

Concrete:
Total: 25027 samples
    9437  37.7%  37.7%     9437  37.7% runtime.findfunc
    3853  15.4%  53.1%     3853  15.4% readvarint
    2366   9.5%  62.6%     2366   9.5% step
    2186   8.7%  71.3%     2186   8.7% runtime.gentraceback
    1816   7.3%  78.5%     1816   7.3% pcvalue
    1016   4.1%  82.6%     1016   4.1% runtime.funcdata
     859   3.4%  86.0%      859   3.4% checkframecopy
     506   2.0%  88.1%      506   2.0% runtime.funcspdelta
     410   1.6%  89.7%      410   1.6% runtime.topofstack
     303   1.2%  90.9%      303   1.2% runtime.newstack
2014-08-05 23:25:02 +09:30
kortschak
1ae81e6d00 Fix typos missed 2014-08-05 22:37:08 +09:30