Commit graph

148 commits

Author SHA1 Message Date
kortschak
401c58426f Create quads hierarchy
* Move nquads into quad.
* Create cquads simplified parser in quad.
* Move Triple (renamed Quad) to quad.

Also made sure mongo actually implements BulkLoader.
2014-07-28 21:36:22 +09:30
kortschak
01bc63810b Specify the limitations of quad/triple parsing 2014-07-26 17:45:56 +09:30
kortschak
64d79ee4d5 Actually perform the N-Quads test 2014-07-26 15:55:53 +09:30
kortschak
b276d117b1 Fix documentation error 2014-07-26 12:03:34 +09:30
kortschak
f1a7710c82 Generate parser code
ragel -Z -G2 parse.rl

ragel version 6.8 (as previously)
2014-07-26 12:00:29 +09:30
kortschak
5953a8dfa0 Add RDF test suite testing and fix grammar errors 2014-07-26 11:58:56 +09:30
kortschak
9ff97602f0 Fix missed case in quadfix... and re-run
Checked with MQL query successfully:

[{
  "<type>": "</film/film>",
  "<name>": null,
  "</film/film/directed_by>": {
    "<name>": "\"David Fincher\""
  },
  "</film/film/starring>": [{
    "</film/performance/actor>": {
      "<name>": null
    }
  }]
}]
2014-07-25 23:10:04 +09:30
kortschak
bf185b7702 Add doc comments for nquads 2014-07-25 15:05:57 +09:30
kortschak
fae860f08e Replace nt with nq in various places throughout
Manually change testdata.nt to testdata.nq.
2014-07-25 14:56:08 +09:30
kortschak
3995d3dd57 Run quadfix on 30kmovies.nt.gz and replace 2014-07-25 11:38:23 +09:30
kortschak
c75cef60d1 Generate parser code
ragel -Z -G2 parse.rl

ragel version 6.8 (as previously)
2014-07-25 11:37:04 +09:30
kortschak
92baa2ecee Cleanup unused vars, move unEscape and add tests 2014-07-25 11:35:48 +09:30
kortschak
d76213fb2d Handle comments in N-Quad documents and REPL
The parser rejects an N-Quad with a comment, so we filter those out
ahead of time. This simplifies the grammar and code generated by the
parser.
2014-07-25 11:22:24 +09:30
kortschak
dc17ccae80 Fix unicode handling and add tests
We now also return an incomplete triple to aid in debugging - non-nil
error means that this is not usable except for manual examination.
2014-07-25 10:48:06 +09:30
kortschak
59e3d620a5 Use more sane quadfix rules 2014-07-25 09:19:26 +09:30
kortschak
22bad1701f Generate code for parser
G2 code generation used after benchmarking.

style	benchmark           old ns/op     new ns/op     delta
T0		BenchmarkParser     672           5631          +737.95%
T1		BenchmarkParser     672           5579          +730.21%
G0		BenchmarkParser     672           4049          +502.53%
G1		BenchmarkParser     672           3868          +475.60%
G2		BenchmarkParser     672           3543          +427.23%

F0 and F1 create massive Go source (6.0M) and so were not tested.

Invalid tests removed, additional tests for invalid input to be added
later.
2014-07-24 21:31:04 +09:30
kortschak
92d50bb9f7 Add parser generation definition 2014-07-24 20:57:19 +09:30
kortschak
f1c606dc19 Provide standalone converter for 30kmoviesdata.nt 2014-07-24 20:08:05 +09:30
31211696a8 Merge pull request #77 from kortschak/appengine
Use build constraints for appengine
2014-07-22 20:39:13 -04:00
kortschak
592ea81687 Use build constraints for appengine 2014-07-23 10:02:11 +09:30
be26f0faf6 Merge pull request #75 from kortschak/parse
Use error returns and interface type for parsing
2014-07-22 20:19:52 -04:00
kortschak
e6ed23ef7c Merge branch 'master' into parse
Conflicts:
	db/load.go
2014-07-23 06:39:56 +09:30
kortschak
018b8ca032 Add package prefix to errors 2014-07-23 06:30:35 +09:30
7c54408951 Merge pull request #76 from barakmich/master
Switch to the latest mgo path.
2014-07-22 15:31:59 -04:00
95a31c179b Switch to the latest mgo path.
This is not only the right thing to do, as per the documentation of the
latest release (yesterday) but it should now be backed by git and not
bzr, which is a big plus and won't break our build so much.
2014-07-22 15:22:29 -04:00
984ab6fd25 Merge pull request #71 from kortschak/compress
Add transparent input decompression
2014-07-22 14:41:14 -04:00
kortschak
0e0e382d2b Use error returns and interface type for parsing
Fixes issue #72

This change simplifies interactions with parsing N-Quads and makes
reading datasets more robust. Changes made while here also improve
performance:

benchmark           old ns/op     new ns/op     delta
BenchmarkParser     1058          667           -36.96%

We still use string concatenation which I'm not wildly happy about, but
I think this can be left for a later change.

Initial changes towards idiomatic error handling have been made. More
significant changes are needed, but these have subtle design implication
and need to be thought about more.

30kmoviesdata.nt.gz has been altered to properly escape double quotes.
This was done mechanically and with manual curation to pick up
straglers.
2014-07-22 20:34:37 +09:30
kortschak
7f99318f0d Add tests for decompressor 2014-07-22 10:58:59 +09:30
kortschak
9bf09a5db5 Add transparent input decompression
This supports gzip and bzip2 by magic number determination.

Trailing whitespace differences in documentation due to opinionated
editor.
2014-07-19 12:49:55 +09:30
abdd649c82 Merge pull request #69 from pbnjay/triplestore_registry
Triplestore registry
2014-07-18 15:01:04 -04:00
Jeremy Jay
923679b36c comment tweaks 2014-07-18 14:54:29 -04:00
Jeremy Jay
fb3e181a62 fix failing build 2014-07-18 14:29:23 -04:00
Jeremy Jay
5731ca7b42 6 of one, half-dozen of the other 2014-07-18 14:19:02 -04:00
Jeremy Jay
a6dc5c9532 fix tests for new method names 2014-07-18 14:09:57 -04:00
Jeremy Jay
0641309a8f adding to A+C 2014-07-18 11:46:36 -04:00
Jeremy Jay
f9c60a5f30 update names per discussion at google/cayley#38 2014-07-18 11:17:57 -04:00
Jeremy Jay
d808d9347c move to registry interface for backends 2014-07-16 16:49:55 -04:00
Jeremy Jay
e780c1ceb9 Register should be RegisterIterator 2014-07-16 13:49:39 -04:00
995b38fb5b Update the README with news 2014-07-12 15:24:39 -04:00
c1471ee09e Merge pull request #59 from alpe/minor_updates
Minor cleanup
2014-07-10 10:59:32 -04:00
Alex Peters
4d34ea50cc Fix typos and minor cleanup 2014-07-10 13:19:30 +02:00
56962c7e18 Merge pull request #55 from kortschak/travis
Re-add go1.2 to .travis.yaml
2014-07-09 13:37:09 -04:00
9ca38d1f10 Merge pull request #56 from kortschak/testing
Tabulate value comparison tests
2014-07-07 14:08:03 -04:00
kortschak
e39063e3ec Rename some methods and funcs since we are here
Very probably some of these can be made private.
2014-07-07 12:23:32 +09:30
kortschak
d1fdba1cbb Fix build
leveldb did s/GetApproximateSizes/SizeOf/
2014-07-07 12:14:47 +09:30
kortschak
5bc1c438ef Tabulate value comparison tests 2014-07-07 12:01:16 +09:30
kortschak
db3597ed2d Re-add go1.2 to .travis.yaml
Fixes issue #54.
2014-07-06 15:53:47 +09:30
c88a24467e Merge pull request #52 from kortschak/testing
Simplify testing code
2014-07-05 21:55:46 -04:00
kortschak
fbb66ddc52 Fix build
goleveldb is broken against go1.2 since de350e, so we cannot support
go1.2. This will change when the goleveldb is changed to again support
go1.2.
2014-07-06 09:23:47 +09:30
kortschak
d489e3da07 Drop testing dependencies 2014-07-05 22:12:56 +09:30