Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Performance issue with deep taxonomy benchmark #1

Open
josd opened this issue Nov 14, 2021 · 3 comments
Open

Performance issue with deep taxonomy benchmark #1

josd opened this issue Nov 14, 2021 · 3 comments

Comments

@josd
Copy link

josd commented Nov 14, 2021

I was testing https://raw.githubusercontent.com/josd/josd.github.io/master/temp/dt.n3
and the good news is that the 30000 triples are produced but at a rather slow pace

$ uname -srmo
Linux 4.19.104-microsoft-standard x86_64 GNU/Linux
$ npm run test

> n3@1.0.0 test
> jest

 PASS  test/parse.test.ts
 PASS  test/reasoner.test.ts (1350.582 s)

Test Suites: 2 passed, 2 total
Tests:       18 passed, 18 total
Snapshots:   0 total
Time:        1350.89 s
Ran all test suites.

FYI, see also mthom/scryer-prolog#1053
esp.

This benchmark was designed by the late Harold Boley http://www.cs.unb.ca/~boley/
and is interesting to evaluate the subsumption capabilities of reasoners (using RDF and OWL)
e.g.
[[
SNOMED CT currently contains more than 300,000 medical concepts, divided into
hierarchies as diverse as body structure, clinical findings, geographic location and
pharmaceutical/biological product
]] -- https://searchhealthit.techtarget.com/definition/SNOMED-CT

@phochste
Copy link
Collaborator

This JavaScript code is indeed shockingly slow compared to EYE. I tried to run this test also this morning and stopped it after 10 minutes .

@josd
Copy link
Author

josd commented Nov 14, 2021

Indeed when I ran it initially it went in timeout

    thrown: "Exceeded timeout of 5000 ms for a test.
    Use jest.setTimeout(newTimeout) to increase the timeout value, if this is a long-running test."

although it ran for more than those 5 sec

Test Suites: 1 failed, 1 passed, 2 total
Tests:       1 failed, 17 passed, 18 total
Snapshots:   0 total
Time:        1245.282 s
Ran all test suites.

It was able to finish after adding

jest.setTimeout(1000000000);

A while ago after a meeting with @RubenVerborgh I worked on eye.js which is doing only backward chaining, always producing a proof and using json instead of N3. Running the deep taxonomy benchmark takes

$ time node dt.js > dt-proof.json

real    0m15.476s
user    0m16.253s
sys     0m1.150s

For this test the reasoner also has to chain (backwards) through the whole taxonomy.

@phochste
Copy link
Collaborator

With another way of parsing N3 files I could bring the deep taxonomy benchmark down to about 110 seconds.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants