Replies: 3 comments 7 replies
-
|
Thanks! |
Beta Was this translation helpful? Give feedback.
-
|
Hi pham! Thanks for the quick turn-around. But now I am getting some strange segmentation faults on mac os big sur 11.6. Terminal info below: kenpchess@Kenneths-Mac-mini ocgdb-main-20220128 % ls kenpchess@Kenneths-Mac-mini src % cd /Volumes/SG5TB_2TB/aPGNs_EPDs_for_tuning_2021/ (rename just built binary and copy into testing directory by dragging in mac os big sur 11.6 gui) kenpchess@Kenneths-Mac-mini aPGNs_EPDs_for_tuning_2021 % ./ocgdb_20220128_kp -pgn enginegames_eco_A.pgn -db enginegames_eco_A.ocgdb_moves_moves2.db3 -cpu 2 -o moves;moves2 Convert PGN files into a database... kenpchess@Kenneths-Mac-mini aPGNs_EPDs_for_tuning_2021 % ./ocgdb_20220128_kp -pgn enginegames_eco_A.pgn -db enginegames_eco_A.ocgdb_moves_moves2.db3 -cpu 2 -o moves Convert PGN files into a database... (use your renamed binary from github for last try) kenpchess@Kenneths-Mac-mini aPGNs_EPDs_for_tuning_2021 % ./ocgdb_20220128_pham -pgn enginegames_eco_A.pgn -db enginegames_eco_A.ocgdb_moves_moves2.db3 -cpu 2 -o moves Convert PGN files into a database... Not at all sure what is happening here. I hope this info helps...
|
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the report. The problem I can't reproduce the issue. Could you try:
If the issue still happens and if you could, run the program with XCode in debug mode. There is a project file in the folder "projects" for XCode. |
Beta Was this translation helpful? Give feedback.
-
|
Hi Pham! OK, now we are making progress! I was kinda pulling my hair and saying WTF! But now it is working and brings to mind a famous line from a sci-fi type movie (Dr. StrangeGlove I think). In a very southern U.S. drawl the main character says "What we have here is a failure to communicate!". So I had to put quotes around any - o pairs such as: ./ocgdb_beta2_0131_kp -pgn mb-3.45.pgn -db mb345.ocgdb_testmoves.db3 -cpu 2 -o "moves;moves2" or ./ocgdb_beta2_0131_kp -db mb345.ocgdb_testmoves.db3 -cpu 2 -q "Q=3" -o "printfen;printpgn" I did not see that anywhere in the documentation and unless that is peculiar to my system you should definitely update the various readme files! So all of the various move and printing options look good now on my end! Thanks for the replies and hard work. I am looking forward to the next revisions!
|
Beta Was this translation helpful? Give feedback.
-
|
Quotes are useful if you want to add spaces into parameters such as |
Beta Was this translation helpful? Give feedback.
-
|
Hi Pham. Perhaps it doesn't matter now, but it might if these OCGDB commands are called from inside a GUI in the future. Maybe I wasn't clear, but option pairs like "moves;moves2" or "printfen;printpgn" simply would NOT work on my system unless the pairs actually were enclosed in the double quotes. On my system WITHOUT the double quotes, the second parameter was always ignored and was then attempted to be parsed by the macos unix shell (zsh or bash) and would of course simply throw an unrecognized command error. I don't know how you are parsing multiple options, but it might be worth double-checking your code. Anyway, you are making good progress and thanks again for your efforts!
|
Beta Was this translation helpful? Give feedback.
-
|
An interesting issue. I have been using an iMac/macOS and didn't see that problem. As the standard way, OCGDB doesn't process quotes or not quotes itself. Instead, the system processes and then gives it the list of arguments/parameters without quotes. Thus I can't do anything more about that since it depends much on the system/shell. You may run OCGDB with parameter -debug to see if it receives and parses correct options and other parameters. |
Beta Was this translation helpful? Give feedback.
-
|
Hi Pham. Yes, strange. Thanks for the debug info. I used -debug and could clearly see that at least on my system that macos terminal was ignoring any string information past a semicolon unless the string info (-o pairs, etc.) was enclosed in double quotes. As i said, not a problem now as I can use double quotes around option pairs but it could certainly cause problems if someone built a GUI for the database and created OCGDB terminal subprocesses to do database work. About the only suggestion I have at the moment is if someone does the GUI per my suggestion above would be to automatically add double quotes around option pairs if they build an OCGDB terminal string. I sure would like to know if someone else has seen this problem. I have not seen many other comments about your project lately ( a few on TalkChess of course). By the way, regarding the duplicates issue, two games that end in the same position are certainly NOT equivalent games. All kinds of tactics and positions with different paths and both potential wins and losses (in-accuracies, blunders, etc.) could occur before happening to reach the same final position. I would say at the very least that the move order has to be the same to be considered a duplicate. Thanks so much for your work on this project. As someone who is mac only and cannot easily run chessbase I am always interested in any and all mac chess database projects. I have looked at all of them and they all have significant weaknesses.
|
Beta Was this translation helpful? Give feedback.
-
Hi Pham! Very good progress! But I am getting strange results on queries, (missing move info, queries with results but no game info, etc. Is this bugs or am I doing something wrong? Hopefully you can look at this and tell what is happening? Particularly look at the last couple of runs. Here is some of my terminal info:
kenpchess@Kenneths-Mac-mini aPGNs_EPDs_for_tuning_2021 % ./ocgdb_20220124_kp -db enginegames_eco_A.ocgdb_keepmoves.db3 -cpu 2 -q "Q=3" -o printpgn
Open Chess Game Database Standard (OCGDB), Database Builder, (C) 2022 - version: Beta
All parameters:
Task: query
PGN paths:
Database path: enginegames_eco_A.ocgdb_keepmoves.db3
Queries:
Q=3
Options: printpgn;
gameNumberLimit: 281474976710655
resultNumberLimit: 281474976710655
cpu: 2, min Elo: 0, min game length: 0
Thread count: 2
Search with query Q=3..., searchField: Moves2
Q=3 DONE. Elapsed: 170829 ms, 02:50, total games: 4034503, total results: 3425, time per results: 49 ms
Completed!
kenpchess@Kenneths-Mac-mini aPGNs_EPDs_for_tuning_2021 % ./ocgdb_20220124_kp
Open Chess Game Database Standard (OCGDB), Database Builder, (C) 2022 - version: Beta
Usage:
ocgdb []
-pgn PGN game database file, repeat to add multi files
-db create database, extension should be .ocgdb.db3
use :memory: to create in-memory database
-cpu number of threads, omit for all cores, works with -pgn, -bench, -query
-bench benchmarch querying games speed, works with -db and -cpu
-q querying positions, repeat to add multi queries, works with -db and -cpu
-elo discard games with Elo under n (for creating)
-plycount discard games with ply-count under n (for creating)
-resultcount stop querying if the number of results above n (for querying)
-o [;] options
moves create text move field Moves
moves1 create binary move field Moves, 1-byte encoding
moves2 create binary move field Moves, 2-byte encoding
acceptnewtag create a new field for a new PGN tag (for creating)
discardcomments discard all comments (for creating)
discardsites discard all Site tag (for creating)
discardnoelo discard games without player Elos (for creating)
printall print all results (for querying)
printfen print FENs of results (for querying)
printpgn print simple PGNs of results (for querying)
Examples:
ocgdb -pgn big.png -db big.ocgdb.db3 -cpu 4 -o moves
ocgdb -pgn big.png -db :memory: -elo 2100 -o moves;moves1;discardsites
ocgdb -bench -db big.ocgdb.db3 -cpu 4
ocgdb -db big.ocgdb.db3 -cpu 4 -q "Q=3" -q"P[d4, e5, f4, g4] = 4 and kb7"
kenpchess@Kenneths-Mac-mini aPGNs_EPDs_for_tuning_2021 % ./ocgdb_20220124_kp -db enginegames_eco_A.ocgdb_keepmoves.db3 -cpu 2 -q "Q=3" -o printfen;printpgn
Open Chess Game Database Standard (OCGDB), Database Builder, (C) 2022 - version: Beta
All parameters:
Task: query
PGN paths:
Database path: enginegames_eco_A.ocgdb_keepmoves.db3
Queries:
Q=3
Options: printfen;
gameNumberLimit: 281474976710655
resultNumberLimit: 281474976710655
cpu: 2, min Elo: 0, min game length: 0
Thread count: 2
Search with query Q=3..., searchField: Moves2
Q=3 DONE. Elapsed: 190002 ms, 03:10, total games: 4034503, total results: 3425, time per results: 55 ms
Completed!
zsh: command not found: printpgn
kenpchess@Kenneths-Mac-mini aPGNs_EPDs_for_tuning_2021 % ./ocgdb_20220124_kp -db enginegames_eco_A.ocgdb_keepmoves.db3 -cpu 2 -q "Q=3" -o printfen
Open Chess Game Database Standard (OCGDB), Database Builder, (C) 2022 - version: Beta
All parameters:
Task: query
PGN paths:
Database path: enginegames_eco_A.ocgdb_keepmoves.db3
Queries:
Q=3
Options: printfen;
gameNumberLimit: 281474976710655
resultNumberLimit: 281474976710655
cpu: 2, min Elo: 0, min game length: 0
Thread count: 2
Search with query Q=3..., searchField: Moves2
Q=3 DONE. Elapsed: 190219 ms, 03:10, total games: 4034503, total results: 3425, time per results: 55 ms
Completed!
kenpchess@Kenneths-Mac-mini aPGNs_EPDs_for_tuning_2021 % cd /Users/kenpchess/Desktop/ocgdb-main-20220126
kenpchess@Kenneths-Mac-mini ocgdb-main-20220126 % ls
LICENSE README.md SqlCmd.md projects samples src
kenpchess@Kenneths-Mac-mini ocgdb-main-20220126 % cd src
kenpchess@Kenneths-Mac-mini src % chmod a+x makefile
kenpchess@Kenneths-Mac-mini src % make
g++ -std=c++17 -DNDEBUG -O3 -c .cpp
g++ -std=c++17 -DNDEBUG -O3 -c board/.cpp
g++ -std=c++17 -DNDEBUG -O3 -c 3rdparty/SQLiteCpp/.cpp
gcc -DNDEBUG -O3 -c 3rdparty/sqlite3/.c
g++ -o ocgdb *.o
kenpchess@Kenneths-Mac-mini src % cd ..
kenpchess@Kenneths-Mac-mini ocgdb-main-20220126 % ./ocgdb_20220126_kp
Open Chess Game Database Standard (OCGDB), Database Builder, (C) 2022 - version: Beta
Usage:
ocgdb []
-pgn PGN game database file, repeat to add multi files
-db create database, extension should be .ocgdb.db3
use :memory: to create in-memory database
-cpu number of threads, omit for all cores, works with -pgn, -bench, -query
-bench benchmarch querying games speed, works with -db and -cpu
-q querying positions, repeat to add multi queries, works with -db and -cpu
-g get game with game ID number, works with -db
-elo discard games with Elo under n (for creating)
-plycount discard games with ply-count under n (for creating)
-resultcount stop querying if the number of results above n (for querying)
-o [;] options
moves create text move field Moves
moves1 create binary move field Moves, 1-byte encoding
moves2 create binary move field Moves, 2-byte encoding
acceptnewtag create a new field for a new PGN tag (for creating)
discardcomments discard all comments (for creating)
discardsites discard all Site tag (for creating)
discardnoelo discard games without player Elos (for creating)
printall print all results (for querying)
printfen print FENs of results (for querying)
printpgn print simple PGNs of results (for querying)
Examples:
ocgdb -pgn big.png -db big.ocgdb.db3 -cpu 4 -o moves
ocgdb -pgn big.png -db :memory: -elo 2100 -o moves;moves1;discardsites
ocgdb -bench -db big.ocgdb.db3 -cpu 4
ocgdb -db big.ocgdb.db3 -cpu 4 -q "Q=3" -q"P[d4, e5, f4, g4] = 4 and kb7"
ocgdb -db big.ocgdb.db3 -g 4432
kenpchess@Kenneths-Mac-mini ocgdb-main-20220126 % cd /Volumes/SG5TB_2TB/aPGNs_EPDs_for_tuning_2021/
kenpchess@Kenneths-Mac-mini aPGNs_EPDs_for_tuning_2021 % ./ocgdb_20220124_kp -db enginegames_eco_A.ocgdb_keepmoves.db3 -cpu 2 -g 4000
Open Chess Game Database Standard (OCGDB), Database Builder, (C) 2022 - version: Beta
All parameters:
Task: create SQL database
PGN paths:
Database path: enginegames_eco_A.ocgdb_keepmoves.db3
Queries:
Options:
gameNumberLimit: 281474976710655
resultNumberLimit: 281474976710655
cpu: 2, min Elo: 0, min game length: 0
Error: Must have at least one PGN path. Mising or wrong parameter -pgn
Usage:
ocgdb []
-pgn PGN game database file, repeat to add multi files
-db create database, extension should be .ocgdb.db3
use :memory: to create in-memory database
-cpu number of threads, omit for all cores, works with -pgn, -bench, -query
-bench benchmarch querying games speed, works with -db and -cpu
-q querying positions, repeat to add multi queries, works with -db and -cpu
-elo discard games with Elo under n (for creating)
-plycount discard games with ply-count under n (for creating)
-resultcount stop querying if the number of results above n (for querying)
-o [;] options
moves create text move field Moves
moves1 create binary move field Moves, 1-byte encoding
moves2 create binary move field Moves, 2-byte encoding
acceptnewtag create a new field for a new PGN tag (for creating)
discardcomments discard all comments (for creating)
discardsites discard all Site tag (for creating)
discardnoelo discard games without player Elos (for creating)
printall print all results (for querying)
printfen print FENs of results (for querying)
printpgn print simple PGNs of results (for querying)
Examples:
ocgdb -pgn big.png -db big.ocgdb.db3 -cpu 4 -o moves
ocgdb -pgn big.png -db :memory: -elo 2100 -o moves;moves1;discardsites
ocgdb -bench -db big.ocgdb.db3 -cpu 4
ocgdb -db big.ocgdb.db3 -cpu 4 -q "Q=3" -q"P[d4, e5, f4, g4] = 4 and kb7"
kenpchess@Kenneths-Mac-mini aPGNs_EPDs_for_tuning_2021 % ./ocgdb_20220126_kp -db enginegames_eco_A.ocgdb_keepmoves.db3 -cpu 2 -g 4000
Open Chess Game Database Standard (OCGDB), Database Builder, (C) 2022 - version: Beta
All parameters:
Task: get game
PGN paths:
Database path: enginegames_eco_A.ocgdb_keepmoves.db3
Queries:
Options:
gameNumberLimit: 281474976710655
resultNumberLimit: 281474976710655
cpu: 2, min Elo: 0, min game length: 0
Game retrieved successfully, PGN:
[Event ""]
[Black "N.N."]
[White "N.N."]
[PlyCount "34"]
[ECO "A00"]
[Result "1/2-1/2"]
[BlackElo "3000"]
[WhiteElo "3000"]
[Round "20"]
[Date "2018-01-29"]
kenpchess@Kenneths-Mac-mini aPGNs_EPDs_for_tuning_2021 % ./ocgdb_20220126_kp -db enginegames_eco_A.ocgdb_keepmoves.db3 -cpu 2 -g 4001
Open Chess Game Database Standard (OCGDB), Database Builder, (C) 2022 - version: Beta
All parameters:
Task: get game
PGN paths:
Database path: enginegames_eco_A.ocgdb_keepmoves.db3
Queries:
Options:
gameNumberLimit: 281474976710655
resultNumberLimit: 281474976710655
cpu: 2, min Elo: 0, min game length: 0
Game retrieved successfully, PGN:
[Event ""]
[Black "Blañk Queen, Stockfish 160919 64"]
[ECO "A00"]
[Result "1/2-1/2"]
[White "Auryn, Brainfish 160919 64"]
[PlyCount "133"]
[BlackElo "2398"]
[WhiteElo "2455"]
[Round "0"]
[Date "2019-09-28"]
kenpchess@Kenneths-Mac-mini aPGNs_EPDs_for_tuning_2021 % ./ocgdb_20220126_kp -db enginegames_eco_A.ocgdb_keepmoves.db3 -cpu 2 -g 40001
Open Chess Game Database Standard (OCGDB), Database Builder, (C) 2022 - version: Beta
All parameters:
Task: get game
PGN paths:
Database path: enginegames_eco_A.ocgdb_keepmoves.db3
Queries:
Options:
gameNumberLimit: 281474976710655
resultNumberLimit: 281474976710655
cpu: 2, min Elo: 0, min game length: 0
Game retrieved successfully, PGN:
[Event ""]
[White "fjjvh"]
[Black "amaker"]
[PlyCount "32"]
[ECO "A00"]
[Round "0"]
[Result "0-1"]
[BlackElo "1309"]
[WhiteElo "819"]
[Date "2011-11-20"]
kenpchess@Kenneths-Mac-mini aPGNs_EPDs_for_tuning_2021 % ocgdb_20220126_kp -pgn famous_players.pgn -cpu 3 -q "Q=3"
zsh: command not found: ocgdb_20220126_kp
kenpchess@Kenneths-Mac-mini aPGNs_EPDs_for_tuning_2021 % ./ocgdb_20220126_kp -pgn famous_players.pgn -cpu 3 -q "Q=3"
Open Chess Game Database Standard (OCGDB), Database Builder, (C) 2022 - version: Beta
All parameters:
Task: query
PGN paths:
famous_players.pgn
Database path:
Queries:
Q=3
Options:
gameNumberLimit: 281474976710655
resultNumberLimit: 281474976710655
cpu: 3, min Elo: 0, min game length: 0
Thread count: 3
Search with query Q=3..., searchField: Moves
Processing PGN file: 'famous_players.pgn'
#games: 213182, elapsed: 696996158ms 193:36:36, speed: 0 games/s, #blocks: 17, processed size: 136 MB
#games: 381897, elapsed: 697006108ms 193:36:46, speed: 0 games/s, #blocks: 31, processed size: 241 MB
Q=3 DONE. Elapsed: 23105 ms, 00:23, total games: 381897, total results: 3, time per results: 7701 ms
Completed!
kenpchess@Kenneths-Mac-mini aPGNs_EPDs_for_tuning_2021 %
Beta Was this translation helpful? Give feedback.
All reactions