Skip to content

Commit

Permalink
Clean up TODO
Browse files Browse the repository at this point in the history
Move some entries around, remove one or two DONE
  • Loading branch information
0-wiz-0 committed Jun 27, 2024
1 parent 80bb641 commit d1136f2
Showing 1 changed file with 65 additions and 86 deletions.
151 changes: 65 additions & 86 deletions TODO.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,6 @@
# nihtest

- Remove mame.db from tests that don't need it.
- Only use mame.db files from inside the test directory.

# Implement

- Fix failing tests.
- Variables in config file (e.g. for collection root directory).
- Empty directory in ArchiveDir is not cleaned up, which makes removing empty archive fail.

# Write Tests for Config
Expand All @@ -16,101 +10,25 @@

# Other

- Move `delete_unknown_pattern` to `DatOptions`.

- status update: SIGINFO support in Archive::commit, via libzip progress callback

- search loose files in zipped mode: Add option in ArchiveDir to ignore zip files, but keep in .ckmame.db.

- speedup idea: when opening archives (in extra dirs) only compute hashes if we need them

- `mkmamedb`: When a game is in two dat files (identical name and ROMs), skip it from second (with warning).

- Make `SIGINFO` handler more responsive.

- handle multiple writers to ckmamedb

- exceptions and error messages:
- who creates which part of the error messages
- catch exceptions in main and print errors (done in ckmame and mkmamedb).
- clean up Exceptions without text

- When checking single game, search for its ROMs everywhere and report any matches.

- rar read support

- add option to keep ROMs with detector applied

- bug: when creating a fixdat and re-checks happen, games end up in the fixdat multiple times

- fixdat-missing-all: two copies of the same file (romof) in fixdat; depends on order of games in datfile

- C++ cleanups:
- move lineno into ParserSource
- add tests for all tokens in all parser backends:
- cm
- rc
- xml

- when committing to garbage fails because source archive is broken, move source archive out of the way.

# Later

- use CkmameDB as old db

- `mkmamedb`: analyze speed, make it faster

- clean up archives in `.ckmame.db` for manually removed dirs/zips
- Variables in config file (e.g. for collection root directory).

# Unsorted

- `mkmamedb` doesn't handle chds (ignores for zipped, includes in games as rom for unzipped)
! [feature] reorder cleanup step when renaming files to remove the copies
earlier; otherwise big renames on big sets don't work.
! [bug] cleanup reports file that were used this time as "not used"
! [bug] cleanup after using disks reports errors because disks were moved
Example:
```
ckmame: new/file.chd: cannot remove: No such file or directory
In image new/file.chd:
image new/file.chd: not used
```
+ [feature] `mkmamedb`: parser for mtree files
+ [feature] server mode: tell location/mamedb of ROM sets, serves files needed by remote `ckmame` client
+ [bug] check/fix database error reporting (pass on sqlite3 errors)
+ [feature] get needed files directly from parent (`inparent.test`)
+ [feature] inconsistent zip in ROM set: copy files to new zip
+ [feature] `mkmamedb`: mark ROMs without checksums as `nogooddump`
+ [bug] `mkmamedb -F dat`:
- write `&` unquoted (perhaps other meta characters as well)
- write Umlaut wrong
```
Entity: line 5986: parser error : Input is not proper UTF-8, indicate encoding !
Bytes: 0x81 0x63 0x6B 0x73
<rom name="Gl�cksrad.ipf" size="1049180" crc="008b8870" sha1="b1a4bba7c96ce0c2
^
```
Suggested solution: using `iconv` and `LC_CTYPE`, try transcribing invalid UTF-8 files to UTF-8
if it fails or no `iconv` or `LC_CTYPE` available, replace invalid characters with `'?'`
- [cleanup] access db directly in `find_*`
- [cleanup] specify globally which parts of lists to fill/maintain
- [cleanup] rom: use flag to specify whether we know the size
- [bug] check if needed/extra are different
- [bug] DB export: pass all dat entries to output backend
- [bug] DB export: export detector
- [bug] diagnostics (fix?): don't process disks if checking samples
- [feature] `mkmamedb`: split to original CM dat files + detector XML on export
- [feature] database consistency checks during `mkmamedb`
- are all roms of one set included in one other set
- are two sets the same, just different name
- [feature] `--cleanup-extra`: remove (unnecessary?) directories
- [feature] support multiple `-O` arguments
- [compatibility] Accept CRCs of length < 8 by zero-padding them (in front)
- [cleanup] refactor fixing code (one function per operation)
- [cleanup] handle archive refreshing in `archive.c`
- [cleanup] rename: file is part of zip archive, rom is part of game

## other features
## Other Features
- fixdat: if only checking child, ROM missing in parent is not in fixdat
- parse: add state checking to `parse-cm.c`
- parse: check for duplicate attributes
Expand All @@ -120,11 +38,24 @@
- `mkmamedb`: warn about sets without parent that use "merge" (`mamedb-merge-no-parent.dump`)
- complete raine support (multiple archive names: `archive ( name "64th_street" name "64street" ))`

## code cleanups
## Code Cleanups
- make `parse_cm` table driven
- fix all TODOs
- Move `delete_unknown_pattern` to `DatOptions`.
- exceptions and error messages:
- who creates which part of the error messages
- catch exceptions in main and print errors (done in ckmame and mkmamedb).
- clean up Exceptions without text
- C++ cleanups:
- move lineno into ParserSource
- add tests for all tokens in all parser backends:
- cm
- rc
- xml

## tests
## Tests
- Remove mame.db from tests that don't need it.
- Only use mame.db files from inside the test directory.
- Add test for `mkmamedb -F cm`.
- extend at least following tests to use md5/sha1 as well:
- `delete-used-superfluous.test`
Expand Down Expand Up @@ -169,3 +100,51 @@
- fix Xcode warnings
- [test] fix preload on OS X
- [feature] if `.ckmame.db` can't be opened, move aside and create new

# Unsorted

- `mkmamedb` doesn't handle chds (ignores for zipped, includes in games as rom for unzipped)
! [feature] reorder cleanup step when renaming files to remove the copies
earlier; otherwise big renames on big sets don't work.
! [bug] cleanup reports file that were used this time as "not used"
! [bug] cleanup after using disks reports errors because disks were moved
Example:
```
ckmame: new/file.chd: cannot remove: No such file or directory
In image new/file.chd:
image new/file.chd: not used
```
+ [feature] `mkmamedb`: parser for mtree files
+ [feature] server mode: tell location/mamedb of ROM sets, serves files needed by remote `ckmame` client
+ [bug] check/fix database error reporting (pass on sqlite3 errors)
+ [feature] get needed files directly from parent (`inparent.test`)
+ [feature] inconsistent zip in ROM set: copy files to new zip
+ [feature] `mkmamedb`: mark ROMs without checksums as `nogooddump`
+ [bug] `mkmamedb -F dat`:
- write `&` unquoted (perhaps other meta characters as well)
- write Umlaut wrong
```
Entity: line 5986: parser error : Input is not proper UTF-8, indicate encoding !
Bytes: 0x81 0x63 0x6B 0x73
<rom name="Gl�cksrad.ipf" size="1049180" crc="008b8870" sha1="b1a4bba7c96ce0c2
^
```
Suggested solution: using `iconv` and `LC_CTYPE`, try transcribing invalid UTF-8 files to UTF-8
if it fails or no `iconv` or `LC_CTYPE` available, replace invalid characters with `'?'`
- [cleanup] access db directly in `find_*`
- [cleanup] specify globally which parts of lists to fill/maintain
- [cleanup] rom: use flag to specify whether we know the size
- [bug] check if needed/extra are different
- [bug] DB export: pass all dat entries to output backend
- [bug] DB export: export detector
- [bug] diagnostics (fix?): don't process disks if checking samples
- [feature] `mkmamedb`: split to original CM dat files + detector XML on export
- [feature] database consistency checks during `mkmamedb`
- are all roms of one set included in one other set
- are two sets the same, just different name
- [feature] `--cleanup-extra`: remove (unnecessary?) directories
- [feature] support multiple `-O` arguments
- [compatibility] Accept CRCs of length < 8 by zero-padding them (in front)
- [cleanup] refactor fixing code (one function per operation)
- [cleanup] handle archive refreshing in `archive.c`
- [cleanup] rename: file is part of zip archive, rom is part of game

0 comments on commit d1136f2

Please sign in to comment.