Releases: emmercm/igir
v2.11.0
Release Highlights
Feature: cache ROM header & file signature detection
This will be an important performance boost ahead of various compressed disc support coming in v3. Thank you @dch82 for the suggestion!
Fixes: various performance bugs
- Checksums for files in the output directory are no longer calculated when not testing.
- Files being written to the same location (when the input and output directories are the same) are now skipped again, skipping expensive checksum calculations when testing.
Fixes: reducing concurrent file actions
- The concurrent filesize reading limit has been lowered to reduce disk thrash.
- Hashing of archive files when raw-copying now respects concurrent filesize read limits.
All changes
26 July 2024 - 3ec4a99 - v2.11.0 (#1240)
26 July 2024 - 54d4b73 - Feature: cache file signature detection (#1238)
26 July 2024 - 5d27f2b - Feature: cache ROM header detection (#1235)
25 July 2024 - e7c43e5 - Refactor: keep only a limited number of cache key mutexes (#1237)
25 July 2024 - c114188 - Fix: save the file cache on exit (#1236)
22 July 2024 - 29a3482 - Fix: File/ArchiveFile equality (#1232)
22 July 2024 - 216994d - Fix: don't throw if a single dat/input/patch directory is empty (#1231)
22 July 2024 - afb6449 - Fix: assume archives are full games when inferring DATs (#1230)
21 July 2024 - 1443fcd - Fix: don't CRC32 checksum files by default (#1229)
21 July 2024 - 9e40147 - Fix: reduce concurrent read & write file size limit (#1227)
21 July 2024 - 9d252bb - Fix: limit the concurrent file size being read when hashing raw archives (#1228)
20 July 2024 - 5d738ef - Refactor: consolidate file signature code (#1225)
20 July 2024 - 3e10abf - Fix: archive extension correction (#1224)
igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.
Contributors
Assets 8
v2.10.1
Release Highlights
Feature: cleaning backup directory
A new --clean-backup <path> option has been added to move cleaned files to a backup directory rather than recycle or delete them. Thank you @maxexcloo for this suggestion!
All changes
19 July 2024 - d7d6222 - v2.10.1 (#1223)
19 July 2024 - f53ee63 - Docs: update (#1222)
19 July 2024 - fd0d355 - Feature: cleaning backup directory option (#1221)
17 July 2024 - 3c3f976 - Fix: ZJONSSON/node-unzipper#324 (#1220)
igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.
Contributors
Assets 8
v2.10.0
Release Highlights
Feature: correction of ROM extensions
igir is most powerful when used in combination with DATs, but DATs aren't required to perform basic operations, and DATs don't always have complete information. For example, libretro's "metadats" don't include filename information for ROMs. Part of the filename can be assumed with the game name, but the extension can't be.
igir can now detect several common ROM formats by reading the file contents, and it will use the preferred ROM extension when information is missing (incomplete DATs as described above), or when the existing extension can't be trusted (when not using DATs). A new documentation page on ROM writing options has been added that details this behavior.
Fix: prevent output path collisions
@Djabal reported that some DATs include different ROMs with duplicate filenames. This would cause the same output file path to be written and rewritten with different ROMs, resulting in a non-deterministic outcome, and introducing the potential for crashes. igir will now prevent duplicate filenames from being written, stopping processing as soon as they're detected.
Options that sort games more granularly such as --dir-game-subdir always can be used to avoid filename collisions.
All changes
17 July 2024 - b4ead34 - v2.10.0 (#1219)
17 July 2024 - 5f30917 - Docs: update (#1218)
17 July 2024 - 351c91b - Feature: check for conflicting output file paths (#1208)
17 July 2024 - d02511c - Fix: enable CRC32 checksums when patch files are provided (#1217)
17 July 2024 - de38745 - Feature: correct bad ROM extensions (#1174)
15 July 2024 - 662ee57 - Chore: update Node.js to v20.15.1 (#1216)
14 July 2024 - 777b4d9 - Docs: update (#1214)
14 July 2024 - 8c7993d - Refactor: separate 'clean' help message options (#1215)
12 July 2024 - 0f0c9aa - CI: consistency with job permissions (#1212)
12 July 2024 - 865955a - Refactor: dir2dat use Game and ROM information from candidates (#1213)
12 July 2024 - 895ce43 - CI: update PR branch testing instructions on merge (#1211)
igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.
Contributors
Assets 8
v2.9.2
Release Highlights
Fix: fixdats with output tokens
@Djabal found a bug where using fixdats and output tokens together may crash. This should be fixed.
Fix: BIOS DAT assumption
igir was assuming that a DAT with a single game but many ROMs should be treated as a BIOS DAT and should split all of those ROMs into individual games. This was to work around a poorly written libretro System.dat. @steveonjava found this to be a bad assumption, some consoles such as the NEC PC-88 have many ROMs per game, and may not have many games. This has been fixed.
All changes
11 July 2024 - 92d4f03 - v2.9.2 (#1210)
11 July 2024 - 0a212cf - Fix: remove duplicate games when combining DATs (#1209)
11 July 2024 - 7c300db - Fix: libretro System.dat BIOS assumption (#1207)
11 July 2024 - c555d9a - Fix: write fixdats to the output dir root (#1205)
10 July 2024 - 17b6860 - Feature: handle No-Intro PS3 BIOS parent/clone inference (#1206)
8 July 2024 - ec8d897 - Refactor: don't log the stack trace of expected errors (#1203)
igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.
Contributors
Assets 8
v2.9.1
Release Highlights
Fix: downloading DAT URLs to a non-existent temp directory
@IDmedia found a fatal error introduced in v2.9.0 where igir would crash when downloading DAT URLs to a temporary directory that didn't already exist. This has been fixed.
All changes
4 July 2024 - 20a9ea7 - v2.9.1 (#1198)
4 July 2024 - b0e1011 - Fix: make multiple passes to delete empty directories (#1197)
3 July 2024 - 81c98e8 - Refactor: turn on the 'allowUnreachableCode' TypeScript option (#1194)
3 July 2024 - e50eee0 - Fix: mkdir before downloading DATs (#1196)
igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.
Contributors
Assets 8
v2.9.0
Release Highlights
Feature/fix: allow exact-matching of archives
Some DATs such as the libretro BIOS System.dat catalog specific zip files, rather than whatever files are contained in that zip file. igir will now detect if any DAT files catalog specific archives and then calculate checksums as appropriate.
This additional checksum calculation can greatly increase the time to scan input ROM files, so an --input-checksum-archives 'never' option has been added to turn off this automatic behavior.
Thank you @IDmedia for the bug report!
Feature: temporary directory option
A new --temp-dir <path> option has been added to specify the directory used for temporary files. A new documentation page has been added to explain the option and the necessity of temporary files.
All changes
30 June 2024 - f6335a6 - v2.9.0 (#1193)
30 June 2024 - 2b3111f - Feature: allow exact-matching of archives in DATs (#1175)
30 June 2024 - 9245a31 - Feature: temp dir option (#1041)
28 June 2024 - eca2260 - Refactor: rename constants directory (#1192)
28 June 2024 - dda552b - Chore: update lockfile (#1191)
28 June 2024 - 860693d - Refactor: split constants into two classes (#1189)
27 June 2024 - f34f9d4 - Chore: update ESLint (#1190)
27 June 2024 - a608c77 - Chore: update dependency eslint-plugin-unicorn to v54 (#1186)
27 June 2024 - d58044f - Chore: update ESLint (#1183)
27 June 2024 - 715122c - Chore: update docs dependencies (#1182)
27 June 2024 - c218b5c - Refactor: fix issues with dependency updates (#1188)
27 June 2024 - c3f34d4 - Chore: update dev dependencies (#1185)
27 June 2024 - 9ec8b85 - Chore: update dependency typescript to v5.5.2 (#1187)
27 June 2024 - 16e260c - Chore: update dependency ts-jest to v29.1.5 (#1181)
27 June 2024 - 77d1d3b - Chore: update adRise/update-pr-branch action to v0.8.1 (#1184)
27 June 2024 - c6b238a - Chore: update Node.js to v20.15.0 (#1180)
23 June 2024 - 6ca0222 - Fix: logging supplied & parsed arguments (#1179)
igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.
Contributors
Assets 8
v2.8.2
Release Highlights
Band-aid: don't crash with zip files >4.29GiB
An issue with the zip extraction library was causing fatal crashes with large zip files (e.g. Wii U games that are in the 8GiB+ range). The fatal crash is now prevented, but the library still isn't reading all compressed files correctly. An issue has been opened with the library and will continue to be monitored.
Thank you @tnvu and @keywal for helping diagnose this issue.
All changes
22 June 2024 - 7fdd2e7 - v2.8.2 (#1178)
22 June 2024 - 7c2c8cf - Fix: logging supplied & parsed arguments (#1177)
22 June 2024 - e0d3e21 - Fix: deferred ArchiveFile checksum waiting message (#1176)
22 June 2024 - 5ebee0c - Refactor: remove unused Options from CandidatePatchGenerator (#1173)
22 June 2024 - 12ce729 - Fix: ZJONSSON/node-unzipper#324 (#1171)
22 June 2024 - b28f248 - Fix: logging supplied & parsed arguments (#1172)
igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.
Contributors
Assets 8
v2.8.1
Release Highlights
Feature: deferred checksum calculation
Checksums of archives that are being raw-copied (aren't being extracted, aren't being zipped) are now deferred until after 1G1R rules have been applied. This can save some significant processing time with large collections with large files.
Token updates
{mister}: added Acorn Archimedes, added TI-99/4A, fixed Atari 800, fixed Microsoft MSX1{pocket}: added GCE Vectrex{gameRegion}: renamed to{region}{gameLanguage}: renamed to{language}{genre}: newly added for DATs that provide this info (uncommon)- Updated console file extensions
- Reduced the log level on replacement errors
As always, see the output path tokens docs for complete information.
All changes
17 June 2024 - fc36582 - v2.8.1 (#1169)
16 June 2024 - 090069e - Feature: console token updates (#1168)
16 June 2024 - aa13e5e - Feature: add {genre} token (#1167)
16 June 2024 - f2b546c - Refactor: remove the "game" suffix from {gameRegion} and {gameLanguage} (#1166)
16 June 2024 - 3b715d6 - Fix: handle missing ROM filenames in clrmamepro DATs (#1165)
13 June 2024 - 2bdf92d - Refactor: remove "concurrently" language from module comments (#1164)
13 June 2024 - 81f0402 - Refactor: update gzip file signature (#1163)
13 June 2024 - e0c40bb - Refactor: defer ArchiveFile checksum calculation until after 1G1R applied (#1162)
13 June 2024 - 01d2138 - Refactor: increase file cache reaper frequency (#1161)
13 June 2024 - 1eae2b0 - Feature: console token updates (#1160)
13 June 2024 - d4db60a - Refactor: log token replacement errors at TRACE instead of WARN (#1159)
igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.
v2.8.0
Release Highlights
Feature: support for MAME "software list" DATs
Did you know that MAME has emulation support for systems other than arcade? Well, it does! You can get the DATs for these systems using the mame -getsoftlist <pattern> command.
Feature: detect archives with bad extensions
Files don't always get distributed with the correct extension, so igir can now detect if a file is an archive and what type of archive it is even if the extension is wrong.
Feature: macOS Homebrew package
You can now install igir on macOS using brew! The full command is:
brew tap emmercm/igir
brew install igirFix: case-insensitive checksums
@knoopx found an issue where DATs with ROM checksums in uppercase weren't working correctly. This has been fixed.
Fix: files that failed to move shouldn't be deleted
If a failure happened during a file move, where the output file wasn't written, the input file was still being deleted. This has been fixed.
All changes
12 June 2024 - 72b1757 - v2.8.0 (#1158)
11 June 2024 - a780983 - Refactor: detect deleted cached files subset on a timer (#1157)
11 June 2024 - 52bfae1 - Fix: don't consider input files as moved when write failed (#1156)
11 June 2024 - ed1f267 - Refactor: optimize ArrayPoly.filterUniqueMapped() further (#1155)
11 June 2024 - c08e57a - Fix: don't put zipped multi-ROM games in a subdirectory (#1154)
11 June 2024 - 860a9d2 - Refactor: limit the number of cached files checked for existence (#1153)
11 June 2024 - 073a975 - Refactor: progress bar fixes & tweaks (#1152)
11 June 2024 - e2d7d78 - Refactor: optimize ArrayPoly.filterUniqueMapped() (#1151)
9 June 2024 - 273ecca - Chore: Security upgrade unzipper from 0.11.6 to 0.12.1 (#1150)
4 June 2024 - 106be63 - Chore: update Node.js to v20.14.0 (#1149)
1 June 2024 - 1aa6339 - Fix: reduce README help message width for GitHub (#1148)
31 May 2024 - 06accc2 - Fix: sanitize non-glob paths when globbing (#1147)
31 May 2024 - aa0d2d9 - Fix: don't test default cache paths when one provided (#1145)
31 May 2024 - 4e9033a - Fix: lowercase DAT ROM checksums (#1146)
29 May 2024 - ffa8559 - Chore: update dependency eslint-plugin-unicorn to v53 (#1142)
29 May 2024 - 2f926ca - Chore: update lockfile (#1143)
27 May 2024 - 53b4e82 - Chore: update dependencies (#1141)
27 May 2024 - e64c7e1 - Chore: update ESLint (#1140)
27 May 2024 - 7dc8e97 - Chore: update docs dependencies (#1139)
27 May 2024 - 90e7453 - Chore: update dependency ts-jest to v29.1.3 (#1138)
27 May 2024 - 42ba589 - Chore: update dependency @types/node to v20.12.12 (#1137)
16 May 2024 - 59a4371 - Chore: update Node.js to v20.13.1 (#1134)
14 May 2024 - ad8560b - Chore: update Node.js to v20.13.0 (#1133)
10 May 2024 - 5d2584e - Feature: warn on Node.js v22 EOL (#1132)
10 May 2024 - 9832172 - Feature: correct invalid extensions when raw-copying archives (#1128)
10 May 2024 - 7adcf96 - CI: increase test coverage (#1131)
10 May 2024 - 2418458 - Fix: don't use the cache file when testing (#1126)
10 May 2024 - 967d2ed - Refactor: use file paths in cache keys (#1125)
10 May 2024 - d0a11ac - CI: increase test coverage (#1130)
10 May 2024 - 362aaae - Docs: Homebrew installation instructions (#1129)
10 May 2024 - 6b9a0cc - Refactor: reduce file reading high water mark to match Node.js v22 (#1127)
10 May 2024 - 6171249 - Feature: better handling of TOSEC naming conventions (#1123)
10 May 2024 - fe650c8 - Fix: attempt to sanitize non-glob paths when globbing (#1124)
9 May 2024 - dbd343d - Feature: detect archives with bad extensions with file signatures (#1121)
9 May 2024 - c32113f - Refactor: Bun: use FileHandle.writeFile() (#1114)
9 May 2024 - 7f59ca6 - Feature: support MAME software list DATs (#1031)
igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.
Contributors
Assets 8
v2.7.0
Release Highlights
Reporting changes
You can now igir report without providing any --input <path>. @IDmedia suggested valid use cases for why someone would want to generate an empty report for DATs, primarily because reports are affected by igir's filter and preference rules.
A new "DUPLICATE" status has been added. This is to resolve some confusion around the "UNUSED" status which gets used for input files that weren't used during copying, moving, or linking - regardless of whether the input file matched a ROM in a DAT. Now, if an input file could have been used for a copying, moving, or linking action but wasn't (because some other input file was), it will report as "DUPLICATE". Thank you @9p6 for the suggestion!
Feature: debug option for changing the cache path
A new --cache-path <path> option has been added to specify where igir should write the cache file. Thank you @TheBrainScrambler for the suggestion!
Fix: validate regions and languages
Validation has been added for regions (--filter-region <regions>, --prefer-region <regions>) and languages (--filter-language <languages>, --prefer-language <languages>). You can no longer provide any invalid regions or languages, which is helpful because some abbreviations aren't obvious to all.
All changes
8 May 2024 - aef5adb - v2.7.0 (#1122)
8 May 2024 - 4e03c51 - Feature: DUPLICATE report status to disambiguate UNUSED (#1120)
8 May 2024 - 825d608 - Fix: report non-preferred games as IGNORED when no game found (#1119)
8 May 2024 - 57d166d - Feature: allow reporting with no input when not writing ROMs (#1118)
8 May 2024 - 4ee0d8c - Feature: infer abbreviated years in game names (#1116)
8 May 2024 - 56ad757 - Feature: validate language and region option values (#1117)
8 May 2024 - 822d001 - Feature: cache path option (#1115)
6 May 2024 - a132387 - Chore: update codecov/codecov-action action to v4 (#1106)
30 April 2024 - f820b17 - Chore: update lockfile (#1110)
30 April 2024 - a5fa53b - Chore: update docs dependencies (#1105)
28 April 2024 - e0baf70 - Chore: update lockfile (#1109)
27 April 2024 - 7eca8f9 - Chore: update dependency eslint-plugin-unicorn to v52 (#1107)
27 April 2024 - bb19cb6 - Chore: update dependency typescript to v5.4.5 (#1108)
27 April 2024 - 01daead - Chore: update dependency @types/node to v20.12.7 (#1104)
27 April 2024 - 89e1db8 - Chore: update dependencies (#1103)
26 April 2024 - 0e8f54f - Chore: update ESLint (#1101)
26 April 2024 - d62e780 - Refactor: ESLint typing fixes (#1102)
26 April 2024 - 3999705 - Chore: update lockfile (#1100)
26 April 2024 - 5d84b65 - Chore: update dependency eslint-plugin-jest to v28 (#1099)
igir, and other open-source projects like it, are a labor of love. They wouldn't be nearly as meaningful without the community supporting them. Constructive feedback and general positivity will always be welcome. Be kind to others, and be kind to yourselves.
