Read config from http(s), new input formats

Important dependencies updates:

• #12 unsafe-libyaml from 0.2.9 to 0.2.10

Breaking Changes

• NEW Added Illumos (OmniOS builds)
• BREAKING OpenBSD builds regression because of msrv 1.74 not available on stable

New features

• NEW Remote config file for instance --mode config --config https://my.server/hostlists/hctl.yml
• NEW Input formats, check out wiki
• NEW Spinner indication for non stdout output

Maintenance

• HUGE code clean, created submodules

Goals for next release 0.3.0

• Function unification
• More unit-tests.
• Better error handling.

Goals for 1.0.0

• In-code documentation
• Code coverage report.
• Automatic testing in gh actions

After that release cycle will change

For 1.X.0

• New non-breaking features

Goals for 1.0.X

• Automated monthly releases with dependencies bump

In future

• Automatic input format recogintion (lot of work, needs help, personally don't need it)
• Void Linux package addition still pending
• homebrew, personal tap
• crates.io

Message to users

• I got no feedback so I assumed no for asked questions.
• If someone wants hctl in official homebrew I am unable to help, they don't want authors to be maintainers as opposite to most linux repos.

Breaking changes, new features, dependencies security fixes

Important dependencies updates:

• #11 OpenSSL
• #10 Rustix

Breaking Changes and new features

• NEW: Now can read as input format additionally: dnsmasq, bind, hostperm1
• BREAKING Because of ^ new obligatory arg was added (or in config) --pattern to specify input format of list
• BREAKING: config file scheme unmarshall has changed, make sure you updated it, see example_config.yml on master branch

Other changes

• BUILDS: Mac OS X: silicon is on sonoma, intel is on stable (will automatically bump from ventura when gh sets is as -latest)
• BUILDS: FreeBSD: is on 14.0-RELEASE-p1
• BUILDS: OpenBSD 7.4 Added! Built with rust from repo, without cargo auditable
• NEW: mode=url now you can fetch directly one list from web without using yaml file

In Future

• Remote config file for instance --mode config --config https://my.server/hostlists/hctl.yml will be useful if you want to modify config without logging into server
• More input formats
• Automatic input format recogintion (lot of work, needs help)
• Function unification
• In-code documentation
• Proper error handling.
• More unit-tests.
• Code coverage report.
• Automatic testing in gh actions
• Void Linux package addition still pending
• homebrew, only personal tap
• crates.io

Needs feedback

• My guess: NO: Should I care about DragonflyBSD, Solaris, Illumos, Haiku, Zircon etc. ?
• My guess: NO: Are you using hctl as critical mission software? Should I care about backward compatibility when it comes to old config schemes?

New feature: DNS validation

EDIT: Security Alert

read here

TD;LR

Contains rustix 0.38.14
If you try to access unlinked symlink may cause "memory explosion"
So you should be safe if used without symlinking.
Advised to update as soon as new version will be out. Probably 0.2.4.1 or 0.2.5
Or do rm Cargo.lock && cargo update then recompile by yourself.

Hi folks!

As promised full DNS-enabled release contains:

• User defined resolver, see example_config.yml, controllable only in mode=config, in other modes it will use hardcoded uncensored DOT resolvers.
• New -t --validate arg, that will just output true or false for each domain (in parallel). Works with grep.
• Since now, intro contains UTC timestamp
• MAINTENANCE: Added parsing + query unit-test for custom resolvers.
• MAINTENANCE: I've set binary embedded information using cargo-auditable
• MAINTENANCE: Changed default branch to master due to condflict with LICENSE amend rebase on other branch.
• fixed typo vesrion -> version in serde dependency, no more warnings!

Note: that:

• IPv6-only custom resolver may fail? I don't have one, please report.
• LICENSE is now just BSD-3-Clause
• IMPORTANT if you use config mode, you need to add (even empty) key resolvers: below, otherwise yaml unmarshall will fail.
• IMPORTANT mode=single + optimize = memory silently ignores --dns
• IMPORTANT Right now please use mode=config for --dns with 2/3 resolvers, as algorithm doesn't respect trust_nx otherwise checking will take ages with inbuild resolvers as there are plenty of them and algorithm will issue query to all of them if previous fails

News:

• hctl has been accepted 🎉 to the alpine repo testing (edge) and in future will be moved to the community (probably at the end of the year), read aports package request for more details.
• I've set up public home repo on SUSE OBS so you can just type opi hctl to get it on *SUSE
• hctl compiles on every architecture that SUSE TW is available, even riscv64

Future:

• Void linux package request pending
• Resolver from preset (to get yaml readable).
• Reduce blind .unwrap() in other words - proper error handling.
• More unit-tests.
• Code coverage report.
• Fedora, AUR, brew, FreeBSD ports

BETA: new feature: dns-over-tls record validation

Hi folks!

This is pre-release because not all features for dns-over-tls has been implemented. Stay tuned!

What actually changed:

• this and future releases by default requires openssl, but you can get rid of it
• small fix for parallel iterators - removed one collect. Should be faster then.
• validate your records for ip or cname by dns-over-tls with dnseec, controlled by cli arg --dns [yes/no] default no

Note that:

• For some reason dot queries are slow... Like ~600 records per 1.5 minute. Needs investigation
• There are inbuilt non-logging, uncensored, privacy friendly dot servers +(cloudflare and quad9), 16 in total.
• /etc/resolv.conf or any other system dns conf is not respected
• Regression with DragonflyBSD builds, as trust-dns-resolver bumped msrv

Full release will contain:

• User-defined dot servers
• Query speedup (hopefully)
• New option for only log-generating while dns validation, without rejecting.

New Output formats, Fix NetBSD build

Hi folks!

Just a small release

Done:

• Added 7 new output formats, documented on wiki
• Fixed CI for DragonFlyBSD and NetBSD Release
• You can watch detailed hctl download stats here on wiki

Remaining:

• If anyone could help with OpenBSD VM issue 12 this will unblock OpenBSD releases
• See also Solaris VM issue 34 if interested in Illumos Builds
• If anyone could help with FreeBSD NetBSD OpenBSD rust crosscompilation, please open new issue

Deleted first release, as accidentally created tag 2.1 instead of 0.2.1

Implemented whitelists with subdomain matching

Merge pull request #4 from michalszmidt/dev

Dev

Testing new CI config

0.1.5

Fix CI 8

Moving forward to 1.0.0 but not yet

Merge pull request #3 from michalszmidt/dev

Dev

Mostly fixes, still not there.

0.1.3

Workflow fix 2 and bump version in cargo.toml

Preview

Just a preview, not tested, regex malfunctions in some rare cases.