Wappalyzer boefje detects less software instances than before

[stephanie0x00]

Describe the bug
With the replacement to the Wappalyzer boefje (#2727) some of the findings from the original Wappalyzer boefje disappeared. This is due to changes in the 'technologies.json' files. A result from this is less Software oois identified and thus less findings relating to outdated software components.

In the old version various jQuery objects were identified. With the new version these objects are not identified.

Expected behavior
At minimum, the wappalyzer boefje should be able to identify all common software instances identified on websites, including the software versions.

OpenKAT version
commit 233dc34 (HEAD -> main, origin/main, origin/HEAD)

Current solution idea
As discussed with @underdarknl - the current wappalyzer boefje is installed using the requirements.txt. The idea is to fork the wappalyzer branch and add more/newer technologies.json files from other projects. This should hopefully solve the reduced number of findings observed by wappalyzer.

The python implementation of our Wappalyzer uses a different format in the technologies file than the 'original' Wappalyzer version. There is a fork of the original Wappalyzer that seems to be active by HTTArchive. This would require adjusting the parsing of the Wappalyzer boefje to match these formats. See: https://github.com/HTTPArchive/wappalyzer

[madelondohmen]

Discussion meeting 18-06-2024:

• technologies.json from community
• Continue to use python-wappalyzer
• Most efficient for current situation
• Don't fork, only use pieces we deem necessary.
• Keep regexing against our files

[stephanie0x00]

Discussion meet notes:

• We provide technology files on a regular interval we update and package these.
• We will setup some testing scenario's to test if software is still detected on some test cases.