Duplicate Software Versions

[troloff]

Code of Conduct

• I agree to follow this project's Code of Conduct

Is there an existing issue for this?

• I have searched the existing issues

Version

10.0.15 / GLPI Agent 1.9

Bug description

After update to 10.0.15, we are seeing multiple (duplicate) entries of the same software version. Example screenshot:

Relevant log output

No response

Page URL

No response

Steps To reproduce

1. deploy software using glpi agent (optional)
2. run glpi inventory agent on different machines
3. same software version shows up multiple times under assets->software->versions

Your GLPI setup information

Informationen über das System, die Installation und die Konfiguration

GLPI 10.0.15 ( => /XXX
Installation mode: TARBALL
Current language:de_DE

Server

 

Operating system: Linux XXX 6.1.0-13-amd64 #​1 SMP PREEMPT_DYNAMIC Debian 6.1.55-1 (2023-09-29) x86_64

PHP 8.2.18 fpm-fcgi (Core, FFI, PDO, Phar, Reflection, SPL, SimpleXML, Zend OPcache, bz2, calendar, cgi-fcgi, ctype, curl, date,

dom, exif, fileinfo, filter, ftp, gd, gettext, hash, iconv, imap, intl, json, ldap, libxml, mbstring, mysqli, mysqlnd, openssl,

pcre, pdo_mysql, posix, random, readline, session, shmop, sockets, sodium, standard, sysvmsg, sysvsem, sysvshm, tokenizer, xml,

xmlreader, xmlwriter, xsl, zip, zlib)

Setup: max_execution_time="300" memory_limit="1024M" post_max_size="512M" safe_mode="" session.save_handler="files"

upload_max_filesize="512M" disable_functions=""

Software: Apache/2.4.59 ()

Mozilla/5.0 (X11; Linux x86_64; rv:126.0) Gecko/20100101 Firefox/126.0

Server Software: Debian 12

Server Version: 10.11.6-MariaDB-0+deb12u1

Server SQL Mode: STRICT_TRANS_TABLES,ERROR_FOR_DIVISION_BY_ZERO,NO_AUTO_CREATE_USER,NO_ENGINE_SUBSTITUTION

Parameters: glpi@localhost/glpi

Host info: Localhost via UNIX socket

PHP version (8.2.18) is supported.

Sessions configuration is OK.

Allocated memory is sufficient.

mysqli extension is installed.

Following extensions are installed: dom, fileinfo, filter, libxml, json, simplexml, xmlreader, xmlwriter.

curl extension is installed.

gd extension is installed.

intl extension is installed.

zlib extension is installed.

The constant SODIUM_CRYPTO_AEAD_XCHACHA20POLY1305_IETF_NPUBBYTES is present.

Database engine version (10.11.6) is supported.

No files from previous GLPI version detected.

The log file has been created successfully.

Write access to /var/lib/glpi/_cache has been validated.

Write access to /var/lib/glpi/_cron has been validated.

Write access to /var/lib/glpi has been validated.

Write access to /var/lib/glpi/_dumps has been validated.

Write access to /var/lib/glpi/_graphs has been validated.

Write access to /var/lib/glpi/_lock has been validated.

Write access to /var/lib/glpi/_pictures has been validated.

Write access to /var/lib/glpi/_plugins has been validated.

Write access to /var/lib/glpi/_rss has been validated.

Write access to /var/lib/glpi/_sessions has been validated.

Write access to /var/lib/glpi/_tmp has been validated.

Write access to /var/lib/glpi/_uploads has been validated.
Web server root directory configuration seems safe.

Sessions configuration is secured.

OS and PHP are relying on 64 bits integers.

exif extension is installed.

ldap extension is installed.

openssl extension is installed.

Following extensions are installed: bz2, Phar, zip.

Zend OPcache extension is installed.

Following extensions are installed: ctype, iconv, mbstring, sodium.

Write access to /var/www/inventory.net.adk.de/marketplace has been validated.

Timezones seems loaded in database.

GLPI constants

 

GLPI_ROOT: ""

GLPI_CONFIG_DIR: "/etc/glpi"

GLPI_VAR_DIR: "/var/lib/glpi"

GLPI_LOG_DIR: "/var/log/glpi"

GLPI_MARKETPLACE_DIR: "XXX"

GLPI_USE_CSRF_CHECK: "1"

GLPI_CSRF_EXPIRES: "7200"

GLPI_CSRF_MAX_TOKENS: "100"

GLPI_USE_IDOR_CHECK: "1"

GLPI_IDOR_EXPIRES: "7200"

GLPI_ALLOW_IFRAME_IN_RICH_TEXT: false

GLPI_SERVERSIDE_URL_ALLOWLIST: ["/^(https?|feed):\/\/[^@:]+(\/.*)?$/"]

GLPI_TELEMETRY_URI: "https://telemetry.glpi-project.org"

GLPI_INSTALL_MODE: "TARBALL"

GLPI_NETWORK_MAIL: "glpi@teclib.com"

GLPI_NETWORK_SERVICES: "https://services.glpi-network.com"

GLPI_MARKETPLACE_ALLOW_OVERRIDE: true

GLPI_MARKETPLACE_MANUAL_DOWNLOADS: true

GLPI_USER_AGENT_EXTRA_COMMENTS: ""

GLPI_DISABLE_ONLY_FULL_GROUP_BY_SQL_MODE: "1"

GLPI_AJAX_DASHBOARD: "1"

GLPI_CALDAV_IMPORT_STATE: 0

GLPI_DEMO_MODE: "0"

GLPI_CENTRAL_WARNINGS: "1"

GLPI_TEXT_MAXSIZE: "4000"

GLPI_DOC_DIR: "/var/lib/glpi"

GLPI_CACHE_DIR: "/var/lib/glpi/_cache"

GLPI_CRON_DIR: "/var/lib/glpi/_cron"

GLPI_DUMP_DIR: "/var/lib/glpi/_dumps"

GLPI_GRAPH_DIR: "/var/lib/glpi/_graphs"

GLPI_LOCAL_I18N_DIR: "/var/lib/glpi/_locales"

GLPI_LOCK_DIR: "/var/lib/glpi/_lock"

GLPI_PICTURE_DIR: "/var/lib/glpi/_pictures"

GLPI_PLUGIN_DOC_DIR: "/var/lib/glpi/_plugins"

GLPI_RSS_DIR: "/var/lib/glpi/_rss"

GLPI_SESSION_DIR: "/var/lib/glpi/_sessions"

GLPI_TMP_DIR: "/var/lib/glpi/_tmp"

GLPI_UPLOAD_DIR: "/var/lib/glpi/_uploads"

GLPI_INVENTORY_DIR: "/var/lib/glpi/_inventories"

GLPI_NETWORK_REGISTRATION_API_URL: "https://services.glpi-network.com/api/registration/"

GLPI_MARKETPLACE_PLUGINS_API_URI: "https://services.glpi-network.com/api/marketplace/"

GLPI_I18N_DIR: "XXX"

GLPI_VERSION: "10.0.15"

GLPI_SCHEMA_VERSION: "10.0.15@2eed74704cb07e0bac48b933cbd5c1c356f09629"

GLPI_MARKETPLACE_PRERELEASES: false

GLPI_MIN_PHP: "7.4.0"

GLPI_MAX_PHP: "8.4.0"

GLPI_YEAR: "2024"

Libraries

 

htmlawed/htmlawed version 1.2.14 in (/var/www/inventory.net.adk.de/vendor/htmlawed/htmlawed)

phpmailer/phpmailer version 6.8.0 in (/var/www/inventory.net.adk.de/vendor/phpmailer/phpmailer/src)

simplepie/simplepie version 1.5.8 in (/var/www/inventory.net.adk.de/vendor/simplepie/simplepie/library)

tecnickcom/tcpdf version 6.7.5 in (/var/www/inventory.net.adk.de/vendor/tecnickcom/tcpdf)

michelf/php-markdown in (/var/www/inventory.net.adk.de/vendor/michelf/php-markdown/Michelf)

true/punycode in (/var/www/inventory.net.adk.de/vendor/true/punycode/src)

iamcal/lib_autolink in (/var/www/inventory.net.adk.de/vendor/iamcal/lib_autolink)

sabre/dav in (/var/www/inventory.net.adk.de/vendor/sabre/dav/lib/DAV)

sabre/http in (/var/www/inventory.net.adk.de/vendor/sabre/http/lib)

sabre/uri in (/var/www/inventory.net.adk.de/vendor/sabre/uri/lib)

sabre/vobject in (/var/www/inventory.net.adk.de/vendor/sabre/vobject/lib)

laminas/laminas-i18n in (/var/www/inventory.net.adk.de/vendor/laminas/laminas-i18n/src)

laminas/laminas-servicemanager in (/var/www/inventory.net.adk.de/vendor/laminas/laminas-servicemanager/src)

monolog/monolog in (/var/www/inventory.net.adk.de/vendor/monolog/monolog/src/Monolog)

sebastian/diff in (/var/www/inventory.net.adk.de/vendor/sebastian/diff/src)

donatj/phpuseragentparser in (/var/www/inventory.net.adk.de/vendor/donatj/phpuseragentparser/src/UserAgent)

elvanto/litemoji in (/var/www/inventory.net.adk.de/vendor/elvanto/litemoji/src)

symfony/console in (/var/www/inventory.net.adk.de/vendor/symfony/console)

scssphp/scssphp in (/var/www/inventory.net.adk.de/vendor/scssphp/scssphp/src)

laminas/laminas-mail in (/var/www/inventory.net.adk.de/vendor/laminas/laminas-mail/src/Protocol)

laminas/laminas-mime in (/var/www/inventory.net.adk.de/vendor/laminas/laminas-mime/src)

rlanvin/php-rrule in (/var/www/inventory.net.adk.de/vendor/rlanvin/php-rrule/src)

ramsey/uuid in (/var/www/inventory.net.adk.de/vendor/ramsey/uuid/src)

psr/log in (/var/www/inventory.net.adk.de/vendor/psr/log/Psr/Log)

psr/simple-cache in (/var/www/inventory.net.adk.de/vendor/psr/simple-cache/src)

psr/cache in (/var/www/inventory.net.adk.de/vendor/psr/cache/src)

league/csv in (/var/www/inventory.net.adk.de/vendor/league/csv/src)

mexitek/phpcolors in (/var/www/inventory.net.adk.de/vendor/mexitek/phpcolors/src/Mexitek/PHPColors)

guzzlehttp/guzzle in (/var/www/inventory.net.adk.de/vendor/guzzlehttp/guzzle/src)

guzzlehttp/psr7 in (/var/www/inventory.net.adk.de/vendor/guzzlehttp/psr7/src)

glpi-project/inventory_format in (/var/www/inventory.net.adk.de/vendor/glpi-project/inventory_format/lib/php)

wapmorgan/unified-archive in (/var/www/inventory.net.adk.de/vendor/wapmorgan/unified-archive/src)

paragonie/sodium_compat in (/var/www/inventory.net.adk.de/vendor/paragonie/sodium_compat/src)

symfony/cache in (/var/www/inventory.net.adk.de/vendor/symfony/cache)

html2text/html2text in (/var/www/inventory.net.adk.de/vendor/html2text/html2text/src)

symfony/css-selector in (/var/www/inventory.net.adk.de/vendor/symfony/css-selector)

symfony/dom-crawler in (/var/www/inventory.net.adk.de/vendor/symfony/dom-crawler)

twig/twig in (/var/www/inventory.net.adk.de/vendor/twig/twig/src)

twig/string-extra in (/var/www/inventory.net.adk.de/vendor/twig/string-extra)

symfony/polyfill-ctype not found

symfony/polyfill-iconv not found

symfony/polyfill-mbstring not found

symfony/polyfill-php80 not found

symfony/polyfill-php81 not found

symfony/polyfill-php82 in (/var/www/inventory.net.adk.de/vendor/symfony/polyfill-php82)

league/oauth2-client in (/var/www/inventory.net.adk.de/vendor/league/oauth2-client/src/Provider)

league/oauth2-google in (/var/www/inventory.net.adk.de/vendor/league/oauth2-google/src/Provider)

thenetworg/oauth2-azure in (/var/www/inventory.net.adk.de/vendor/thenetworg/oauth2-azure/src/Provider)

phpCas version 1.6.0 in (/usr/share/php/CAS/source)

SQL replicas

 
Not active

Notifications

 

Way of sending emails: PHP

Plugins list

 

datainjection        Name: Data injection                 Version: 2.13.5     State: Enabled

Install Method: Manual

formcreator          Name: Form Creator                   Version: 2.13.9     State: Enabled

Install Method: Manual

glpiinventory        Name: GLPI Inventory                 Version: 1.3.5      State: Enabled

Install Method: Manual

Anything else?

No response

[troloff]

I did an inventory run on two machines which show up as different versions. This is the output of the example software above:

Agent 1:

  <SOFTWARES>
     <ARCH>x86_64</ARCH>
     <FROM>registry</FROM>
     <GUID>Camunda Modeler</GUID>
     <INSTALLDATE>15/06/2024</INSTALLDATE>
     <NAME>Camunda Modeler</NAME>
     <PUBLISHER>Camunda</PUBLISHER>
     <SYSTEM_CATEGORY>application</SYSTEM_CATEGORY>
     <UNINSTALL_STRING>C:\Program Files\Camunda Modeler\uninstall.bat</UNINSTALL_STRING>
     <URL_INFO_ABOUT>https://camunda.com/de/download/modeler/</URL_INFO_ABOUT>
     <VERSION>5.24.0</VERSION>
   </SOFTWARES>

Agent 2:

    <SOFTWARES>
      <ARCH>x86_64</ARCH>
      <FROM>registry</FROM>
      <GUID>Camunda Modeler</GUID>
      <INSTALLDATE>14/06/2024</INSTALLDATE>
      <NAME>Camunda Modeler</NAME>
      <PUBLISHER>Camunda</PUBLISHER>
      <SYSTEM_CATEGORY>application</SYSTEM_CATEGORY>
      <UNINSTALL_STRING>C:\Program Files\Camunda Modeler\uninstall.bat</UNINSTALL_STRING>
      <URL_INFO_ABOUT>https://camunda.com/de/download/modeler/</URL_INFO_ABOUT>
      <VERSION>5.24.0</VERSION>
    </SOFTWARES>

As far as I can see, the only difference is the install date.

[troloff]

Maybe related:
https://forum.glpi-project.org/viewtopic.php?id=290215

[troloff]

Could this #16598 be the cause? Maybe, another "version" (in terms of glpi) gets added, if installation dates differ? (no developer here, so just a guess...)

[troloff]

CSV export of the glpi_softwareversions table (filtered according to the example above):
glpi_softwareversions_202406180753.csv

[trasher]

#16598 targets next major version, it's not included in 10.x series.

We need inventory file(s) in order to reproduce from a fresh install.

[troloff]

Do you mean the complete xml produced on clients after installation of software?
What do you mean by "fresh install" - glpiinventory agent?

[troloff]

Just upgraded to glpi 10.0.16, and we are still seeing duplicate software versions.
I even purged glpi_softwareversion and glpi_item_softwareversion tables to make sure there is no conflict because of already existing duplicates.
Please find attached the software inventories of two hosts in question.
roloffvm2020-2024-07-01-12-57-04.xml.txt
ernstvm2022-2024-07-10-21-16-50.xml.txt

[stonebuzz]

Hi @troloff

After importing (several times) your files, I do not reproduce

Camunda Modeler software does not have a duplicated version

When importing XML files, do you see errors in GLPI logs (php-error.log / sql-error.log)?

One clue could be an inconsistent state in your database data.

The presence of several identical versions (name, arch, os, linked software) should not be possible.

GLPI cannot find a single version and therefore adds it by default (1 +1 +1 +1 ....)

You can try to clean up the software with the following SQL queries

DELETE FROM glpi_softwares;
DELETE FROM glpi_softwareversion;
DELETE FROM glpi_items_softwareversions;

Software information will be updated as and when the inventory is updated.

[troloff]

Deleted the three tables as suggested. Now, software inventory rebuilds, but we have duplicate versions for every software/version installed on every computer.

There are no errors in sql-error.log

Only error in php-errors.log (but only <10 times a day) is

[2024-06-28 16:27:45] glpiphplog.WARNING:   *** PHP Warning (2): Undefined array key "HTTP_REFERER" in /var/www/inventory.net.adk.de/plugins/glpiinventory/inc/deploypackage.class.php at lin>
  Backtrace :
  src/MassiveAction.php:842                          PluginGlpiinventoryDeployPackage->getForbiddenStandardMassiveAction()
  src/Search.php:1835                                MassiveAction::getAllMassiveActions()
  src/Search.php:213                                 Search::displayData()
  src/Search.php:177                                 Search::showList()
  ...s/glpiinventory/inc/deploypackage.class.php:454 Search::show()
  plugins/glpiinventory/front/deploypackage.php:48   PluginGlpiinventoryDeployPackage->showList()
  public/index.php:82                                require()

We did not see this behaviour before update to GLPI 10.0.15 / GLPI-Agent 1.9 (which we unfortunately did in the same run...).

[troloff]

Maybe, I can add some debugging output to /src/Inventory/Asset/Software.php (line 410 ff?) to see what is happening, if new software "arrives" from the GLPI agents?

[troloff]

Just to be sure:

root@glpi:/var/www/glpi # php bin/console database:check_schema_integrity
Das Datenbankschema ist in Ordnung.

(which reads "database schema is ok")

[stonebuzz]

Do you have several entities?

Are the PCs in different entities?

How is software management configured in the entity?

[troloff]

Software management looks the same to me:

There is only one "root" entity,

and all computers belong to this entity (did a search for "entity is not root" and received an empty result).

[troloff]

Are inventories processed in parallel? If so, might there be a race condition in

glpi/src/Inventory/Asset/Software.php

Line 247 in d721c1f

public function handle()

?

Like:

• inventory a queries versions from db, software version v not found
• inventory b queries versions from db, software version v not found
• inventory a adds version v
• inventory b adds version v
• from this point, there are duplicate results if inventory queries for version v, which @stonebuzz mentioned as possible cause for problems

[squirellpl]

Same problem with two different installations.
GLPI 10.0.16, Agent 1.10, GLPI Inventory plugin 1.3.5. Some versions are without OS, some are I guess from different OS version. But this is the same software and version i.e. 7-zip 23.01 x86_64

[trasher]

Operating System is part of the dedup key; as it already was in the plugin. As already said, we need a way to reproduce on a fresh GLPI instance, so please provide related inventories files and detailed steps to do so.

[brunosrodrigues]

Same problem here. Any ideas?

[Nour-Ivory]

Same issue here, Server version 10.0.16 and clients on version 1.10 (both Windows and MacOS) . I have even deleted the software tables from the glpi database and allowed the agents to repopulate the tables and I am getting single entries for every installed application despite the fact that most of them are the same version
See screenshots below, in some cases, it is the same version of the software and yet it is listed 20 times

Any ideas of what the issue might be?

[trasher]

Not reproducible on our side. Feel free to reopen with a reproducble use case on a fresh GLPI latest stable release installation.