- All notable changes to this project will be documented in this file.
- optimization : keep intermediate columns in gui batch mode
- optimization : single batch connection and vacuum strategy
- optimization : single batch connection
- fix bug in data retieval webservice
- fix bug in data retieval webservice
- fix java null pointer exception when crc error on reader
- fix parquet file export memory leak
- download data to s3 gui button works again
- if s3 is active, files which have triggered batch error are sent to s3
- fix executor ephemral pvc
- ignore independance rules when child tag doesn't exist
- add priority rules to sort incoming files
- common.io security version update
- postgres scanner version upgrade
- bug fix to catch when parquet export fail in order to interrupt batch correctly
- spring security update
- datawarehouse and archives inside datawarehouse can be loaded according to an order defined by user. Check "entrepot" gui.
- add kubernetes ephemeral volume management for executor pods
- fix web service not waiting for large data to be created
- spring update
- parquet support for data retrieval webservice
- enhanced logs for data retrieval webservice
- clear parquet export directory if s3 output bucket is defined
- jwt authentification for data retrieval webservice if keycloak is set
- user_restricted right fix
- pvc is deleted in volatile mode after batch ends
- fix file action (delete, replay, archive replay)
- fix batch exit when production is set to off
- fix the total file size limiter used by reception phase in batch mode
- Add a new database parameter LanceurIHM.tailleMaxReceptionEnMb to limit the total file size to be received in gui mode.
- bug fix for deirectory creation in kubernetes pvc
- add parameters KUBERNETES_EXECUTOR_CPU, KUBERNETES_EXECUTOR_RAM
- file retention period can be in set fr.insee.arc.files.retention.days
- KO will now be deleted from database after data retention period
- rework external table column red for file to be valid in database
- bug fix on parquet encryption key
- reduce privilege level for kubernetes executor databases
- sql constraints on the table parameter
- debug gui access can now be set in a distinct keycloak group (fr.insee.gui.debug.disable)
- exception handling for access denied
- docker images now have version. They are deployed through the "released on tag" workflow and handle same version number.
- rest webservice securisation for injection
- gui database securisation for injection (add gui constraints for all rules)
- new gui generic error message
- docker add : parquet encryption key for minio s3 export
- docker add : keycloak
- checkmarx fix
- databases ip caching to avoid dsn spam in batch mode
- deprecated spring security fix (rest webservice and web)
- action are now logged with idep
- ARC batch exports data to parquet (when fr.insee.arc.process.export property is not empty). Default is off
- ARC batch pops, uses and removes executor database (when fr.insee.arc.kubernetes.executor.volatile is not empty). Default is off
- bug fix for gz archive entry name
- parquet export
- bug fix with filter gui
- gui bug fix : import rules and update comment in nomenclature screen
- tomcat fix as a library was missing in new version of tomcat 10.8
- removed webservice entrypoint /execute/engine/{serviceName}/{serviceId}/{bas} as it looks unused
- webservice data retrieval servlet access fix
- kubernetess support
- disablegui property now working
- tomcat 10, java 17, spring 6
- fix : ws import wait for large data tables
- last version in tomcat 9, java 11
- avoid overflow when large file data are set on a single line
- batch failed with functional warning
- upload mapping rules now the model declared variables
- scalable data retrieval webservice
- csv_gzip format is available for the data retrieval webservice
- dependencies security fix
- fix performance issue in to_delete reception query
- bind variable in gererFamille
- production version
- deployment for version qualification
- initialization optimization
- the webservice for data retrieval deletes the client pending data tables at the start of a new invoke
- the webservice for data retrieval acts creates data table asynchronously to avoid timeout problems
- functionnal tests
- pg 14.8 bug fix
- bug fix in gui file report
- scalability connection management
- passage en production
- version beta avec suppression de la phase de filtrage
- github workflow java 11
- show sand box informations at the home page
- loader performance enhancement
- gui quality refactor
- wrong xsd location in webservice configuration
- unit test for technical and core class (part 1)
- upload data file via GUI is fixed to match with new security constraints
- web health-check is now public
- bug fix in normage view actions
- bug fix FileUtils.cleanDirectory with symbolic lock
- bug fix in duplicate detection
- gui refactor
- parallelism refactor
- healthcheck quality refactor and add-on for web module
- DDI model implementation
- bug fix in maintenance screen
- The batch logger default level was incorrect and fixed
- code quality
- secured the data retrieval service (https + whitelist)
- normage optimization for cartesian reduction with new independance rules
- model metadata can now be exported
- esane test for ci/cd
- deliverable name fix for internal ci/cd tested
- urls for hello, healthcheck and version pages are now public access
- batch optimization : better queuing for files processing to avoid downtime
- bug fix to download database output from the gui
- button to apply directly rules in production environments
- optimization of cartesian reduction algorithm
- spring and springboot security fix
- multiselection on export gui
- button to apply directly rules in production environments
- optimization of cartesian reduction algorithm
- spring and springboot security fix
- multiselection on export gui
- button to apply directly rules in production environments
- optimization of cartesian reduction algorithm
- add exclusion rules on structurize phase
- model tablename can now contain digits
- postgres driver security bump
- spring security bump
- h2 security fix
- git informations retrieval bug fix for batch
- fix version display message for batch
- revert phase ihm bugfix for reception phase
- files timestamp fix
- log4j 2.17.1
- log4j 2.17.0
- database h2 security fix
- fix #2 for log4j security jndi injection problem
- git-commit-id-plugin depandency update to get the tag number correctly
- sonar security fix
- for security purpose, ihm has now distinct public page index and a secured index
- keycloak.resource=keycloak.json settings is now default. ARC now detects if keycloak is activated by scanning the "realm" and "auth-server-url" value. No changes required for current installations
- log4j security fix
- the data retrieval webservice can now be used to retrieve specific data types ("mapping", "nomenclature", "metadata")
- maintenance parameters GUI to set the way ARC works is now available
- archive reader now works for tar.gz archives that contain several directories
- bug fix in user management interface due to a null attribute
- database and application version now directly picked from git metadata
- tabs in gui are now persitant
- new maintenance menu to set parameters and test loggers
- bugfixes in gui
- fixes bug preventing writing log to file (web-service)
- adds new property fr.insee.arc.log.level to configure ARC log level
- allows logging to file through property
- extends expressions to control rules
- fixes multithread bug on SimpleDateFormat causing NPE in mapping
- adds /healthcheck to webservice
- improves error message on key-value loading failing
- adds clarifying report to files marked KO because one other file in the archive could not be read