OpenQuake Engine 3.2.0

[Matteo Nastasi (@nastasi-oq)]

• specified 'amd64' as the only architecture supported by ubuntu packages

[Michele Simionato (@micheles)]

• Changed the source writer: now the srcs_weights are written in the XML
  file only if they are nontrivial
• Changed the algorithm assigning the seeds: they are now generated before
  the source splitting; also, a seed-related bug in the splitting was fixed
• For event based, moved the rupture generation in the prefiltering phase

[Daniele Viganò (@daniviga)]

• Fixed a bug with CTRL-C when using the processpool distribution

[Robin Gee (@rcgee)]

• Raised the source ID length limit in the validation from 60 to 75 characters
  to allow sources with longer IDs

[Michele Simionato (@micheles)]

• Introduced a multi_node flag in openquake.cfg and used it to
  fully parallelize the prefiltering in a cluster
• Used the rupture seed as rupture ID in event based calculations
• Changed the deprecation mechanism of GSIMs to use a class attribute
  superseded_by=NewGsimClass
• Solved the pickling bug in event based hazard by using generator tasks
• Improved the distribution of the risk tasks by changing the weight

[Pablo Heresi (@pheresi)]

• Contributed the HM2018CorrelationModel

[Michele Simionato (@micheles)]

• Restored the individual_curves flag that for the moment is used for the
  risk curves
• Introduced two experimental new parameters floating_distance and
  spinning_distance to reduce hypocenter distributions and nodal plane
  distributions of ruptures over the corresponding distances
• Optimized the parsing of the logic tree when there is no "applyToSources"
• Made the IMT classes extensible in client code
• Reduced the hazard maps from 64 to 32 bit, to be consistent with the
  hazard curves and to reduce by half the download time

[Graeme Weatherill (@g-weatherill)]

• Implements a fix of Montalva et al (2016) for new coefficients (now
  Montalva et al. (2017))

[Michele Simionato (@micheles)]

• Parallelized the reading of the source models
• Optimized oq info --report by not splitting the sources in that case
• Speedup the download of the hazard curves, maps and uhs
• Honored concurrent_tasks in the prefiltering phase too
• It is now legal to compute uniform hazard spectra for a single period
• Added command oq plot_memory
• Introduced a MultiGMPE concept
• Saved the size of the datastore in the database and used it in the WebUI

[Graeme Weatherill (@g-weatherill)]

• Adds geotechnical related IMTs

[Michele Simionato (@micheles)]

• Renamed /extract/agglosses -> /extract/agg_losses and same for aggdamages
• Supported equivalent epicentral distance with a reqv_hdf5 file
• Fixed the risk from ShakeMap feature in the case of missing IMTs
• Changed the way gmf_data/indices and ruptures are stored
• Added experimental support for dask
• Added 11 new site parameters for geotechnic hazard
• Changed the SiteCollection to store only the parameters required by the
  GSIMs

[Robin Gee (@rcgee)]

• The number of sites is now an argument in the method _get_stddevs()
  in the GMPE of Kanno, 2006

[Michele Simionato (@micheles)]

• Changed the serialization of ruptures to HDF5: the geometries are now
  stored in a different dataset
• Bug fix: the asset->site association was performed even when not needed
• Made it possible to serialize to .hdf5 multipoint sources and
  nonparametric gridded sources
• Added a check on source model logic tree files: the uncertaintyModel
  values cannot be repeated in the same branchset
• Added a flag std_hazard_curves; by setting it to true the user can
  compute the standard deviation of the hazard curves across realizations

[Marco Pagani (@mmpagani)]

• Added Thingbaijam et al. (2017) magnitude-scaling relationship

[Michele Simionato (@micheles)]

• Added an /extract/ API for event_based_mfd
• Fixed a bug in the classical_damage calculators: multiple loss types
  were not treated correctly

[Marco Pagani (@mmpagani)]

• Adding tests to the method computing decimal time

[Michele Simionato (@micheles)]

• Removed the event_based_rupture calculator and three others
• Added a field size_mb to the output table in the database and made
  it visible in the WebUI as a tooltip
• Added a command oq check_input job.ini to check the input files
• Made the loss curves and maps outputs from an event based risk calculation
  visible to the engine and the WebUI (only the stats)
• Added a check on duplicated branchIDs in GMPE logic trees

[Daniele Viganò (@daniviga)]

• Fixed a bug when reading exposure with utf8 names on systems with non-utf8
  terminals (Windows)
• Changed the openquake.cfg file and added a dbserver.listen parameter
• Added the hostname in the WebUI page. It can be customize by the user
  via the local_settings.py file

[Michele Simionato (@micheles)]

• Added a Content-Length to the outputs downloadable from the WebUI
• Fixed a bug when extracting gmf_data from a hazard calculation with a
  filtered site collection
• Stored an attributed events.max_gmf_size
• Added a check on exposures with missing loss types
• Added a LargeExposureGrid error to protect the user by tricky exposures
  (i.e. France with assets in the Antilles)
• Changed the event_based_risk calculator to compute the loss curves and
  maps directly; removed the asset_loss_table
• Changed the event_based_risk calculator to distribute by GMFs always
• Optimized the memory consumption in the UCERF classical calculator
• Added a parameter minimum_magnitude in the job.ini
• Added an utility utils/combine_mean_curves.py

OpenQuake Engine 3.1.0

[Marco Pagani (@mmpagani) and Changlong Li (@mstlgzfdh)]

• Added a version of the Yu et al. (2013) GMPE supporting Mw

[Michele Simionato (@micheles)]

• Reduced the data transfer in the UCERF calculators
• Stored the zipped input files in the datastore for reproducibility
• Fixed a regression when reading GMFs from an XML in absence of a sites.csv
  file

[Robin Gee (@rcgee)]

• Extend oq to_shapefile method to also work with YoungsCoppersmithMFD
  and arbitraryMFD MFD typologies.

[Michele Simionato (@micheles)]

• Now the hazard statistics can be computed efficiently even in a single
  calculation, i.e. without the --hc option
• Added a check on the Python version in the oq command
• Reduced the data transfer when sending the site collection
• Changed the default filter_distance

[Daniele Viganò (@daniviga)]

• Fixed a bug where the PID was not saved into the database
  when using the command line interface
• Made it impossible to fire multiple CTRL-C in sequence
  to allow processes teardown and tasks revocation when Celery is used

[Michele Simionato (@micheles)]

• Used scipy.spatial.distance.cdist in Mesh.get_min_distance
• Prefiltered sites and assets in scenario calculations
• Made it possible to specify the filter_distance in the job.ini
• Made rtree optional again and disabled it in macOS
• Optimized the SiteCollection class and doubled the speed of distance
  calculations in most continental scale calculations
• Fixed an ordering bug in event based risk from GMFs when using a
  vulnerability function with PMF
• Replaced Rtree with KDtree except in the source filtering
• Parallelized the source prefiltering
• Removed the tiling feature from the classical calculator
• Undeprecated hazardlib.calc.stochastic.stochastic_event_set and
  made its signature right
• Removed the source typology from the ruptures and reduced the rupture
  hierarchy
• Removed the mesh spacing from PlanarSurfaces
• Optimized the instantiation of the rtree index
• Replaced the old prefiltering mechanism with the new one

[Daniele Viganò (@daniviga)]

• Managed the case of a dead controlling terminal (SIGHUP)

[Michele Simionato (@micheles)]

• Removed Decimal numbers from the PMF distribution in hazardlib
• Fixed another tricky bug with rtree filtering across the international
  date line
• Added a parameter prefilter_sources with values rtree|numpy|no
• Removed the prefiltering on the workers, resulting in a huge speedup
  for gridded ruptures at the cost of a larger data transfer
• Changed the losses_by_event output to export a single .csv file with
  all realizations
• Added a cross_correlation parameter used when working with shakemaps
• Now sites and exposure can be set at the same time in the job.ini
• Introduced a preclassical calculator
• Extended the scenario_damage calculator to export dmg_by_event
  outputs as well as losses_by_event outputs if there is a consequence
  model
• Unified region and region_constraint parameters in the job.ini
• Added a check to forbid duplicated GSIMs in the logic tree
• Introduced some changes to the realizations exporter (renamed field
  uid -> branch_path and removed the model field)
• Added a command oq celery inspect
• Reduced the check on too many realizations to a warning, except for
  event based calculations
• Improved the hazard exporter to exports only data for the filtered
  site collection and not the full site collection
• Extended the BCR exporter to export the asset tags

[Catalina Yepes (@CatalinaYepes)]

• Revised/enhanced the risk demos

[Michele Simionato (@micheles)]

• Added a warning about the option optimize_same_id_sources when the user
  should take advantage of it

[Daniele Viganò (@daniviga)]

• celery-status script converted into oq celery status command
• Removed Django < 1.10 backward compatibility
• Updated Python dependices (numpy 1.14, scipy 1.0.1,
  Django 1.10+, Celery 4+)

[Michele Simionato (@micheles)]

• Implemented scenario_risk/scenario_damage from shakemap calculators
• Exported the asset tags in the asset based risk outputs
• Fixed a numeric issue for nonparametric sources causing the hazard curves
  to saturate at high intensities
• Added an utility to download shakemaps
• Added an XML exporter for the site model
• Slight change to the correlation module to fix a bug in the SMTK
• Added a distribution mechanism threadpool

OpenQuake Engine 3.0.1

[Michele Simionato (@micheles)]

• Fixed a numeric bug affecting nonparametric sources and causing saturating
  hazard curves at high intensity

OpenQuake Engine 3.0.0

[Michele Simionato (@micheles)]

• Fixed a bug with newlines in the logic tree path breaking the CSV exporter
  for the realizations output
• When setting the event year, each stochastic event set is now considered
  independent
• Fixed a bug in the HMTK plotting libraries and added the ability to
  customize the figure size
• Fixed bug in the datastore: now we automatically look for the attributes
  in the parent dataset, if the dataset is missing in the child datastore
• Extended extract_losses_by_asset to the event based risk calculator
• Stored in source_info the number of events generated per source
• Added a script utils/reduce_sm to reduce the source model of a calculation
  by removing all the sources not affecting the hazard
• Deprecated openquake.hazardlib.calc.stochastic.stochastic_event_set
• Fixed the export of ruptures with a GriddedSurface geometry
• Added a check for wrong or missing <occupancyPeriods> in the exposure
• Fixed the issue of slow tasks in event_based_risk from precomputed GMFs
  for sites without events
• Now the engine automatically associates the exposure to a grid if
  region_grid_spacing is given and the sites are not specified otherwise
• Extracting the site mesh from the exposure before looking at the site model
• Added a check on probs_occur summing up to 1 in the SourceWriter
• oq show job_info now shows the received data amount while the
  calculation is progressing

[Daniele Viganò (@daniviga)]

• Removed support for Python 2 in setup.py
• Removed files containing Python 2 dependencies
• Added support for WebUI groups/permissions on the
  export outputs and datastore API endpoints

[Michele Simionato (@micheles)]

• Fixed oq show for multiuser with parent calculations
• Fixed get_spherical_bounding_box for griddedSurfaces
• Implemented disaggregation by source only for the case
  of a single realization in the logic tree (experimental)
• Replaced celery with celery_zmq as distribution mechanism
• Extended oq info to work on source model logic tree files
• Added a check against duplicated fields in the exposure CSV
• Implemented event based with mutex sources (experimental)
• Add an utility to read XML shakemap files in hazardlib
• Added a check on IMTs for GMFs read from CSV

[Daniele Viganò (@daniviga)]

• Changed the default DbServer port in Linux packages from 1908 to 1907

[Michele Simionato (@micheles)]

• Logged rupture floating factor and rupture spinning factor
• Added an extract API for losses_by_asset
• Added a check against GMF csv files with more than one realization
• Fixed the algorithm setting the event year for event based with sampling
• Added a command oq importcalc to import a remote calculation in the
  local database
• Stored avg_losses-stats in event based risk if there are multiple
  realizations
• Better error message in case of overlapping sites in sites.csv
• Added a an investigation time attribute to source models with
  nonparametric sources
• Bug fix: in some cases the calculator event_based_rupture was generating
  too few tasks and the same happened for classical calculation with
  `optimize_same_id_sources=true
• Changed the ordering of the epsilons in scenario_risk
• Added the ability to use a pre-imported risk model
• Very small result values in scenario_damage (< 1E-7) are clipped to zero,
  to hide numerical artifacts
• Removed an obsolete PickleableSequence class
• Fixed error in classical_risk when num_statistics > num_realizations
• Fixed a TypeError when reading CSV exposures with occupancy periods
• Extended the check on duplicated source IDs to models in format NRML 0.5
• Added a warning when reading the sources if .count_ruptures() is
  suspiciously slow
• Changed the splitting logic: now all sources are split upfront
• Improved the splitting of complex fault sources
• Added a script to renumber source models with non-unique source IDs
• Made the datastore of calculations using GMPETables relocatable; in
  practice you can run the Canada model on a cluster, copy the .hdf5 on
  a laptop and do the postprocessing there, a feat previously impossible.

[Valerio Poggi (@klunk386)]

• Included a method to export data directly from the Catalogue() object into
  standard HMTK format.

[Michele Simionato (@micheles)]

• Now the parameter disagg_outputs is honored, i.e. only the specified
  outputs are extracted from the disaggregation matrix and stored
• Implemented statistical disaggregation outputs (experimental)
• Fixed a small bug: we were reading the source model twice in disaggregation
• Added a check to discard results coming from the wrong calculation
  for the distribution mode celery_zmq
• Removed the long time deprecated commands
  oq engine --run-hazard and oq engine --run-risk
• Added a distribution mode celery_zmq
• Added the ability to use a preimported exposure in risk calculations
• Substantial cleanup of the parallelization framework
• Fixed a bug with nonparametric sources producing negative probabilities

OpenQuake Engine 2.9.0

[Michele Simionato (@micheles)]

• Deprecated the NRML format for the GMFs

[Matteo Nastasi (@nastasi-oq)]

• Debian package moved to Python 3.5

[Graeme Weatherill (@g-weatherill)]

• Small bug fix for Derras et al (2014) GMPE when Rjb = 0.0

[Michele Simionato (@micheles)]

• Improved the .rst reports for classical calculations with tiling
• Reduced the data transfer in the event based risk calculator by
  reading the event IDs directly from the workers
• Integrated the gmf_ebrisk calculator inside the event based calculator
• Improved the weighting algorithm for the sources in the event based
  rupture calculator
• Improved error message for source model files declared as nrml/0.5 when
  they actually are nrml/0.4
• Optimized the classical_bcr calculator for the case of many realizations
• Extended the exposure CSV importer to manage the retrofitted field

[Marco Pagani (@mmpagani), Changlong Li]

• Adds the Yu et al. (2013) GMPEs

[Michele Simionato (@micheles)]

• Fixed a bug in the hazard outputs: they were displayed in the WebUI even
  if they were missing
• Implemented splitting of nonparametric sources

[Marco Pagani (@mmpagani)]

• Fixed the 'get_closest_points' method for the
  openquake.hazardlib.geo.surface.gridded.GriddedSurface class

[Michele Simionato (@micheles)]

• Now the source model paths are relative to the source model logic tree file
• Fixed an international date line bug when using rtree for prefiltering
• Deprecated nrml.parse, it is now called nrml.to_python
• Improved the task distribution by splitting the AreaSources upfront
  and by improving the weighting algorithm

[Daniele Viganò (@daniviga)]

• TMPDIR can be customized via the openquake.cfg file
• Updated dependencies compatibility in setup.py

[Michele Simionato (@micheles)]

• If the hazard is precomputed, setting the site_model_file,
  gsim_logic_tree_file or source_model_logic_tree_file gives a warning
• Removed the obsolete API /extract/qgis- and added extract/hcurves,
  extract/hmaps, extract/uhs for use with the QGIS plugin
• Removed the deprecated GeoJSON exporters
• Fixed a bug with oq engine --run job.ini --exports npz
• Fixed the ordering of the IMTs in hazardlib
• oq engine --delete-calculation now aborts the calculation first
• Added some documentation about how to access the datastore
• Introduced a minimum_distance for the GSIMs
• Fixed several small issues with the UCERF calculators; now ucerf_hazard
  can be used as a precalculator of gmf_ebrisk
• Initial support for disaggregation by source
• Added the ability to import large exposures as CSV (experimental)
• Changed the source weights to be proportional to the number of GSIMs
  relevant for the tectonic region type, thus improving the task distribution

[Daniele Viganò (@daniviga)]

• The RPM python3-oq-engine package replaced python-oq-engine
• RPM packages moved to Python 3.5

[Michele Simionato (@micheles)]

• Added the ability to dump a specific calculation
• Changed the signature of the extract command to oq extract what calc_id,
  where what is a path info plus query string;

[Graeme Weatherill (@g-weatherill)]

• Implements significant duration GMPEs of Bommer et al. (2009) and Afshari &
  Stewart (2016)
• Adds significant duration IMT definitions to support IMTs

[Michele Simionato (@micheles)]

• Run the DbServer as a detached process
• Improved the test coverage for event based with GMF correlation
• Optimized the event based risk calculator from ruptures: now the ruptures
  are instantiated in the workers and not in the controller if possible
• Exported the parameter ses_per_logic_tree_path in the ruptures.csv file
• Improved the display names for the risk outputs
• Added a /v1/:calc_id/abort route to the engine server and Abort buttons
  to the WebUI
• Fixed the seeds properly in the case of vulnerability functions with PMFs:
  now even if the ground motion fields are all equal, the risk numbers
  will be different since there is a different seed per each field
• Stored a rupture loss table in event based risk calculations
• Made sure that the number of effective ruptures is stored in csm_info
• Fixed the HMTK tests to work with numpy from 1.11 to 1.14
• Added a command oq shell to open an embedded (I)Python shell
• Turned the 'realizations' output into a dynamic output

[Matteo Nastasi (@nastasi-oq)]

• Split package from python-oq-engine to python-oq-engine,
  python-oq-engine-master and python-oq-engine-worker
• Implemented an API /v1/on_same_fs to check filesystem accessibility
  between engine and a client application

[Michele Simionato (@micheles)]

• Reduced the data transfer when computing the hazard curves in postprocessing
• Removed the FilteredSiteCollection class

[Nick Ackerley (@nackerley)]

• Some improvements to the plotting routines of the HMTK

[Michele Simionato (@micheles)]

• Removed the ability to run oq engine --run job_h.ini,job_r.ini
• Forbidden the site model in gmf_ebrisk calculations
• When the option --hc is given the ruptures can now be read directly
  from the workers, thus saving some startup time
• Optimized the storage of the ruptures: the site IDs are not saved anymore
• Added a check for missing risk_investigation_time
• Reduced the data transfer in the gmf_ebrisk calculator
• Now the gmf_ebrisk calculator builds the aggregate loss curves too
• Extended the gmf_ebrisk calculator to use the GMFs produced by an event
  based hazard calculation, both via CSV and via the --hc option
• Fixed a performance bug in the computations of the aggregate loss curves
  by reading the full event loss table at once
• Fixed oq zip to work with gmf_ebrisk calculations
• Fixed a serious bug in the gmf_ebrisk calculator: the results were in most
  cases wrong and dependent on the number of spawned tasks
• Now the master_seed controls the generation of the epsilons in all
  situations (before in event_based_risk without asset_correlation it was
  managed by random_seed)
• Changed the management of the epsilons in the gmf_ebrisk calculator to
  be the same as in the event_based_risk calculator
• Added a check on the input files: the listed paths must be relative paths
• Fixed a bug when storing the disagg-bins in the case the lenghts of the
  arrays are not the same for all sites
• In the case of a single tile the prefiltering was applied twice: fixed the
  problem and generally improved the filtering/weighting of the sources
• Fixed the CSV exporter for disaggregation outputs when iml_disagg is set

[Graeme Weatherill (@g-weatherill)]

• Fixed ASK14 GMPE behaviour to remove ValueError
• Implements comprehensive suite of NGA East ground motion models for
  central and eastern United States
• Minor modification of check_gsim functions to permit instantiated classes
  to be passed

OpenQuake Engine 2.8.0

[Michele Simionato (@micheles)]

• iml_disagg and poes_disagg cannot coexist in the job.ini file
• Added a check on conditional_loss_poes in the event_based_risk calculator:
  now it requires asset_loss_table to be set

[Anirudh Rao (@raoanirudh)]

• Sorted taxonomies before comparison in the BCR calculator

[Michele Simionato (@micheles)]

• Optimized the disaggregation calculation by performing the PMF extraction
  only once at the end of the calculation and not in the workers
• Added an oq zip command
• Avoided running an useless classical calculation if iml_disagg is given

[Valerio Poggi (@klunk386)]

• Implemented subclasses for ZhaoEtAl2006Asc and AtkinsonBoore2006 to account
  for the distance filter used by SGS in their PSHA model for Saudi Arabia.
  Distance threshold is hard coded to 5km in this implementation.

[Michele Simionato (@micheles)]

• Added a warning if the aggregated probability of exceedence (poe_agg) in
  a disaggregation output is very dissimilar from poes_disagg
• Removed the flag split_sources
• Optimized the operation arrange_data_in_bins in the disaggregation
  calculator and reduced the data transfer by the number of tectonic
  region types
• Improved the sending of the sources to the workers, especially for the
  MultiPointSources
• Better error message if the user sets a wrong site_id in the sites.csv file
• Now the distance and lon lat bins for disaggregation are built directly
  from the integration distance
• Used uniform bins for disaggregation (before they were potentially
  different across realizations / source models)
• Improved the error message if the user forgets both sites and sites.csv
  in a calculation starting from predetermined GMFs
• Improved the error message if the user specifies a non-existing file in
  the job.ini
• Change the ordering of the TRT bins in disaggregation: now they are
  ordered lexicographically
• Added more validity checks on the job.ini file for disaggregation
• Changed the .hdf5 format generated by oq extract -1 hazard/rlzs; you can
  still produce the old format by using oq extract -1 qgis-hazard/rlzs
• Optimized the disaggregation calculator by instantiating
  scipy.stats.truncnorm only once per task and not once per rupture
• Optimized the disaggregation calculator when iml_disagg is specified,
  by caching duplicated results
• Made sure that oq dbserver stop also stops the zmq workers if the zmq
  distribution is enabled
• Added a check when disaggregating for a PoE too big for the source model
• If iml_disagg is given, forbid intensity_measure_types_and_levels
• Fixed the disaggregation outputs when iml_disagg is given: the PoE has
  been removed by the name of the output and correct PoE is in the XML file
• Fixed oq export loss_curves/rlzs for event_based_risk/case_master
• Removed the obsolete parameter loss_curve_resolution
• Fixed a Python 3 unicode error with oq engine --run job.zip
• Added a command oq abort <calc_id>
• Stored the avg_losses in classical risk in the same way as in
  event_based_risk and made them exportable with the same format
• Removed the outputs losses_by_tag from the event based risk calculators
  and changed the default for the avg_losses flag to True
• WebUI: now every job runs in its own process and has name oq-job-
• Refactored the WebUI tests to use the DbServer and django.test.Client
• Added an experimental feature optimize_same_id_sources
• Fixed a bug in gmf_ebrisk when there are zero losses and added more
  validity checks on the CSV file
• The parameter number_of_ground_motion_fields is back to being optional in
  scenario calculators reading the GMFs from a file, since it can be inferred
• Removed the deprecated risk outputs dmg_by_tag, dmg_total,
  losses_by_tag, losses_total
• Deprecated the .geojson exporters for hazard curves and maps
• We now keep the realization weights in case of logic tree sampling (before
  they were rescaled to 1 / R and considered all equals)
• Optimized sampling for extra-large logic trees
• Added a check on missing source_model_logic_tree
• Fix to the gmf_ebrisk calculator: the realization index in the event loss
  table was incorrect and too many rows were saved
• Added a way to restrict the source logic model tree by setting a sm_lt_path
  variable in the job.ini (experimental)
• Fixed the precedence order when reading openquake.cfd

OpenQuake Engine 2.7.0

[Michele Simionato (@micheles)]

• Fixed the risk exporters for tags containing non-ASCII characters

[Valerio Poggi (@klunk386)]

• Implemented the Pankow and Pechmann 2004 GMPE (not verified)

[Graeme Weatherill (@g-weatherill)]

• Added Derras et al. (2014) GMPE
• Implemented the Zhao et al. (2016) GMPE for active shallow crustal,
  subduction interface and subduction slab events
• Adds 'rvolc' (volcanic path distance) to the distance context

[Michele Simionato (@micheles)]

• The outputs loss_curves-rlzs and loss_curves-stats are now visible again
  as engine outputs (before they were hidden)
• Added a debug command oq plot_assets and fixed oq plot_sites
• Added a flag --multipoint to the command oq upgrade_nrml
• Deprecated several outputs: hcurves-rlzs, agg_loss_table, losses_total,
  dmg_by_tag, dmg_total, losses_by_tag, losses_by_tag-rlzs
• Extended the command oq extract job_id to check the database
• Optimized the scenario damage calculator by vectorizing the calculation
  of the damage states
• Extended the FragilityFunctions to accept sequences/arrays of intensity
  levels, as requested by Hyeuk Ryu

[Daniele Viganò (@daniviga)]

• Added support for groups in the WebUI/API server

[Michele Simionato (@micheles)]

• Added an experimental distribution mode of kind "zmq"
• Implemented an API /extract/agglosses/loss_type?tagname1=tagvalue1&...
  with the ability to select all tagvalues (*) for a given tagname
• Deprecated reading hazard curves from CSV, since it was an experimental
  features and nobody is using it
• Changed the exporter of the event loss table to export all realizations
  into a single file

[Graeme Weatherill (@g-weatherill)]

• Adds the Bindi et al. (2017) GMPEs for Joyner-Boore and Hypocentral
  Distance

[Michele Simionato (@micheles)]

• Made it mandatory to specify the sites for all calculators reading the
  GMFs from a CSV file
• Tested also the case of calculators requiring a shared_dir
• Improved the error checking when parsing vulnerability functions with PMF

[Daniele Viganò (@daniviga)]

• Fixed a bug in oq reset command which was not stopping
  the DbServer properly

[Michele Simionato (@micheles)]

• Implemented an API /extract/aggcurves/loss_type?tagname1=tagvalue1&...
• Implemented an API /extract/aggdamages/loss_type?tagname1=tagvalue1&...
• Every time a new calculation starts, we check if there is a newer version
  of the engine available on GitHub
• Changed the search logic for the configuration file openquake.cfg
• Implemented an API /extract/agglosses/loss_type?tagname1=tagvalue1&...
• Fixed several bugs in the gmf_ebrisk calculator, in particular in presence
  of asset correlation and missing values on some sites
• Fixed a bug with logging configured a WARN level instead of INFO level
  if rtree was missing (affecting only oq run)
• Changed the ScenarioDamage demo to use two GSIMs
• Added a node <tagNames> in the exposure
• Added a web API to extract the attributes of a datastore object
• Fixed oq to_shapefile and oq from_shapefile to work with NRML 0.5
  (except MultiPointSources)
• Added information about the loss units to the agg_curve outputs
• oq extract hazard/rlzs now extracts one realization at the time
• The rupture filtering is now applied during disaggregation
• Changed the /extract wen API to return a compressed .npz file
• Fixed a bug with multi-realization disaggregation, celery and no
  shared_dir: now the calculation does not hang anymore

OpenQuake Engine 2.6.0

[Michele Simionato (@micheles)]

• Fixed the GMF .npz export when the GMFs are extracted from a file
• Stored the number of nonzero losses per asset and realization in
  event_based_risk calculations with asset_loss_table=True

[Daniele Viganò (@daniviga)]

• Fixed 'openquake' user creation in RPM when SELinux is in enforcing mode
• Changed the behaviour during RPM upgrades:
  the old openquake.cfg configuration file is left untouched and the new one
  installed as openquake.cfg.rpmnew

[Michele Simionato (@micheles)]

• Added a check on number_of_ground_motion_fields when the GMFs are
  extracted from a NRML file
• Added a command oq extract able to extract hazard outputs into HDF5 files
• Fixed a bug when reading GMFs from a NRML file: the hazard sites were
  read from the exposure (incorrectly) and not from the GMFs
• Fixed a bug in MultiMFDs of kind arbitraryMFD

[Valerio Poggi (@klunk386)]

• Implemented the Atkinson (2010) GMPE as subclass Atkinson2010Hawaii
  of BooreAtkinson2008

[Michele Simionato (@micheles)]

• Used the new loss curves algorithm for the asset loss curves and loss maps
• Added a generic extract functionality to the web API
• Fixed a bug when computing the rjb distances with multidimensional meshes
• Changed the GMF CSV exporter to export the sites too; unified it with the
  event based one

[Daniele Viganò (@daniviga)]

• Changed the 'CTRL-C' behaviour to make sure that all children
  processes are killed when a calculation in interrupted

[Michele Simionato (@micheles)]

• Fixed a bug in the statistical loss curves exporter for classical_risk
• Replaced the agg_curve outputs with losses by return period outputs
• Turned the DbServer into a multi-threaded server
• Used zmq in the DbServer
• Fixed correct_complex_sources.py
• Fixed oq export hcurves-rlzs -e hdf5
• Changed the source weighting algorithm: now it is proportional to the
  the number of affected sites
• Added a command oq show dupl_sources and enhances oq info job.ini
  to display information about the duplicated sources
• Added a flag split_sources in the job.ini (default False)
• Updated the demos to the format NRML 0.5

[Valerio Poggi (@klunk386)]

• Implemented the Munson and Thurber 1997 (Volcanic) GMPE

[Graeme Weatherill (@g-weatherill)]

• Adapts CoeffsTable to be instantiated with dictionaries as well as strings

[Daniele Viganò (@daniviga)]

• Extended the 'oq reset' command to work on multi user installations

[Michele Simionato (@micheles)]

• Fixed a bug: if there are multiple realizations and no hazard stats,
  it is an error to set hazard_maps=true or uniform_hazard_spectra=true
• Implemented aggregation by asset tag in the risk calculators
• Fixed a small bug in the HMTK (in get_depth_pmf)
• Extended the demo LogicTreeCase1ClassicalPSHA to two IMTs and points
• Added a documentation page oq-commands.md
• Removed the automatic gunzip functionality and added an automatic
  checksum functionality plus an oq checksum command
• Made the demo LogicTreeCase2ClassicalPSHA faster
• Fixed the export by realization of the hazard outputs
• Changed the generation of loss_maps in event based risk, without the option
  --hc: now it is done in parallel, except when reading the loss ratios
• Renamed --version-db to --db-version, to avoid
  confusions between oq --version and oq engine -version
• Fixed bug in the exported outputs: a calculation cannot export the results
  of its parent
• Extended the sz field in the rupture surface to 2 bytes, making it
  possible to use a smaller mesh spacing
• Changed the ordering of the fields in the loss curves and loss maps
  generated by the event based risk calculator; now the insured fields
  are at the end, before they were intermixed with each loss type
• Changed the format of array all_loss_ratios/indices
• The size in bytes of the GMFs was saved incorrectly
• Added an exporter gmf_scenario/rup-XXX working also for event based
• First version of the calculator gmf_ebrisk
• Implemented risk statistics for the classical_damage calculator
• Added a .csv importer for the ground motion fields
• Implemented risk statistics for the classical_bcr calculator

[Armando Scarpati (@hascar)]

• Show to the user the error message when deleting a calculation
  in the WebUI fails

[Michele Simionato (@micheles)]

• Better error message when running a risk file in absence of hazard
  calculation
• Changed the sampling logic in event based calculators
• Imported GMFs from external file into the datastore

[Daniele Viganò (@daniviga)]

• Added the 'celery-status' script in 'utils' to check the
  task distribution in a multi-node celery setup

[Michele Simionato (@micheles)]

• Removed an excessive check from the WebUI: now if an output exists,
  it can be downloaded even if the calculation was not successful

[Armando Scarpati (@hascar)]

• Visualized the calculation_mode in the WebUI

[Michele Simionato (@micheles)]

• Made the upgrade_manager transactional again
• Changed the storage of the GMFs; as a consequence the exported .csv
  has a different format

[Daniele Viganò (@daniviga)]

• Fixed a bug introduced by a change in Django 1.10 that was causing
  the HTTP requests log to be caught by our logging system and
  then saved in the DbServer
• Updated requirements to allow installation of Django 1.11 (LTS)

[Michele Simionato (@micheles)]

• Added two commands oq dump and oq restore
• Added a check that on the number of intensity measure types when
  generating uniform hazard spectra (must be > 1)

OpenQuake Engine 2.5.0

[Armando Scarpati (@hascar)]

• Added a confirmation dialog when trying to remove a calculation via the
  WebUI

[Michele Simionato (@micheles)]

• Hazard maps were not exposed to the engine in event based calculations
• Fixed the check on the DbServer instance: it was failing in presence
  of symbolic links
• Optimized MultiMFD objects for the case of homogeneous parameters
• Added an .npz exporter for the scenario_damage output dmg_by_asset
• Removed the pickled CompositeSourceModel from the datastore
• Improved the error message when the rupture mesh spacing is too small
• Unified the versions of baselib, hazardlib and engine
• Raised a clear error if the user does not set the calculation_mode
• Made it is possible to pass the hdf5 full path to the DataStore class
• Made it possible to use CELERY_RESULT_BACKEND != 'rpc://'

[Michele Simionato (@micheles), Daniele Viganò (@daniviga)]

• Merged the oq-hazardlib repository into oq-engine.
  The python-oq-hazardlib package is now provided by python-oq-engine

[Michele Simionato (@micheles)]

• Added CSV exports for the agg_curve outputs
• Fixed a bug in oq export hcurves-rlzs --exports hdf5
• Restored the parameter sites_per_tile with a default of 20,000, i.e.
  tiling starts automatically if there are more than 20,000 sites
• Better error message for invalid exposures
• Removed the deprecated XML outputs of the risk calculators
• Added an end point v1/calc/XXX/oqparam to extract the calculation
  parameters as a JSON dictionary
• Fixed the excessive logic tree reduction in event based calculators
• Improved the command oq db
• Fixed an encoding bug when logging a filename with a non-ASCII character
• Fixed a bug when exporting a GMF with ruptureId=0
• Added a parameter disagg_outputs to specify the kind of disaggregation
  outputs to export
• Raised an early error if the consequence model is missing some taxonomies
• Restored the tiling functionality in the classical calculator; to enable
  it, set num_tiles in the job.ini file
• If there are no statistical hazard curves to compute, do not transfer
  anything
• Fixed a small bug in oq plot and added a test

[Daniele Viganò (@daniviga)]

• Added collectstatic and createsuperuser subcommands to the
  oq webui command
• Added a local_settings.py.pam template to use PAM as the authentication
  provider for API and WebUI
• Now the command oq webui start tries to open a browser tab
  with the WebUI loaded

OpenQuake Engine 2.4.0

[Michele Simionato (@micheles)]

• Now the command oq export loss_curves/rlz-XXX works both for the
  classical_risk calculator and the event_based_risk calculator

[Daniele Viganò (@daniviga)]

• Remove the default 30 day-old view limit in the WebUI calculation list

[Michele Simionato (@micheles)]

• Fixed a broken import affecting the command oq upgrade_nrml
• Made it possible to specify multiple file names in
  in the source_model_logic_tree file
• Reduced the data transfer in the object RlzsAssoc and improved the
  postprocessing of hazard curves when the option --hc is given
• Changed the ruptures.xml exporter to export unique ruptures
• Fixed a bug when downloading the outputs from the WebUI on Windows
• Made oq info --report fast again by removing the rupture fine filtering
• Improved the readibility of the CSV export dmg_total
• Removed the column eid from the CSV export ruptures; also
  renamed the field serial to rup_id and reordered the fields
• Changed the event loss table exporter: now it exports an additional
  column with the rup_id
• Changed scenario npz export to export also the GMFs outside the maximum
  distance
• Fixed scenario npz export when there is a single event
• Replaced the event tags with numeric event IDs
• The mean hazard curves are now generated by default
• Improved the help message of the command oq purge
• Added a @reader decorator to mark tasks reading directly from the
  file system
• Removed the .txt exporter for the GMFs, used internally in the tests
• Fixed a bug with relative costs which affected master for a long time,
  but not the release 2.3. The insured losses were wrong in that case.
• Added an .hdf5 exporter for the asset loss table
• Loss maps and aggregate losses are computed in parallel or sequentially
  depending if the calculation is a postprocessing calculation or not
• Deprecated the XML risk exporters
• Removed the .ext5 file
• Restored the parameter asset_loss_table in the event based calculators
• Added a full .hdf5 exporter for hcurves-rlzs
• Removed the individual_curves flag: now by default only the statistical
  hazard outputs are exported
• Saved a lot of memory in the computation of the hazard curves and stats
• Renamed the parameter all_losses to asset_loss_table
• Added an experimental version of the event based risk calculator which
  is able to use GMFs imported from an external file
• Added a max_curve functionality to compute the upper limit of the
  hazard curves amongst realizations
• Raised an error if the user specifies quantile_loss_curves
  or conditional_loss_poes in a classical_damage calculation
• Added a CSV exporter for the benefit-cost-ratio calculator
• The classical_risk calculator now reads directly the probability maps,
  not the hazard curves
• Turned the loss curves into on-demand outputs
  for the event based risk calculator
• The loss ratios are now stored in the datastore and not in an
  external .ext5 file
• The engine outputs are now streamed by the WebUI
• Used a temporary export directory in the tests, to avoid conflicts
  in multiuser situations
• Added an .npz exporter for the loss maps
• Raised an error early when using a complex logic tree in scenario
  calculations
• Changed the CSV exporter for the loss curves: now it exports all the
  curves for a given site for the classical_risk calculator
• Fixed the save_ruptures procedure when there are more than 256
  surfaces in the MultiSurface
• Renamed the csq_ outputs of the scenario_damage to losses_
• Changed the way scenario_damage are stored internally to be more
  consistent with the other calculators
• Removed the GSIM from the exported file name of the risk outputs
• New CSV exporter for GMFs generated by the event based calculator
• The event IDs are now unique and a constraint on the maximum
  number of source groups (65,536) has been added
• Added an output losses_by_event to the scenario_risk calculator
• Changed the output ruptures.csv to avoid duplications
• Added an output losses_by_taxon to the scenario_risk calculator
• Fixed a performance bug in get_gmfs: now the scenario risk and damage
  calculators are orders of magnitude faster for big arrays
• Added an export test for the event loss table in the case of multiple TRTs
• Removed the experimental rup_data output
• Added an .npz export for the output losses_by_asset
• Exported the scenario_risk aggregate losses in a nicer format

[Daniele Viganò (@daniviga)]

• The 'oq webui' command now works on a multi-user installation
• Splitted RPM packages into python-oq-engine (single node) and
  python-oq-engine-master/python-oq-engine-worker (multi-node)

[Paolo Tormene (@ptormene)]

• The 'Continue' button in the Web UI is now available also for risk
  calculations

[Michele Simionato (@micheles)]

• Fixed a Python 3 bug in the WebUI when continuing a calculation: the
  hazard_calculation_id was passed as a string and not as an integer
• Changed to rupture storage to use variable length-arrays, with a speedup
  of two orders of magnitude
• Avoided storing twice the rupture events
• Optimized the serialization of ruptures on HDF5 by using a sids output
• Changed the Web UI button from "Run Risk" to "Continue"
• The avg field in the loss curves is computed as the integral of the curve
  again, and it is not extracted from the avg_losses output anymore
• Made the fullreport exportable
• Fixed the rup_data export, since the boundary field was broken
• Restored the output losses_by_taxon in the event_based_risk calculator
• Fixed the calculator event based UCERF so that average losses can
  be stored

[Daniele Viganò (@daniviga)]

• Added a check to verify that an 'oq' client is talking to the
  right DbServer instance
• Introduced an optional argument for 'oq dbserver' command line
  to be able to override its default interface binding behaviour

[Michele Simionato (@micheles)]

• Optimized the event based calculators by reducing the number of calls
  to the GmfComputer and by using larger arrays
• Added a check on missing vulnerability functions for some loss type
  for some taxonomy
• Now we save the GMFs on the .ext5 file, not the datastore
• Fixed bug in event_based_risk: it was impossible to use vulnerability
  functions with "PM" distribution
• Fixed bug in event_based_risk: the ebrisk calculator is required as
  precalculator of event_based_risk, not others
• Fixed bug in scenario_risk: the output all_losses-rlzs was aggregated
  incorrectly
• Now the ucerf_risk calculators transfer only the events, not the ruptures,
  thus reducing the data transfer of several orders of magnitude
• Added a view get_available_gsims to the WebUI and fixed the API docs
• Introduced a configuration parameter max_site_model_distance with default
  of 5 km
• Implemented sampling in the UCERF event based hazard calculator

[Daniele Viganò (@daniviga)]

• Use threads instead of processes in DbServer because SQLite3
  isn't fork-safe on macOS Sierra

[Michele Simionato (@micheles)]

• Fixed a TypeError when deleting a calculation from the WebUI
• Extended the command oq to_hdf5 to manage source model files too
• Improved significantly the performance of the event based calculator
  when computing the GMFs and not the hazard curves
• Stored information about the mean ground motion in the datastore
• Saved the rupture mesh with 32 floats instead of 64 bit floats
• Raised the limit on the event IDs from 2^16 to 2^32 per task
• Fixed classical_risk: there was an error when computing the statistics
  in the case of multiple assets of the same taxonomy on the same site
• Changed the UCERF event based calculators to parallelize by SES
• Fixed a site model bug: when the sites are extracted from the site model
  there is no need to perform geospatial queries to get the parameters
• Added a command oq normalize to produce good sites.csv files
• Introduced a ses_seed parameter to specify the seed used to generate
  the stochastic event sets; random_seed is used for the sampling only
• Changed the build_rcurves procedure to read the loss ratios directly from
  the workers