Remove curative stop criterion and 'optimize curative if preventive u…

…nsecure' and partially replace them by 'enforce curative security'

NB. Curative min objective does no longer exist and is replaced by Curative have the same objective than the Preventive results

Signed-off-by: Pauline Jean-Marie <pauline.jean-marie@artelys.com>

docs/parameters.md

@@ -51,34 +51,13 @@ These parameters (objective-function) configure the remedial action optimisation
   - **SECURE**: the search-tree will stop as soon as it finds a solution where the minimum margin is positive.  
   *Note: if the best possible minimum margin is negative, both stop criterion will return the same solution.*
 
-#### curative-stop-criterion
-- **Expected value**: one of the following:
-  - "MIN_OBJECTIVE"
-  - "SECURE"
-  - "PREVENTIVE_OBJECTIVE"
-  - "PREVENTIVE_OBJECTIVE_AND_SECURE"
-- **Default value**: "MIN_OBJECTIVE"
-- **Usage**: stop criterion for the curative RAO search-tree.  
-  - **MIN_OBJECTIVE**: the search-tree will maximize the minimum margin until it converges to a
-    maximum value, or until another stop criterion has been reached (e.g. [max-curative-search-tree-depth](#max-curative-search-tree-depth)).  
-  - **SECURE**: the search-tree will stop as soon as it finds a solution where the minimum margin is positive.  
-  - **PREVENTIVE_OBJECTIVE**: the search-tree will stop as soon as the preventive RAO's objective value is reached, 
-    and improved by at least ["curative-rao-min-obj-improvement"](#curative-min-obj-improvement).  
-  - **PREVENTIVE_OBJECTIVE_AND_SECURE**: the search-tree will stop as soon as the preventive RAO's objective value is 
-    reached, and improved by at least ["curative-rao-min-obj-improvement"](#curative-min-obj-improvement), and all 
-    margins are positive.  
-
-  *The values "PREVENTIVE_OBJECTIVE" and "PREVENTIVE_OBJECTIVE_AND_SECURE" allow you to speed up the curative RAO
-  without deteriorating the final solution (minimum margin over all perimeters). However, using them means the flow-based
-  domain is not maximised for all perimeters.*
-
 #### curative-min-obj-improvement
 - **Expected value**: numeric value, where the unit is that of the objective function
 - **Default value**: 0
 - **Usage**: used as a minimum improvement of the preventive RAO objective value for the curative RAO stop criterion,
   when it is set to PREVENTIVE_OBJECTIVE or PREVENTIVE_OBJECTIVE_AND_SECURE.
 
-#### optimize-curative-if-preventive-unsecure
+#### enforce-curative-security
 - **Expected value**: true/false
 - **Default value**: false
 - **Usage**: if this parameter is set to true, OpenRAO will continue optimizing curative states even if preventive state
@@ -314,7 +293,7 @@ These parameters (second-preventive-rao) tune the behaviour of the [second preve
     during curative RAO  
   - **POSSIBLE_CURATIVE_IMPROVEMENT**: a 2nd preventive RAO is run only if it is possible to improve a curative perimeter,
     i.e. if the curative RAO stop criterion on at least one contingency is not reached.  
-    This depends on the value of parameter [curative-stop-criterion](#curative-stop-criterion):
+    This depends on the value of parameter [preventive-stop-criterion](#preventive-stop-criterion):
     - **SECURE**: 2nd preventive RAO is run if one curative perimeter is not secure after optimisation
     - **PREVENTIVE_OBJECTIVE**: 2nd preventive RAO is run if one curative perimeter reached an objective function value
       after optimisation that is worse than the preventive perimeter's (decreased by [curative-min-obj-improvement](#curative-min-obj-improvement))
@@ -575,8 +554,7 @@ Zones are seperated by + or -.
     "type" : "MAX_MIN_RELATIVE_MARGIN_IN_AMPERE",
     "curative-min-obj-improvement" : 0.0,
     "preventive-stop-criterion" : "SECURE",
-    "curative-stop-criterion" : "PREVENTIVE_OBJECTIVE",
-    "optimize-curative-if-preventive-unsecure" : true
+    "enforce-curative-security" : true
   },
   "range-actions-optimization" : {
     "max-mip-iterations" : 5,
@@ -693,7 +671,6 @@ Based on PowSyBl's [configuration mechanism](inv:powsyblcore:std:doc#user/config
 rao-objective-function:
   type: MAX_MIN_MARGIN_IN_AMPERE
   preventive-stop-criterion: SECURE
-  curative-stop-criterion: SECURE
 
 rao-range-actions-optimization:
   max-mip-iterations: 5

python-util/rao_parameter_modification.py

@@ -6,35 +6,114 @@
 
 import os
 import json
+from json import JSONDecodeError
+import yaml
 
-root_directory = os.path.join(os.getcwd(), "..")
+current_directory = os.getcwd()
 
 
+tag_by_file_type = {"json": "objective-function", "yaml": "rao-objective-function"}
+
 def rao_parameters_file(file_path):
     correct_version = False
-    obj_fun = False
-    if file_path.endswith(".json") or file_path.endswith(".yml"):
+    score = 0
+    if "target" not in file_path and (file_path.endswith(".json") or file_path.endswith(".yml")):
+        ftype = "json" if file_path.endswith(".json") else "yaml"
         with open(os.path.join(dirpath, filename), 'r') as file:
             for line in file:
-                if "objective-function" in line:
-                    obj_fun = True
                 if '"version" : "2.4"' in line or '"version" : "2.5"' in line:
                     correct_version = True
-                if correct_version and obj_fun:
+                if tag_by_file_type[ftype] in line:
+                    score += 1
+                if "MAX_MIN_MARGIN_IN" in line or "MAX_MIN_RELATIVE_MARGIN_IN" in line:
+                    score += 1
+                if correct_version and score >= 2:
                     return True
     return False
 
+def read_data(file_path) -> tuple[dict, str]:
+    if file_path.endswith(".json"):
+        with open(file_path, 'r') as file:
+            try:
+                return json.load(file), "json"
+            except JSONDecodeError as je:
+                print("in file " + file_path)
+                raise je
+    if file_path.endswith(".yml"):
+        with open(file_path, 'r') as file:
+            return yaml.safe_load(file), "yaml"  # ["rao-parameters"]
+
+def extract_leading_whitespace(line):
+    leading_whitespace = ""
+    for char in line:
+        if char.isspace():
+            leading_whitespace += char
+        else:
+            break
+    return leading_whitespace
+
+def write_data(new_data, file_path, file_type):
+    with open(file_path, 'r') as file:
+        lines = file.readlines()
+        lines_to_write = []
+        inside_obj_fun_to_replace = False
+        for line in lines:
+            if tag_by_file_type[file_type] in line and tag_by_file_type[file_type] in new_data:
+                leading_whitespace = extract_leading_whitespace(line)
+                inside_obj_fun_to_replace = True
+            if inside_obj_fun_to_replace and ((file_type == "json" and "}" in line) or (file_type == "yaml" and line == "\n")):
+                obj_fun_str = f'"{tag_by_file_type[file_type]}" : ' + obj_function_as_str_lines(new_data, file_type)
+                for new_line in obj_fun_str.splitlines(True):
+                    lines_to_write.append(leading_whitespace + new_line)
+                inside_obj_fun_to_replace = False
+            elif not inside_obj_fun_to_replace:
+                lines_to_write.append(line)
+    with open(file_path, 'w') as file:
+        file.writelines(lines_to_write)
+
+
+def obj_function_as_str_lines(new_data, file_type):
+    if file_type == "json":
+        return json.dumps(new_data[tag_by_file_type[file_type]], indent=2, separators=(',', ' : ')) + ',\n'
+    else:
+        return yaml.dump(new_data[tag_by_file_type[file_type]], default_flow_style=False) + '\n'
+
+
+
+def new_rao_param(data: dict, file_path: str, file_type: str) -> dict:
+    try:
+        old_obj_fun = data[tag_by_file_type[file_type]]
+        new_obj_fun = {}
+        for key in old_obj_fun:
+            if key not in ("curative-stop-criterion", "optimize-curative-if-preventive-unsecure"):
+                new_obj_fun[key] = old_obj_fun[key]
+        prev_secure = "preventive-stop-criterion" not in old_obj_fun or old_obj_fun["preventive-stop-criterion"] == "SECURE"
+        if prev_secure:
+            if "optimize-curative-if-preventive-unsecure" in old_obj_fun:
+                new_obj_fun["enforce-curative-security"] = old_obj_fun["optimize-curative-if-preventive-unsecure"]
+        else:
+            cur_secure = "curative-stop-criterion" in old_obj_fun and old_obj_fun["curative-stop-criterion"] in ("SECURE", "PREVENTIVE_OBJECTIVE_AND_SECURE")
+            if cur_secure:
+                new_obj_fun["enforce-curative-security"] = True
+            else:
+                new_obj_fun["enforce-curative-security"] = False
+                cur_min = "curative-stop-criterion" in old_obj_fun and old_obj_fun["curative-stop-criterion"] == "MIN_OBJECTIVE"
+                if cur_min:
+                    new_obj_fun["curative-min-obj-improvement"] = 10000.0 + (old_obj_fun["curative-min-obj-improvement"] if "curative-min-obj-improvement" in old_obj_fun else 0.0)
+    except KeyError as ke:
+        raise KeyError("in file " + file_path) from ke
+    data[tag_by_file_type[file_type]] = new_obj_fun
+    return data
+
+
 
-for dirpath, dirnames, filenames in os.walk(root_directory):
-    for filename in filenames:
-        file_path = os.path.join(dirpath, filename)
-        if rao_parameters_file(file_path):
-            print("file to change : " + file_path)
-            lines = None
-            if file_path.endswith(".json"):
-                with open(file_path, 'r') as file:
-                    lines = file.readlines()
-                    lines = [line.replace('"version" : "2.4"', '"version" : "2.5"') for line in lines if "forbid-cost-increase" not in line]
-            if lines is not None:
-                with open(file_path, 'w') as file:
-                    file.writelines(lines)
+if __name__ == "__main__":
+    base_dir = os.path.join(current_directory, "..")
+    print(base_dir)
+    for dirpath, dirnames, filenames in os.walk(base_dir):
+        for filename in filenames:
+            file_path = os.path.join(dirpath, filename)
+            if rao_parameters_file(file_path):
+                data, file_type = read_data(file_path)
+                new_rao_params = new_rao_param(data, file_path, file_type)
+                write_data(new_rao_params, file_path, file_type)

ra-optimisation/rao-api/src/main/java/com/powsybl/openrao/raoapi/RaoParametersCommons.java

@@ -27,8 +27,7 @@ private RaoParametersCommons() {
     public static final String TYPE = "type";
     public static final String CURATIVE_MIN_OBJ_IMPROVEMENT = "curative-min-obj-improvement";
     public static final String PREVENTIVE_STOP_CRITERION = "preventive-stop-criterion";
-    public static final String CURATIVE_STOP_CRITERION = "curative-stop-criterion";
-    public static final String OPTIMIZE_CURATIVE_IF_PREVENTIVE_UNSECURE = "optimize-curative-if-preventive-unsecure";
+    public static final String ENFORCE_CURATIVE_SECURITY = "enforce-curative-security";
 
     // range actions optimization parameters
     public static final String RANGE_ACTIONS_OPTIMIZATION = "range-actions-optimization";

ra-optimisation/rao-api/src/main/java/com/powsybl/openrao/raoapi/json/JsonObjectiveFunctionParameters.java

@@ -28,9 +28,8 @@ static void serialize(RaoParameters parameters, JsonGenerator jsonGenerator) thr
         jsonGenerator.writeObjectFieldStart(OBJECTIVE_FUNCTION);
         jsonGenerator.writeObjectField(TYPE, parameters.getObjectiveFunctionParameters().getType());
         jsonGenerator.writeObjectField(PREVENTIVE_STOP_CRITERION, parameters.getObjectiveFunctionParameters().getPreventiveStopCriterion());
-        jsonGenerator.writeObjectField(CURATIVE_STOP_CRITERION, parameters.getObjectiveFunctionParameters().getCurativeStopCriterion());
         jsonGenerator.writeNumberField(CURATIVE_MIN_OBJ_IMPROVEMENT, parameters.getObjectiveFunctionParameters().getCurativeMinObjImprovement());
-        jsonGenerator.writeBooleanField(OPTIMIZE_CURATIVE_IF_PREVENTIVE_UNSECURE, parameters.getObjectiveFunctionParameters().getOptimizeCurativeIfPreventiveUnsecure());
+        jsonGenerator.writeBooleanField(ENFORCE_CURATIVE_SECURITY, parameters.getObjectiveFunctionParameters().getEnforceCurativeSecurity());
         jsonGenerator.writeEndObject();
     }
 
@@ -43,16 +42,13 @@ static void deserialize(JsonParser jsonParser, RaoParameters raoParameters) thro
                 case PREVENTIVE_STOP_CRITERION:
                     raoParameters.getObjectiveFunctionParameters().setPreventiveStopCriterion(stringToPreventiveStopCriterion(jsonParser.nextTextValue()));
                     break;
-                case CURATIVE_STOP_CRITERION:
-                    raoParameters.getObjectiveFunctionParameters().setCurativeStopCriterion(stringToCurativeStopCriterion(jsonParser.nextTextValue()));
-                    break;
                 case CURATIVE_MIN_OBJ_IMPROVEMENT:
                     jsonParser.nextToken();
                     raoParameters.getObjectiveFunctionParameters().setCurativeMinObjImprovement(jsonParser.getValueAsDouble());
                     break;
-                case OPTIMIZE_CURATIVE_IF_PREVENTIVE_UNSECURE:
+                case ENFORCE_CURATIVE_SECURITY:
                     jsonParser.nextToken();
-                    raoParameters.getObjectiveFunctionParameters().setOptimizeCurativeIfPreventiveUnsecure(jsonParser.getBooleanValue());
+                    raoParameters.getObjectiveFunctionParameters().setEnforceCurativeSecurity(jsonParser.getBooleanValue());
                     break;
                 default:
                     throw new OpenRaoException(String.format("Cannot deserialize objective function parameters: unexpected field in %s (%s)", OBJECTIVE_FUNCTION, jsonParser.getCurrentName()));
@@ -76,12 +72,4 @@ private static ObjectiveFunctionParameters.PreventiveStopCriterion stringToPreve
         }
     }
 
-    private static ObjectiveFunctionParameters.CurativeStopCriterion stringToCurativeStopCriterion(String string) {
-        try {
-            return ObjectiveFunctionParameters.CurativeStopCriterion.valueOf(string);
-        } catch (IllegalArgumentException e) {
-            throw new OpenRaoException(String.format("Unknown curative stop criterion: %s", string));
-        }
-    }
-
 }

ra-optimisation/rao-api/src/main/java/com/powsybl/openrao/raoapi/parameters/ObjectiveFunctionParameters.java

@@ -24,14 +24,12 @@ public class ObjectiveFunctionParameters {
     private static final ObjectiveFunctionType DEFAULT_OBJECTIVE_FUNCTION = ObjectiveFunctionType.MAX_MIN_MARGIN_IN_MEGAWATT;
     private static final double DEFAULT_CURATIVE_MIN_OBJ_IMPROVEMENT = 0;
     private static final PreventiveStopCriterion DEFAULT_PREVENTIVE_STOP_CRITERION = PreventiveStopCriterion.SECURE;
-    private static final CurativeStopCriterion DEFAULT_CURATIVE_STOP_CRITERION = CurativeStopCriterion.MIN_OBJECTIVE;
-    private static final boolean DEFAULT_OPTIMIZE_CURATIVE_IF_PREVENTIVE_UNSECURE = false;
+    private static final boolean DEFAULT_ENFORCE_CURATIVE_SECURITY = false;
     // Attributes
     private ObjectiveFunctionType type = DEFAULT_OBJECTIVE_FUNCTION;
     private double curativeMinObjImprovement = DEFAULT_CURATIVE_MIN_OBJ_IMPROVEMENT;
     private PreventiveStopCriterion preventiveStopCriterion = DEFAULT_PREVENTIVE_STOP_CRITERION;
-    private CurativeStopCriterion curativeStopCriterion = DEFAULT_CURATIVE_STOP_CRITERION;
-    private boolean optimizeCurativeIfPreventiveUnsecure = DEFAULT_OPTIMIZE_CURATIVE_IF_PREVENTIVE_UNSECURE;
+    private boolean enforceCurativeSecurity = DEFAULT_ENFORCE_CURATIVE_SECURITY;
 
     // Enum
     public enum ObjectiveFunctionType {
@@ -60,13 +58,6 @@ public enum PreventiveStopCriterion {
         SECURE
     }
 
-    public enum CurativeStopCriterion {
-        MIN_OBJECTIVE, // only stop after minimizing objective
-        SECURE, //stop when objective is strictly negative
-        PREVENTIVE_OBJECTIVE, // stop when preventive objective is reached, or bested by curativeRaoMinObjImprovement
-        PREVENTIVE_OBJECTIVE_AND_SECURE // stop when preventive objective is reached or bested by curativeRaoMinObjImprovement, and the situation is secure
-    }
-
     // Getters and setters
     public ObjectiveFunctionType getType() {
         return type;
@@ -88,20 +79,12 @@ public PreventiveStopCriterion getPreventiveStopCriterion() {
         return preventiveStopCriterion;
     }
 
-    public CurativeStopCriterion getCurativeStopCriterion() {
-        return curativeStopCriterion;
-    }
-
-    public void setCurativeStopCriterion(CurativeStopCriterion curativeStopCriterion) {
-        this.curativeStopCriterion = curativeStopCriterion;
-    }
-
-    public boolean getOptimizeCurativeIfPreventiveUnsecure() {
-        return optimizeCurativeIfPreventiveUnsecure;
+    public boolean getEnforceCurativeSecurity() {
+        return enforceCurativeSecurity;
     }
 
-    public void setOptimizeCurativeIfPreventiveUnsecure(boolean optimizeCurativeIfPreventiveUnsecure) {
-        this.optimizeCurativeIfPreventiveUnsecure = optimizeCurativeIfPreventiveUnsecure;
+    public void setEnforceCurativeSecurity(boolean enforceCurativeSecurity) {
+        this.enforceCurativeSecurity = enforceCurativeSecurity;
     }
 
     public static ObjectiveFunctionParameters load(PlatformConfig platformConfig) {
@@ -114,9 +97,7 @@ public static ObjectiveFunctionParameters load(PlatformConfig platformConfig) {
                     parameters.setCurativeMinObjImprovement(config.getDoubleProperty(CURATIVE_MIN_OBJ_IMPROVEMENT, DEFAULT_CURATIVE_MIN_OBJ_IMPROVEMENT));
                     parameters.setPreventiveStopCriterion(config.getEnumProperty(PREVENTIVE_STOP_CRITERION, PreventiveStopCriterion.class,
                             DEFAULT_PREVENTIVE_STOP_CRITERION));
-                    parameters.setCurativeStopCriterion(config.getEnumProperty(CURATIVE_STOP_CRITERION, CurativeStopCriterion.class,
-                            DEFAULT_CURATIVE_STOP_CRITERION));
-                    parameters.setOptimizeCurativeIfPreventiveUnsecure(config.getBooleanProperty(OPTIMIZE_CURATIVE_IF_PREVENTIVE_UNSECURE, DEFAULT_OPTIMIZE_CURATIVE_IF_PREVENTIVE_UNSECURE));
+                    parameters.setEnforceCurativeSecurity(config.getBooleanProperty(ENFORCE_CURATIVE_SECURITY, DEFAULT_ENFORCE_CURATIVE_SECURITY));
                 });
         return parameters;
     }

ra-optimisation/rao-api/src/test/java/com/powsybl/openrao/raoapi/json/JsonRaoParametersTest.java

@@ -54,9 +54,8 @@ void roundTrip() throws IOException {
         // Objective Function parameters
         parameters.getObjectiveFunctionParameters().setType(ObjectiveFunctionParameters.ObjectiveFunctionType.MAX_MIN_MARGIN_IN_AMPERE);
         parameters.getObjectiveFunctionParameters().setPreventiveStopCriterion(ObjectiveFunctionParameters.PreventiveStopCriterion.MIN_OBJECTIVE);
-        parameters.getObjectiveFunctionParameters().setCurativeStopCriterion(ObjectiveFunctionParameters.CurativeStopCriterion.PREVENTIVE_OBJECTIVE_AND_SECURE);
         parameters.getObjectiveFunctionParameters().setCurativeMinObjImprovement(983);
-        parameters.getObjectiveFunctionParameters().setOptimizeCurativeIfPreventiveUnsecure(true);
+        parameters.getObjectiveFunctionParameters().setEnforceCurativeSecurity(true);
         // RangeActionsOptimization parameters
         parameters.getRangeActionsOptimizationParameters().setMaxMipIterations(30);
         parameters.getRangeActionsOptimizationParameters().setPstPenaltyCost(10);
@@ -176,7 +175,7 @@ void testFailOnOldVersion() {
     }
 
     @ParameterizedTest
-    @ValueSource(strings = {"LoopFlowError", "PrevStopCriterionError", "CurStopCriterionError", "WrongField"})
+    @ValueSource(strings = {"LoopFlowError", "PrevStopCriterionError", "WrongField"})
     void importNokTest(String source) {
         InputStream inputStream = getClass().getResourceAsStream("/RaoParametersWith" + source + "_v2.json");
         assertThrows(OpenRaoException.class, () -> JsonRaoParameters.read(inputStream));