component_data_objects(active=False) does not yield correct objects

[qtothec]

Test case

Discovered by @sjkale. Simple GDP test example. Trying to find inactive disjunctions using component_data_objects not working. Tested on fd2bd02.

from __future__ import division

from pyomo.environ import *
from pyomo.gdp import *

model = ConcreteModel()
model.lt = Var(initialize=0)
model.x1 = Var(initialize=0, bounds=(0, 15))
model.x2 = Var(initialize=0, bounds=(0, 15))
model.x3 = Var(initialize=0, bounds=(0, 15))
model.y1 = Var(initialize=6, bounds=(6, 10))
model.y2 = Var(initialize=7, bounds=(7, 10))
model.y3 = Var(initialize=3, bounds=(3, 10))

model.c1 = Constraint(expr=model.lt >= model.x1 + 6)
model.c2 = Constraint(expr=model.lt >= model.x2 + 5)
model.c3 = Constraint(expr=model.lt >= model.x3 + 4)


def disjunct11(disj):
    disj.indicator_var.value = 1
    disj.c = Constraint(expr=model.x1 + 6 <= model.x2)


def disjunct12(disj):
    disj.indicator_var.value = 0
    disj.c = Constraint(expr=model.x2 + 5 <= model.x1)


def disjunct13(disj):
    disj.indicator_var.value = 0
    disj.c = Constraint(expr=model.y1 - 6 >= model.y2)


def disjunct14(disj):
    disj.indicator_var.value = 0
    disj.c = Constraint(expr=model.y2 - 7 >= model.y1)


def disjunct21(disj):
    disj.indicator_var.value = 1
    disj.c = Constraint(expr=model.x1 + 6 <= model.x3)


def disjunct22(disj):
    disj.indicator_var.value = 0
    disj.c = Constraint(expr=model.x3 + 4 <= model.x1)


def disjunct23(disj):
    disj.indicator_var.value = 0
    disj.c = Constraint(expr=model.y1 - 6 >= model.y3)


def disjunct24(disj):
    disj.indicator_var.value = 0
    disj.c = Constraint(expr=model.y3 - 3 >= model.y1)


model.d11 = Disjunct(rule=disjunct11)
model.d12 = Disjunct(rule=disjunct12)
model.d13 = Disjunct(rule=disjunct13)
model.d14 = Disjunct(rule=disjunct14)
model.d21 = Disjunct(rule=disjunct21)
model.d22 = Disjunct(rule=disjunct22)
model.d23 = Disjunct(rule=disjunct23)
model.d24 = Disjunct(rule=disjunct24)


def disjunction1(m):
    return [m.d11, m.d12, m.d13, m.d14]


def disjunction2(m):
    return [m.d21, m.d22, m.d23, m.d24]


model.d1 = Disjunction(rule=disjunction1, xor=True)
model.d2 = Disjunction(rule=disjunction2, xor=True)

model.obj = Objective(expr=model.lt, sense=minimize)

active_disj = list(
    model.component_data_objects(ctype=Disjunction, active=True))
for disj in active_disj:
    disj.deactivate()
print(active_disj)

inactive_disj = list(
    model.component_data_objects(ctype=Disjunction, active=False))
print(inactive_disj)

Expected output

[<pyomo.gdp.disjunct.SimpleDisjunction object at 0x7f04c6d18890>, <pyomo.gdp.disjunct.SimpleDisjunction object at0x7f04a52c9ef0>]
[<pyomo.gdp.disjunct.SimpleDisjunction object at 0x7f04c6d18890>, <pyomo.gdp.disjunct.SimpleDisjunction object at0x7f04a52c9ef0>]

Actual output

[<pyomo.gdp.disjunct.SimpleDisjunction object at 0x7f04c6d18890>, <pyomo.gdp.disjunct.SimpleDisjunction object at0x7f04a52c9ef0>]
[]

[blnicho]

This isn't isolated to GDP, I can recreate this using Constraints.

[ghackebeil]

I tried to raise this issue about 3 years ago on trac (https://software.sandia.gov/trac/pyomo/ticket/4595). It was largely ignored.

In Kernel, the component iterators only accept active=True/None to avoid this issue.

[blnicho]

Looks like this is being caused by block_data_objects using the same value for active that is passed into component_data_objects which means that you can only ever get active components on active Blocks or inactive components on inactive Blocks.

[qtothec]

I'm ok with requiring active=True/None if we deprecate active=False so that people stop trying to use it.

[jsiirola]

@blnicho: correct. We have been burned by using the same flag for the block iteration and the component identification in the past. The original motivation was searching for active components, where an active constraint within an inactive block should not be returned. But that has caused initially surprising side effects.

@qtothec: the restriction to active=True/None does not actually solve the problem. You can get equally (at least initially) unintuitive results like:

m = ConcreteModel()
m.x = Var()
m.b = Block()
m.b.d = Block()
m.b.d.c = Constraint(expr=m.x>=0)
m.b.deactivate()
list(m.b.component_data_objects(Constraint, active=True)) # == []

The best solution is probably to change how we handle descend_into ... I am just not certain as to the best API.

[ghackebeil]

@jsiirola: If that behavior is unintuitive, it is a reflection of poor documentation for that method. Either active is a bad name for that keyword, or documentation should be more clear that active means something more general in that context (i.e., reachable from the root calling block through a path of active parents). Of course, that definition might not even make sense if you change the tree traversal order for the AML Block.

I believe I've handled this correctly in Kernel for both pre-order and post-order traversal. The documentation there could be made more clear about it though.

[Robbybp]

I hit something similar when passing the active flag to component_objects to generate Vars. There appears to be inconsistency in how the flag is applied to vars. I would expect vars to be generated with active=True if they live on a BlockData with active=True, and similarly for active=False.

In the following script, it appears vars do not get generated when active=False even if they live under a block with active=False.

import pyomo.environ as pyo
m = pyo.ConcreteModel()
m.b = pyo.Block()
m.v = pyo.Var()

print("active=False")
m.deactivate()
m.b.deactivate()
for comp in m.component_objects(pyo.Var, active=False):
    print(comp.name)

print("active=True")
m.activate()
m.b.activate()
for comp in m.component_objects(pyo.Var, active=True):
    print(comp.name)

The output is:

active=False
active=True
b.v

where I would have expected:

active=False
b.v
active=True
b.v

Edited to add m.activate() and m.deactivate(), which are necessary for this behavior to be unexpected.

Edit 2: This behavior appears to be because Var implements an active flag with a hard-coded value of True.