Global unit not starting with multiple MachineMetadata

[alecmev]

I have a cluster looking like this:

IP       METADATA
1.2.3.1  rank=pet
1.2.3.2  rank=pet
1.2.3.3  rank=cattle,role=api
1.2.3.4  rank=pet

And I have a service which looks like this:

...

[X-Fleet]
MachineMetadata=role=api
MachineMetadata=rank=pet
Global=true

It works fine, if I remove either one of the MachineMetadata and start it with fleetctl, but if I leave both in, then... nothing happens. The unit does get submitted, but it doesn't start on any machine, fleetctl list-units is empty and journalctl -u my.service is empty too.

Is this a bug or just not supported?

[kayrus]

If you wish to schedule your unit on all your machines described above, just use this metadata:

[X-Fleet]
MachineMetadata=rank=pet
MachineMetadata=rank=cattle
Global=true

It will create following condition:

rank==pet OR rank==cattle

As for your current metadata:

[X-Fleet]
MachineMetadata=role=api
MachineMetadata=rank=pet
Global=true

it will create following condition:

role==api AND rank==pet

If you create this unit:

[X-Fleet]
MachineMetadata=role=api
MachineMetadata=rank=pet
MachineMetadata=rank=cattle
Global=true

it will produce following condition:

role==api AND (rank==pet OR rank==cattle)

[alecmev]

Thanks for the clarification, you should add this to the docs.

Why does it automatically AND different types of metadata though? That's kind of counter-intuitive, considering that there's special syntax for AND-ing:

[X-Fleet]
MachineMetadata="role=api" "rank=pet"
Global=true

And in my case, there's a problem with this solution:

[X-Fleet]
MachineMetadata=rank=pet
MachineMetadata=rank=cattle
Global=true

I have other machines with rank=cattle which aren't role=api (e.g. they're role=db instead).

What I'm doing right now is just assigning api role to pet machines explicitly, which works fine. However, if I were to decide that I don't want to run my API on pet instances anymore, I'd have to manually update all three pet machines (which is quite a challenge on EC2 + ASG, without messing up the cluster) or specify in the unit file that I want to run API only on cattle machines. The latter is pretty easy, but it would be confusing to the future maintainer, since he'd see role=api on the pet instances, even though they aren't supposed to be running API.

Hope this makes sense.

[kayrus]

@jeremejevs OR is being used only when you have multiple values for one key. In other cases AND logic works. If we implement OR top logic instead of AND it will cause backward-compatibility.

Probably we can add new feature which will make your life easier and preserve backward compatibility: #1368

As for the documentation, we will improve it soon.

[alecmev]

Yeah, I get that changing the behavior isn't an option, was just wondering about the rationale.

Maybe some simple expression format could be developed for MachineMetadata (since it's not standard INI anyway), like "rank=pet" AND ("role=db" OR "role=db-backup") AND "foo!=bar"?

[kayrus]

@jeremejevs it is better to post your thoughts in this ticket #1368

[alecmev]

@kayrus Well, it's not just about "inverted / negative" conditions. I'd rather open a new ticket instead, and reference this and #1368 in it. Is that okay?

[kayrus]

@jeremejevs NP