Assign pod to default mizar network only when Mizar VPC/subnet annotation is not present in pod

[Sindica]

This fix issue #1285

. This change keeps that original implementation that pod created w/o mizar vpc/subnet annotation to be attached to default tenant network.
. It also allows pod created with mizar annotation to be attached to the designated vpc/subnet.
. It is also possible to update mizar annotation so that the pod can be moved across different vpc/subnet - mizar is having issue to support this. see #1286

ubuntu@ip-172-30-0-14:~/go/src/k8s.io/arktos$ ./cluster/kubectl.sh get pods -AT -o wide
TENANT   NAMESPACE     NAME                               HASHKEY               READY   STATUS              RESTARTS   AGE   IP            NODE             NOMINATED NODE   READINESS GATES
aaa      default       netpod1-aaa                        3908766484272865849   1/1     Running             0          56m   21.0.21.2     ip-172-30-0-14   <none>           <none>
aaa      default       netpod2-aaa                        5963475252902892960   1/1     Running             0          56m   21.0.21.10    ip-172-30-0-14   <none>           <none>
aaa      kube-system   coredns-default-65df55d94b-bfzf5   5411032390339887240   0/1     CrashLoopBackOff    15         57m   21.0.21.5     ip-172-30-0-14   <none>           <none>
system   default       mizar-daemon-9rdss                 7593643411712881783   1/1     Running             0          60m   172.30.0.14   ip-172-30-0-14   <none>           <none>
system   default       mizar-operator-5c97f7478d-blqvv    7978921042419618136   1/1     Running             0          60m   172.30.0.14   ip-172-30-0-14   <none>           <none>
system   default       netpod1                            1142677440335999305   0/1     ContainerCreating   0          20m   <none>        ip-172-30-0-14   <none>           <none>
system   default       netpod2                            447511401861587042    0/1     ContainerCreating   0          20m   <none>        ip-172-30-0-14   <none>           <none>
system   default       ying-nginx-1                       8821160919168682061   1/1     Running             0          55m   22.0.1.5      ip-172-30-0-14   <none>           <none>
system   default       ying-nginx-2                       7457920908913110836   1/1     Running             0          55m   22.0.1.13     ip-172-30-0-14   <none>           <none>
system   kube-system   coredns-default-65df55d94b-xh9bh   6926818154246951590   0/1     CrashLoopBackOff    16         60m   21.0.21.14    ip-172-30-0-14   <none>           <none>
system   kube-system   kube-dns-554c5866fc-zt5tr          2096958600518597650   1/3     CrashLoopBackOff    44         60m   21.0.21.2     ip-172-30-0-14   <none>           <none>
system   kube-system   virtlet-wtcll                      2852824073873523462   3/3     Running             0          60m   172.30.0.14   ip-172-30-0-14   <none>           <none>

[q131172019]

/LGTM

[centaurus-cloud-bot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: q131172019
To complete the pull request process, please assign dingyin
You can assign the PR to them by writing /assign @dingyin in a comment when ready.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[centaurus-cloud-bot]

New changes are detected. LGTM label has been removed.