istio shadowing

[ryandawsonuk]

fixes #741

Tried on this example:

{
  "kind": "SeldonDeployment",
  "apiVersion": "machinelearning.seldon.io/v1alpha2",
  "metadata": {
    "name": "loanclassifier"
  },
  "spec": {
    "name": "loanclassifier",
    "predictors": [
      {
        "name": "default",
        "graph": {
          "name": "loanclassifier-default",
          "implementation": "SKLEARN_SERVER",
          "modelUri": "gs://seldon-models/sklearn/income/model"
        },
        "replicas": 1
      },
      {
        "name": "shadow",
        "graph": {
          "name": "loanclassifier-shadow",
          "implementation": "SKLEARN_SERVER",
          "modelUri": "gs://seldon-models/sklearn/income/model"
        },
        "replicas": 1,
        "shadow": "true"
      }
    ]
  }
}

And traffic gets shadowed and grafana dashboards reflect it:

Need to check edge cases such as multiple predictors and explainers. Also need to create notebook.

[ukclivecox]

Can we fix these issues of formatting. Why did they happen?

[ryandawsonuk]

Here's the combination of default, canary and also shadow:

{
  "kind": "SeldonDeployment",
  "apiVersion": "machinelearning.seldon.io/v1alpha2",
  "metadata": {
    "name": "income-classifier",
    "namespace": "default",
    "creationTimestamp": null
  },
  "spec": {
    "name": "income-classifier",
    "predictors": [
      {
        "name": "default",
        "graph": {
          "name": "income-classifier-default",
          "type": "UNKNOWN_TYPE",
          "implementation": "SKLEARN_SERVER",
          "endpoint": {
            "service_host": "localhost",
            "service_port": 9000,
            "type": "REST"
          },
          "modelUri": "gs://seldon-models/sklearn/income/model"
        },
        "componentSpecs": [
          {
            "metadata": {
              "creationTimestamp": "2019-11-13T10:15:27Z"
            },
            "spec": {
              "containers": [
                {
                  "name": "income-classifier-default",
                  "image": "seldonio/sklearnserver_rest:0.2",
                  "resources": {},
                  "volumeMounts": [
                    {
                      "name": "podinfo",
                      "mountPath": "/etc/podinfo"
                    }
                  ]
                }
              ]
            }
          }
        ],
        "replicas": 1,
        "engineResources": {},
        "labels": {
          "version": "default"
        },
        "svcOrchSpec": {},
        "traffic": 90,
        "explainer": {
          "containerSpec": {
            "name": "",
            "resources": {}
          }
        }
      },
      {
        "name": "canary",
        "graph": {
          "name": "income-classifier-canary",
          "implementation": "SKLEARN_SERVER",
          "endpoint": {
            "type": "REST"
          },
          "modelUri": "gs://seldon-models/sklearn/income/model"
        },
        "componentSpecs": [
          {
            "metadata": {
              "creationTimestamp": "2019-11-13T10:16:24Z"
            },
            "spec": {
              "containers": [
                {
                  "name": "income-classifier-canary",
                  "resources": {}
                }
              ]
            }
          }
        ],
        "replicas": 1,
        "engineResources": {},
        "svcOrchSpec": {},
        "traffic": 10,
        "explainer": {
          "containerSpec": {
            "name": "",
            "resources": {}
          }
        }
      },
      {
        "name": "shadow",
        "graph": {
          "name": "income-classifier-shadow",
          "type": "UNKNOWN_TYPE",
          "implementation": "SKLEARN_SERVER",
          "endpoint": {
            "service_host": "localhost",
            "service_port": 9001,
            "type": "REST"
          },
          "modelUri": "gs://seldon-models/sklearn/income/model"
        },
        "componentSpecs": [
          {
            "metadata": {
              "creationTimestamp": "2019-11-08T15:19:58Z"
            },
            "spec": {
              "containers": [
                {
                  "name": "income-classifier-shadow",
                  "image": "seldonio/sklearnserver_rest:0.2",
                  "resources": {},
                  "volumeMounts": [
                    {
                      "name": "podinfo",
                      "mountPath": "/etc/podinfo"
                    }
                  ]
                }
              ]
            }
          }
        ],
        "replicas": 1,
        "engineResources": {},
        "labels": {
          "version": "shadow"
        },
        "svcOrchSpec": {},
        "shadow": "true",
        "explainer": {
          "containerSpec": {
            "name": "",
            "resources": {}
          }
        }
      }
    ],
    "annotations": {
      "seldon.io/engine-seldon-log-messages-externally": "true"
    }
  },
  "status": {}
}

Which also works

[ryandawsonuk]

Looks like tests pass - https://github.com/SeldonIO/seldon-core/blob/gh-pages/jenkins-x/logs/SeldonIO/seldon-core/PR-1102/10.log#L17384

[ukclivecox]

/approve

[seldondev]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: cliveseldon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [cliveseldon]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment