route/virtual cluster priorities

docs/configuration/cluster_manager/cluster_circuit_breakers.rst

@@ -3,7 +3,8 @@
 Circuit breakers
 ================
 
-Circuit breaking :ref:`architecture overview <arch_overview_circuit_break>`.
+* Circuit breaking :ref:`architecture overview <arch_overview_circuit_break>`.
+* Priority routing :ref:`architecture overview <arch_overview_http_routing_priority>`.
 
 Circuit breaking settings can be specified individually for each defined priority. How the
 different priorities are used are documented in the sections of the configuration guide that use

docs/configuration/http_conn_man/route_config/route.rst

@@ -22,7 +22,8 @@ next (e.g., redirect, forward, rewrite, etc.).
     "runtime": "{...}",
     "retry_policy": "{...}",
     "rate_limit": "{...}",
-    "shadow": "{...}"
+    "shadow": "{...}",
+    "priority": "..."
   }
 
 prefix
@@ -89,6 +90,10 @@ content_type
 :ref:`shadow <config_http_conn_man_route_table_route_shadow>`
   *(optional, object)* Indicates that the route has a shadow policy.
 
+priority
+  *(optional, string)* Optionally specifies the :ref:`routing priority
+  <arch_overview_http_routing_priority>`.
+
 .. _config_http_conn_man_route_table_route_runtime:
 
 Runtime

docs/configuration/http_conn_man/route_config/vcluster.rst

@@ -22,7 +22,8 @@ such that they include network level failures.
   {
     "pattern": "...",
     "name": "...",
-    "method": "..."
+    "method": "...",
+    "priority": "..."
   }
 
 pattern
@@ -38,4 +39,8 @@ method
   *(optional, string)* Optionally specifies the HTTP method to match on. For example *GET*, *PUT*,
   etc.
 
+priority
+  *(optional, string)* Optionally specifies the virtual cluster :ref:`routing priority
+  <arch_overview_http_routing_priority>`.
+
 Documentation for :ref:`virtual cluser statistics <config_http_filters_router_stats>`.

docs/configuration/http_filters/router_filter.rst

@@ -87,7 +87,7 @@ retriable-4xx
   Envoy will attempt a retry if the upstream server responds with a retriable 4xx response code.
   Currently, the only response code in this category is 409.
 
-  * NOTE: Be careful turning on this retry type. There are certain cases where a 409 can indicate
+  * **NOTE:** Be careful turning on this retry type. There are certain cases where a 409 can indicate
     that an optimistic locking revision needs to be updated. Thus, the caller should not retry and
     needs to read then attempt another write. If a retry happens in this type of case it will always
     fail with another 409.

docs/intro/arch_overview/http_routing.rst

@@ -28,6 +28,7 @@ request. The router filter supports the following features:
 * Virtual cluster specifications. A virtual cluster is specified at the virtual host level and is
   used by Envoy to generate additional statistics on top of the standard cluster level ones. Virtual
   clusters can use regex matching.
+* :ref:`Priority <arch_overview_http_routing_priority>` based routing.
 
 Route table
 -----------
@@ -56,3 +57,22 @@ headers <config_http_filters_router_headers>`. The following configurations are
 * **Retry conditions**: Envoy can retry on different types of conditions depending on application
   requirements. For example, network failure, all 5xx response codes, idempotent 4xx response codes,
   etc.
+
+.. _arch_overview_http_routing_priority:
+
+Priority routing
+----------------
+
+Envoy supports priority routing both at the :ref:`route <config_http_conn_man_route_table_route>`
+and the :ref:`virtual cluster <config_http_conn_man_route_table_vcluster>` level. The current
+priority implementation uses different :ref:`connection pool <arch_overview_conn_pool>` and
+:ref:`circuit breaking <config_cluster_manager_cluster_circuit_breakers>` settings for each priority
+level. This means that even for HTTP/2 requests, two physical connections will be used to an
+upstream host. In the future Envoy will likely support true HTTP/2 priority over a single
+connection.
+
+Note that if a route matches a virtual cluster, the virtual cluster priority is used. This feature
+is useful for splitting circuit breaking limits between different traffic priorities such that low
+priority traffic does not starve higher priority traffic.
+
+The currently supported priorities are *default* and *high*.

include/envoy/router/router.h

@@ -3,6 +3,7 @@
 #include "envoy/common/optional.h"
 #include "envoy/http/codec.h"
 #include "envoy/http/header_map.h"
+#include "envoy/upstream/resource_manager.h"
 
 namespace Router {
 
@@ -112,6 +113,25 @@ class ShadowPolicy {
   virtual const std::string& runtimeKey() const PURE;
 };
 
+/**
+ * Virtual cluster definition (allows splitting a virtual host into virtual clusters orthogonal to
+ * routes for stat tracking and priority purposes).
+ */
+class VirtualCluster {
+public:
+  virtual ~VirtualCluster() {}
+
+  /**
+   * @return the name of the virtual cluster.
+   */
+  virtual const std::string& name() const PURE;
+
+  /**
+   * @return the priority of the virtual cluster.
+   */
+  virtual Upstream::ResourcePriority priority() const PURE;
+};
+
 /**
  * An individual resolved route entry.
  */
@@ -132,6 +152,11 @@ class RouteEntry {
    */
   virtual void finalizeRequestHeaders(Http::HeaderMap& headers) const PURE;
 
+  /**
+   * @return the priority of the route.
+   */
+  virtual Upstream::ResourcePriority priority() const PURE;
+
   /**
    * @return const RateLimitPolicy& the rate limit policy for the route.
    */
@@ -157,9 +182,9 @@ class RouteEntry {
   /**
    * Determine whether a specific request path belongs to a virtual cluster for use in stats, etc.
    * @param headers supplies the request headers.
-   * @return the virtual cluster or empty string if there is no match.
+   * @return the virtual cluster or nullptr if there is no match.
    */
-  virtual const std::string& virtualClusterName(const Http::HeaderMap& headers) const PURE;
+  virtual const VirtualCluster* virtualCluster(const Http::HeaderMap& headers) const PURE;
 
   /**
    * @return const std::string& the virtual host that owns the route.

source/common/router/config_impl.cc

@@ -45,6 +45,17 @@ ShadowPolicyImpl::ShadowPolicyImpl(const Json::Object& config) {
   runtime_key_ = config.getObject("shadow").getString("runtime_key", "");
 }
 
+Upstream::ResourcePriority ConfigUtility::parsePriority(const Json::Object& config) {
+  std::string priority_string = config.getString("priority", "default");
+  if (priority_string == "default") {
+    return Upstream::ResourcePriority::Default;
+  } else if (priority_string == "high") {
+    return Upstream::ResourcePriority::High;
+  } else {
+    throw EnvoyException(fmt::format("invalid resource priority '{}'", priority_string));
+  }
+}
+
 RouteEntryImplBase::RouteEntryImplBase(const VirtualHost& vhost, const Json::Object& route,
                                        Runtime::Loader& loader)
     : case_sensitive_(route.getBoolean("case_sensitive", true)),
@@ -56,7 +67,7 @@ RouteEntryImplBase::RouteEntryImplBase(const VirtualHost& vhost, const Json::Obj
       host_redirect_(route.getString("host_redirect", "")),
       path_redirect_(route.getString("path_redirect", "")), retry_policy_(route),
       content_type_(route.getString("content_type", "")), rate_limit_policy_(route),
-      shadow_policy_(route) {
+      shadow_policy_(route), priority_(ConfigUtility::parsePriority(route)) {
 
   // Check to make sure that we are either a redirect route or we have a cluster.
   if (!(isRedirect() ^ !cluster_name_.empty())) {
@@ -220,18 +231,21 @@ VirtualHost::VirtualHost(const Json::Object& virtual_host, Runtime::Loader& runt
 
   if (virtual_host.hasObject("virtual_clusters")) {
     for (const Json::Object& virtual_cluster : virtual_host.getObjectArray("virtual_clusters")) {
-      Optional<std::string> method;
-      if (virtual_cluster.hasObject("method")) {
-        method = virtual_cluster.getString("method");
-      }
-
-      virtual_clusters_.push_back(
-          {std::regex{virtual_cluster.getString("pattern"), std::regex::optimize}, method,
-           virtual_cluster.getString("name")});
+      virtual_clusters_.push_back(VirtualClusterEntry(virtual_cluster));
     }
   }
 }
 
+VirtualHost::VirtualClusterEntry::VirtualClusterEntry(const Json::Object& virtual_cluster) {
+  if (virtual_cluster.hasObject("method")) {
+    method_ = virtual_cluster.getString("method");
+  }
+
+  pattern_ = std::regex{virtual_cluster.getString("pattern"), std::regex::optimize};
+  name_ = virtual_cluster.getString("name");
+  priority_ = ConfigUtility::parsePriority(virtual_cluster);
+}
+
 RouteMatcher::RouteMatcher(const Json::Object& config, Runtime::Loader& runtime,
                            Upstream::ClusterManager& cm) {
   for (const Json::Object& virtual_host_config : config.getObjectArray("virtual_hosts")) {
@@ -315,25 +329,25 @@ const RouteEntry* RouteMatcher::routeForRequest(const Http::HeaderMap& headers,
   }
 }
 
-const std::string VirtualHost::VIRTUAL_CLUSTER_CATCH_ALL_NAME = "other";
+const VirtualHost::CatchAllVirtualCluster VirtualHost::VIRTUAL_CLUSTER_CATCH_ALL;
 const SslRedirector VirtualHost::SSL_REDIRECTOR;
 
-const std::string& VirtualHost::virtualClusterFromEntries(const Http::HeaderMap& headers) const {
+const VirtualCluster* VirtualHost::virtualClusterFromEntries(const Http::HeaderMap& headers) const {
   for (const VirtualClusterEntry& entry : virtual_clusters_) {
     bool method_matches =
         !entry.method_.valid() || headers.get(Http::Headers::get().Method) == entry.method_.value();
 
     if (method_matches &&
         std::regex_match(headers.get(Http::Headers::get().Path), entry.pattern_)) {
-      return entry.name_;
+      return &entry;
     }
   }
 
   if (virtual_clusters_.size() > 0) {
-    return VIRTUAL_CLUSTER_CATCH_ALL_NAME;
+    return &VIRTUAL_CLUSTER_CATCH_ALL;
   }
 
-  return EMPTY_STRING;
+  return nullptr;
 }
 
 ConfigImpl::ConfigImpl(const Json::Object& config, Runtime::Loader& runtime,

source/common/router/config_impl.h

@@ -34,10 +34,21 @@ typedef std::shared_ptr<RouteEntryImplBase> RouteEntryImplBasePtr;
  */
 class SslRedirector : public RedirectEntry {
 public:
-  // RedirectEntry
+  // Router::RedirectEntry
   std::string newPath(const Http::HeaderMap& headers) const override;
 };
 
+/**
+ * Utility routines for loading route configuration.
+ */
+class ConfigUtility {
+public:
+  /**
+   * @return the resource priority parsed from JSON.
+   */
+  static Upstream::ResourcePriority parsePriority(const Json::Object& config);
+};
+
 /**
  * Holds all routing configuration for an entire virtual host.
  */
@@ -51,18 +62,35 @@ class VirtualHost {
                                            uint64_t random_value) const;
   const RouteEntryImplBase* routeFromEntries(const Http::HeaderMap& headers, bool redirect,
                                              uint64_t random_value) const;
-  const std::string& virtualClusterFromEntries(const Http::HeaderMap& headers) const;
+  const VirtualCluster* virtualClusterFromEntries(const Http::HeaderMap& headers) const;
 
 private:
   enum class SslRequirements { NONE, EXTERNAL_ONLY, ALL };
 
-  struct VirtualClusterEntry {
+  struct VirtualClusterEntry : public VirtualCluster {
+    VirtualClusterEntry(const Json::Object& virtual_cluster);
+
+    // Router::VirtualCluster
+    const std::string& name() const override { return name_; }
+    Upstream::ResourcePriority priority() const override { return priority_; }
+
     std::regex pattern_;
     Optional<std::string> method_;
     std::string name_;
+    Upstream::ResourcePriority priority_;
+  };
+
+  struct CatchAllVirtualCluster : public VirtualCluster {
+    // Router::VirtualCluster
+    const std::string& name() const override { return name_; }
+    Upstream::ResourcePriority priority() const override {
+      return Upstream::ResourcePriority::Default;
+    }
+
+    std::string name_{"other"};
   };
 
-  static const std::string VIRTUAL_CLUSTER_CATCH_ALL_NAME;
+  static const CatchAllVirtualCluster VIRTUAL_CLUSTER_CATCH_ALL;
   static const SslRedirector SSL_REDIRECTOR;
 
   const std::string name_;
@@ -131,10 +159,11 @@ class RouteEntryImplBase : public RouteEntry, public Matchable, public RedirectE
   // Router::RouteEntry
   const std::string& clusterName() const override;
   void finalizeRequestHeaders(Http::HeaderMap& headers) const override;
+  Upstream::ResourcePriority priority() const override { return priority_; }
   const RateLimitPolicy& rateLimitPolicy() const override { return rate_limit_policy_; }
   const RetryPolicy& retryPolicy() const override { return retry_policy_; }
   const ShadowPolicy& shadowPolicy() const override { return shadow_policy_; }
-  const std::string& virtualClusterName(const Http::HeaderMap& headers) const override {
+  const VirtualCluster* virtualCluster(const Http::HeaderMap& headers) const override {
     return vhost_.virtualClusterFromEntries(headers);
   }
   const std::string& virtualHostName() const override { return vhost_.name(); }
@@ -175,6 +204,7 @@ class RouteEntryImplBase : public RouteEntry, public Matchable, public RedirectE
   const std::string content_type_;
   const RateLimitPolicyImpl rate_limit_policy_;
   const ShadowPolicyImpl shadow_policy_;
+  const Upstream::ResourcePriority priority_;
 };
 
 /**