From b5cf8de2ebb9013994195dfcd6a4b2cd699ed902 Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Thu, 19 Jan 2017 08:52:53 -0800 Subject: [PATCH 01/12] Initial draft of design for service configs in DNS. --- A2.md | 110 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 110 insertions(+) create mode 100644 A2.md diff --git a/A2.md b/A2.md new file mode 100644 index 000000000..b7f1b4ea2 --- /dev/null +++ b/A2.md @@ -0,0 +1,110 @@ +Service Config via DNS +---------------------- +* Author(s): Mark D. Roth (roth@google.com) +* Approver: a11r +* Status: Draft +* Implemented in: N/A +* Last updated: 2017-01-19 +* Discussion at: (filled after thread exists) + +## Abstract + +This document proposes a mechanism for encoding gRPC service config data +in DNS for use in the open-source world. + +## Background + +The [service +config](https://github.com/grpc/grpc/blob/master/doc/service_config.md) +mechanism was originally designed for use inside of Google. However, +all but one part of the original design will work fine in the open-source +world. That one part is the specification of how the service config +data will be encoded in DNS. + +### Related Proposals: + +N/A + +## Proposal + +There are two parts to this proposal. The first part is to add some +JSON wrapping for controlling how service config changes are canary +tested. The second part describes how the service config is encoded in +DNS. + +### Canarying Changes + +When deploying a change to a service config, it is useful to be able to +canary test changes to avoid wide-spread breakage by slowly increasing the +number of clients that see the new version. To that end, multiple +service configs choices can be listed, in order, along with criteria that +determine which choice will be selected by a given client: + +``` +# A list of one or more service config choices. +# The first matching entry wins. +[ + { + # Criteria used to select this choice. + # Client language (e.g., 'c++', 'java', 'go', 'python', etc) + # Percentage: number from 0 to 100 indicating the percentage of + # clients that should use this choice. + 'clientLanguage': string, + 'percentage': number, + + # The service config data object for clients that match the above + # criteria. (The format for this object is defined in + # https://github.com/grpc/grpc/blob/master/doc/service_config.md.) + 'serviceConfig': object, + } +] +``` + +### Encoding in DNS TXT Records + +In DNS, the service config data (in the form documented in the previous +section) will be encoded in a TXT record with the attribute name `grpc`. +The attribute value will be a JSON list containing service config choices. +For example, here is an example TXT record: + +``` +myserver 3600 TXT "grpc=[{'serviceConfig':{'loadBalancingPolicy':'round_robin','methodConfig':[{'name':[{'service':'MyService','method':'Foo'}],'waitForReady':true}]}}]" +``` + +## Rationale + +The service config is designed to be returned as part of name +resolution, so encoding it in DNS makes the most sense. Sites that use +a naming system other than DNS can, of course, implement their own +resolvers with their own mechanism for encoding service config data. + +When encoding the service config in DNS, TXT records are the "obvious" +choice, since the service config is effectively additional metadata +associated with the DNS name. + +## Implementation + +The implementation will be done in C-core first. Once the new c-ares +DNS resolver code (https://github.com/grpc/grpc/pull/7771) has been +merged, we will extend it to query for the TXT records and return the +resulting service config JSON data. + +## Open issues (if applicable) + +DNS TXT records do have some limitations that need to be taken into +account here. + +TXT records are limited to 255 bytes per string. There can be multiple +strings, which will be concatenated together. However, there are a few +considerations with regard to length limitations: + +- If a DNS response exceeds 512 bytes, it will fall back from UDP to + TCP, which adds overhead. +- The total DNS response cannot exceed 65536 bytes. +- It is not clear whether individual DNS implementations will allow + anywhere close to 65536 bytes, even though the spec says that they + should. + +Feedback is requested on whether these considerations will be a +significant drawback for this design (in which case the design will +probably have to be changed). From 254c89d6936b4e0d6e010bac6b0ed5fd4d11ad91 Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Thu, 19 Jan 2017 08:58:29 -0800 Subject: [PATCH 02/12] Add link to discussion thread. --- A2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/A2.md b/A2.md index b7f1b4ea2..080065636 100644 --- a/A2.md +++ b/A2.md @@ -5,7 +5,7 @@ Service Config via DNS * Status: Draft * Implemented in: N/A * Last updated: 2017-01-19 -* Discussion at: (filled after thread exists) +* Discussion at: https://groups.google.com/d/topic/grpc-io/DkweyrWEXxU/discussion ## Abstract From d0d0424db680a95bf90ffcce0064aa0d95c8547e Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Mon, 23 Jan 2017 07:27:03 -0800 Subject: [PATCH 03/12] Apply reviewer suggestions. --- A2.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/A2.md b/A2.md index 080065636..9cb4324f4 100644 --- a/A2.md +++ b/A2.md @@ -47,7 +47,7 @@ determine which choice will be selected by a given client: { # Criteria used to select this choice. # Client language (e.g., 'c++', 'java', 'go', 'python', etc) - # Percentage: number from 0 to 100 indicating the percentage of + # Percentage: integer from 0 to 100 indicating the percentage of # clients that should use this choice. 'clientLanguage': string, 'percentage': number, @@ -55,7 +55,7 @@ determine which choice will be selected by a given client: # The service config data object for clients that match the above # criteria. (The format for this object is defined in # https://github.com/grpc/grpc/blob/master/doc/service_config.md.) - 'serviceConfig': object, + 'serviceConfig': object } ] ``` From 37461fa0a38c1670c41a767cafd2a8fe2756853a Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Wed, 25 Jan 2017 07:38:55 -0800 Subject: [PATCH 04/12] Add note about the contents being restricted to ASCII. --- A2.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/A2.md b/A2.md index 9cb4324f4..0a9ee7ed5 100644 --- a/A2.md +++ b/A2.md @@ -71,6 +71,10 @@ For example, here is an example TXT record: myserver 3600 TXT "grpc=[{'serviceConfig':{'loadBalancingPolicy':'round_robin','methodConfig':[{'name':[{'service':'MyService','method':'Foo'}],'waitForReady':true}]}}]" ``` +Note that because TXT records must be ASCII, this also imposes the +restruction that the contents of the service config are also ASCII +(e.g., service and method names, load balancing policy names, etc). + ## Rationale The service config is designed to be returned as part of name From 74e70b071024280d14ac9d48b6f0720b24563a5b Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Thu, 26 Jan 2017 09:32:28 -0800 Subject: [PATCH 05/12] Change TXT attribute name from "grpc" to "grpc_config". --- A2.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/A2.md b/A2.md index 0a9ee7ed5..114c5bf97 100644 --- a/A2.md +++ b/A2.md @@ -63,12 +63,12 @@ determine which choice will be selected by a given client: ### Encoding in DNS TXT Records In DNS, the service config data (in the form documented in the previous -section) will be encoded in a TXT record with the attribute name `grpc`. -The attribute value will be a JSON list containing service config choices. -For example, here is an example TXT record: +section) will be encoded in a TXT record with the attribute name +`grpc_config`. The attribute value will be a JSON list containing +service config choices. For example, here is an example TXT record: ``` -myserver 3600 TXT "grpc=[{'serviceConfig':{'loadBalancingPolicy':'round_robin','methodConfig':[{'name':[{'service':'MyService','method':'Foo'}],'waitForReady':true}]}}]" +myserver 3600 TXT "grpc_config=[{'serviceConfig':{'loadBalancingPolicy':'round_robin','methodConfig':[{'name':[{'service':'MyService','method':'Foo'}],'waitForReady':true}]}}]" ``` Note that because TXT records must be ASCII, this also imposes the From ae2dfb052b338268d6b7b864e97956501da1e333 Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Mon, 30 Jan 2017 10:14:20 -0800 Subject: [PATCH 06/12] Add clientHostname selector field. Describe matching semantics. --- A2.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/A2.md b/A2.md index 114c5bf97..501243c36 100644 --- a/A2.md +++ b/A2.md @@ -46,11 +46,17 @@ determine which choice will be selected by a given client: [ { # Criteria used to select this choice. - # Client language (e.g., 'c++', 'java', 'go', 'python', etc) + # If a field is absent or empty, it matches all clients. + # All fields must match a client for this choice to be selected. + # + # Client language(s): a list of strings (e.g., 'c++', 'java', 'go', + # 'python', etc). + 'clientLanguage': [string], # Percentage: integer from 0 to 100 indicating the percentage of # clients that should use this choice. - 'clientLanguage': string, 'percentage': number, + # Client hostname(s): a list of strings. + 'clientHostname': [string], # The service config data object for clients that match the above # criteria. (The format for this object is defined in From adafae76c3845151c38c8ffee2aa172d8ce4ab99 Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Mon, 30 Jan 2017 10:53:39 -0800 Subject: [PATCH 07/12] Address reviewer feedback. --- A2.md | 51 ++++++++++++++++++++++++++++----------------------- 1 file changed, 28 insertions(+), 23 deletions(-) diff --git a/A2.md b/A2.md index 501243c36..13c1ef169 100644 --- a/A2.md +++ b/A2.md @@ -40,27 +40,27 @@ number of clients that see the new version. To that end, multiple service configs choices can be listed, in order, along with criteria that determine which choice will be selected by a given client: -``` -# A list of one or more service config choices. -# The first matching entry wins. +```json +// A list of one or more service config choices. +// The first matching entry wins. [ { - # Criteria used to select this choice. - # If a field is absent or empty, it matches all clients. - # All fields must match a client for this choice to be selected. - # - # Client language(s): a list of strings (e.g., 'c++', 'java', 'go', - # 'python', etc). + // Criteria used to select this choice. + // If a field is absent or empty, it matches all clients. + // All fields must match a client for this choice to be selected. + // + // Client language(s): a list of strings (e.g., 'c++', 'java', 'go', + // 'python', etc). 'clientLanguage': [string], - # Percentage: integer from 0 to 100 indicating the percentage of - # clients that should use this choice. + // Percentage: integer from 0 to 100 indicating the percentage of + // clients that should use this choice. 'percentage': number, - # Client hostname(s): a list of strings. + // Client hostname(s): a list of strings. 'clientHostname': [string], - # The service config data object for clients that match the above - # criteria. (The format for this object is defined in - # https://github.com/grpc/grpc/blob/master/doc/service_config.md.) + // The service config data object for clients that match the above + // criteria. (The format for this object is defined in + // https://github.com/grpc/grpc/blob/master/doc/service_config.md.) 'serviceConfig': object } ] @@ -69,14 +69,23 @@ determine which choice will be selected by a given client: ### Encoding in DNS TXT Records In DNS, the service config data (in the form documented in the previous -section) will be encoded in a TXT record with the attribute name -`grpc_config`. The attribute value will be a JSON list containing -service config choices. For example, here is an example TXT record: +section) will be encoded in a TXT record via the mechanism described in +[RFC-1464](https://tools.ietf.org/html/rfc1464) using the attribute name +`grpc_config`. The attribute value will be a JSON list containing service +config choices. For example, here is an example TXT record: ``` myserver 3600 TXT "grpc_config=[{'serviceConfig':{'loadBalancingPolicy':'round_robin','methodConfig':[{'name':[{'service':'MyService','method':'Foo'}],'waitForReady':true}]}}]" ``` +Note that TXT records are limited to 255 bytes per string, as per +[RFC-1035 section 3.3](https://tools.ietf.org/html/rfc1035#section-3.3). +However, there can be multiple strings, which will be +concatenated together, as described in [RFC-4408 section +3.1.3](https://tools.ietf.org/html/rfc4408#section-3.1.3). The total +across all strings is limited to 65536 bytes. (See the "Open Issues" +section below for more discussion.) + Note that because TXT records must be ASCII, this also imposes the restruction that the contents of the service config are also ASCII (e.g., service and method names, load balancing policy names, etc). @@ -102,11 +111,7 @@ resulting service config JSON data. ## Open issues (if applicable) DNS TXT records do have some limitations that need to be taken into -account here. - -TXT records are limited to 255 bytes per string. There can be multiple -strings, which will be concatenated together. However, there are a few -considerations with regard to length limitations: +account here. In particular: - If a DNS response exceeds 512 bytes, it will fall back from UDP to TCP, which adds overhead. From 1d394bf18c6b4f5b2038661480ac4fff05858463 Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Mon, 30 Jan 2017 10:56:03 -0800 Subject: [PATCH 08/12] Remove json-specific formatting, which didn't work. --- A2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/A2.md b/A2.md index 13c1ef169..3acfc7557 100644 --- a/A2.md +++ b/A2.md @@ -40,7 +40,7 @@ number of clients that see the new version. To that end, multiple service configs choices can be listed, in order, along with criteria that determine which choice will be selected by a given client: -```json +``` // A list of one or more service config choices. // The first matching entry wins. [ From d9783f98966d5fdafab82b7053ef618171110e28 Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Mon, 30 Jan 2017 13:19:48 -0800 Subject: [PATCH 09/12] Tweak language about max DNS response size. --- A2.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/A2.md b/A2.md index 3acfc7557..738f6d544 100644 --- a/A2.md +++ b/A2.md @@ -83,7 +83,7 @@ Note that TXT records are limited to 255 bytes per string, as per However, there can be multiple strings, which will be concatenated together, as described in [RFC-4408 section 3.1.3](https://tools.ietf.org/html/rfc4408#section-3.1.3). The total -across all strings is limited to 65536 bytes. (See the "Open Issues" +DNS response cannot exceed 65535 bytes. (See the "Open Issues" section below for more discussion.) Note that because TXT records must be ASCII, this also imposes the @@ -115,9 +115,9 @@ account here. In particular: - If a DNS response exceeds 512 bytes, it will fall back from UDP to TCP, which adds overhead. -- The total DNS response cannot exceed 65536 bytes. +- The total DNS response cannot exceed 65535 bytes. - It is not clear whether individual DNS implementations will allow - anywhere close to 65536 bytes, even though the spec says that they + anywhere close to 65535 bytes, even though the spec says that they should. Feedback is requested on whether these considerations will be a From f3fefbb1d704fbd791a688878a2152d0cdefffb5 Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Thu, 2 Feb 2017 08:43:15 -0800 Subject: [PATCH 10/12] Add note about initially not supporting this for Windows or Node. --- A2.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/A2.md b/A2.md index 738f6d544..07268c261 100644 --- a/A2.md +++ b/A2.md @@ -108,6 +108,10 @@ DNS resolver code (https://github.com/grpc/grpc/pull/7771) has been merged, we will extend it to query for the TXT records and return the resulting service config JSON data. +Note that, due to platform support issues, we will initially *not* +support the c-ares resolver under Windows or for Node. Alternatives +will need to be found for these environments. + ## Open issues (if applicable) DNS TXT records do have some limitations that need to be taken into From e49aaaf7094dc7f870efadee473fbd9adeb96bbc Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Tue, 14 Mar 2017 10:17:01 -0700 Subject: [PATCH 11/12] Renamed as per https://github.com/grpc/proposal/pull/17 --- A2.md => A2-service-configs-in-dns.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename A2.md => A2-service-configs-in-dns.md (100%) diff --git a/A2.md b/A2-service-configs-in-dns.md similarity index 100% rename from A2.md rename to A2-service-configs-in-dns.md From 9c2c18fb3679ca531c534472147153d98b9bae24 Mon Sep 17 00:00:00 2001 From: "Mark D. Roth" Date: Wed, 22 Mar 2017 08:10:18 -0700 Subject: [PATCH 12/12] Address reviewer comments. --- A2-service-configs-in-dns.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/A2-service-configs-in-dns.md b/A2-service-configs-in-dns.md index 07268c261..a82ddfbf7 100644 --- a/A2-service-configs-in-dns.md +++ b/A2-service-configs-in-dns.md @@ -16,10 +16,10 @@ in DNS for use in the open-source world. The [service config](https://github.com/grpc/grpc/blob/master/doc/service_config.md) -mechanism was originally designed for use inside of Google. However, -all but one part of the original design will work fine in the open-source -world. That one part is the specification of how the service config -data will be encoded in DNS. +mechanism was originally designed for use inside of Google. However, all +but one part of the original design works fine in the open-source world. +That one part is the specification of how the service config data will +be encoded in DNS. This proposal fills in this missing piece. ### Related Proposals: