Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement predefined field constraints #246

Merged
merged 25 commits into from
Sep 17, 2024
Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
0d5bfe9
Implement shared field rules
jchadwick-buf Sep 3, 2024
8657cd2
Fix bad formatting
jchadwick-buf Sep 4, 2024
bf72afb
Add shared rule tests for each type
jchadwick-buf Sep 4, 2024
9030e64
Add tests for 'rule' constant
jchadwick-buf Sep 4, 2024
0236de1
Add tests using shared rules from proto3
jchadwick-buf Sep 4, 2024
d72b7d7
Add docs for shared field rules
jchadwick-buf Sep 4, 2024
4ef8796
Further documentation improvements
jchadwick-buf Sep 4, 2024
13aee7a
Merge branch 'main' of https://github.com/bufbuild/protovalidate into…
jchadwick-buf Sep 4, 2024
09dcec8
Fix signed/unsigned confusion in shared rules test
jchadwick-buf Sep 6, 2024
23b00c8
Merge branch 'main' of https://github.com/bufbuild/protovalidate into…
jchadwick-buf Sep 6, 2024
74e3f64
Documentation improvements
jchadwick-buf Sep 6, 2024
bb9336f
Add test cases covering mixed shared and custom constraints
jchadwick-buf Sep 6, 2024
006e5d5
Add test with standard, shared and custom constraints on field
jchadwick-buf Sep 6, 2024
ddde592
Merge branch 'main' of https://github.com/bufbuild/protovalidate into…
jchadwick-buf Sep 9, 2024
628a280
Add workaround for bufbuild/buf#3306
jchadwick-buf Sep 9, 2024
40d8d14
Fix bad formatting
jchadwick-buf Sep 9, 2024
1696bea
Merge everything into validate.proto + proto2 conversion
jchadwick-buf Sep 10, 2024
e22dab3
Re-add bufbuild/buf#3306 workaround for shared_rules_proto3.proto only
jchadwick-buf Sep 10, 2024
d2d22f3
generate
jchadwick-buf Sep 10, 2024
4fd7a36
shared_field: Fix type, better comment hopefully
jchadwick-buf Sep 10, 2024
95e8a98
Shared -> predefined, fix docs, gen, etc.
jchadwick-buf Sep 11, 2024
b1890fc
updates
jchadwick-buf Sep 16, 2024
eef5323
generate
jchadwick-buf Sep 16, 2024
c1862ea
oops, no need for a helper here
jchadwick-buf Sep 17, 2024
444deab
Merge branch 'main' of https://github.com/bufbuild/protovalidate into…
jchadwick-buf Sep 17, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,11 @@ library.
This guide will help you understand when and how to use these standard
constraints effectively.

- [Shared Constraints](shared-constraints.md): This section discusses how to
extend `protovalidate` with custom reusable rules that behave similarly
to the pre-defined standard constraints. This can be useful in order to
share similar custom validation logic across multiple fields or messages.

- [Errors](errors.md): This section explains the error system in `protovalidate`
and provides guidance on how to handle them effectively.

Expand All @@ -38,7 +43,7 @@ designed to help you implement new language support and assist in migrating your
existing projects to `protovalidate`.

- [Conformance](conformance.md): This document is dedicated to explaining the
Conformance tool. Learn how to use this tool to ensure all implementations
Conformance tool. Learn how to use this tool to ensure all implementations
align with `protovalidate`'s rules and constraints effectively.

- [Migrate](migrate.md): If you're planning to migrate your existing project to
Expand Down
69 changes: 69 additions & 0 deletions docs/shared-constraints.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# Shared Constraints

Custom constraints in `protovalidate` afford a lot of power, but can become
cumbersome and repetitive when the same kind of custom constraints are needed
across multiple fields or messages. To this end, `protovalidate` provides a
mechanism for creating reusable constraints that can be applied on multiple
fields.

Shared constraints require Protobuf extensions, which are not available in
proto3. Either proto2 or at least Protobuf 2023 Edition must be used to define
shared constraints. Shared constraints defined in proto2 or Protobuf 2023
Edition or later can be imported and utilized in proto3 files.

## Shared Field Constraints

To define a shared field rule, extend one of the standard rules messages. For
example, to define a new rule for `float` fields, extend
`buf.validate.FloatRules`, as follows:

```proto
import "buf/validate/shared/constraints.proto";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that these docs no longer match impl

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Documentation has been updated.

import "buf/validate/validate.proto";

extend buf.validate.FloatRules {
float abs_range = 80048952 [(buf.validate.shared.field).cel = {
id: "float.abs_range"
expression: "this >= -rule && this <= rule"
message: "float value is out of range"
}];
}
jchadwick-buf marked this conversation as resolved.
Show resolved Hide resolved
```

> [!TIP]
> Rules can refer to their own value with the `rule` constant.

> [!WARNING]
> Be mindful that extension numbers must not conflict with any other extension
> to the same message across _all_ Protobuf files in a given process. This
> restriction also applies to users that consume Protobuf files indirectly as
> dependencies. The same extension number may be re-used across different kinds
> of rules.
>
> Extension numbers may be from 1000 to 536870911, inclusive. Values from 1000
> to 50000 are reserved for [Protobuf Global Extension Registry][1] entries, and
> values from 50000 to 536870911 are reserved for randomly-generated integers.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't actually true. 50000-99999 are reserved for internal use for organizations. The random number thing is actually a somewhat-uncomfortable recommendation for me - I get the birthday paradox, but I'm not thrilled that this is what we're recommending people do. I would prefer saying "choose a number that won't conflict" and leave this up to the user.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I was mistaken on the ranges somehow, that's what I get for not actually checking.

I agree we shouldn't recommend it, so I've adjusted the documentation altogether:

  • Fixed the ranges.
  • No longer explicitly recommends randomly-generated integers
  • Discourages use of 100000....536870911 for publicly-consumed schemas at all, due to risk of conflicts
  • But keeping a suggestion from Miguel, I kept a note that suggests using a high-quality random source if the user decides to use a randomly generated integer anyway, under the belief that it's better if they do that versus merely choose arbitrary numbers.

WDYT?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Remove random number generation references
  • Coalesce extension ranges

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

> If using a randomly-generated integer as a tag number, please use an
> appropriate source of randomness. [This link to random.org][2] can be used to
> generate such an appropriate number.

[1]: https://github.com/protocolbuffers/protobuf/blob/main/docs/options.md "Protobuf Global Extension Registry"
[2]: https://www.random.org/integers/?num=1&min=50000&max=536870911&format=html&col=1&base=10 "RANDOM.ORG - Integer Generator"

Similarly to the standard constraints, a rule will take effect when it is set on
the options of a field. Here is how one might use a shared rule:

```proto
message MyMessage {
float normal_value = 1 [(buf.validate.field).float.(abs_range) = 1.0];
}
```

> [!TIP]
> Extensions are qualified by the context they are declared in. Therefore, the
> name in the parenthesis may need to be more specific if the extension appears
> in a different package or under a message. Like other type identifiers, parts
> of the path that are common with the current context can be omitted, e.g. in
> the same package the extension was defined in, the package may be entirely
> omitted. In this example, we are assuming the extension is declared in the
> same package.
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,9 @@ proto_library(
"required_field_proto2.proto",
"required_field_proto3.proto",
"required_field_proto_editions.proto",
"shared_rules_proto2.proto",
"shared_rules_proto3.proto",
"shared_rules_proto_editions.proto",
"strings.proto",
"wkt_any.proto",
"wkt_duration.proto",
Expand All @@ -50,6 +53,7 @@ proto_library(
"//proto/protovalidate-testing/buf/validate/conformance/cases/other_package:buf_validate_conformance_cases_other_package_proto",
"//proto/protovalidate-testing/buf/validate/conformance/cases/yet_another_package:buf_validate_conformance_cases_yet_another_package_proto",
"//proto/protovalidate/buf/validate:validate_proto",
"//proto/protovalidate/buf/validate/shared:shared_proto",
"@com_google_protobuf//:any_proto",
"@com_google_protobuf//:duration_proto",
"@com_google_protobuf//:timestamp_proto",
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,252 @@
// Copyright 2023 Buf Technologies, Inc.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

syntax = "proto2";

package buf.validate.conformance.cases;

import "buf/validate/shared/constraints.proto";
import "buf/validate/validate.proto";
import "google/protobuf/duration.proto";
import "google/protobuf/timestamp.proto";

extend buf.validate.FloatRules {
optional float float_abs_range_proto2 = 1161 [(buf.validate.shared.field).cel = {
id: "float.abs_range.proto2"
expression: "this >= -rule && this <= rule"
message: "float value is out of range"
}];
}

extend buf.validate.DoubleRules {
optional double double_abs_range_proto2 = 1161 [(buf.validate.shared.field).cel = {
id: "double.abs_range.proto2"
expression: "this >= -rule && this <= rule"
message: "double value is out of range"
}];
}

extend buf.validate.Int32Rules {
optional bool int32_even_proto2 = 1161 [(buf.validate.shared.field).cel = {
id: "int32.even.proto2"
expression: "this % 2 == 0"
message: "int32 value is not even"
}];
}

extend buf.validate.Int64Rules {
optional bool int64_even_proto2 = 1161 [(buf.validate.shared.field).cel = {
id: "int64.even.proto2"
expression: "this % 2 == 0"
message: "int64 value is not even"
}];
}

extend buf.validate.UInt32Rules {
optional bool uint32_even_proto2 = 1161 [(buf.validate.shared.field).cel = {
id: "uint32.even.proto2"
expression: "this % 2u == 0u"
message: "uint32 value is not even"
}];
}

extend buf.validate.UInt64Rules {
optional bool uint64_even_proto2 = 1161 [(buf.validate.shared.field).cel = {
id: "uint64.even.proto2"
expression: "this % 2u == 0u"
message: "uint64 value is not even"
}];
}

extend buf.validate.SInt32Rules {
optional bool sint32_even_proto2 = 1161 [(buf.validate.shared.field).cel = {
id: "sint32.even.proto2"
expression: "this % 2 == 0"
message: "sint32 value is not even"
}];
}

extend buf.validate.SInt64Rules {
optional bool sint64_even_proto2 = 1161 [(buf.validate.shared.field).cel = {
id: "sint64.even.proto2"
expression: "this % 2 == 0"
message: "sint64 value is not even"
}];
}

extend buf.validate.Fixed32Rules {
optional bool fixed32_even_proto2 = 1161 [(buf.validate.shared.field).cel = {
id: "fixed32.even.proto2"
expression: "this % 2u == 0u"
message: "fixed32 value is not even"
}];
}

extend buf.validate.Fixed64Rules {
optional bool fixed64_even_proto2 = 1161 [(buf.validate.shared.field).cel = {
id: "fixed64.even.proto2"
expression: "this % 2u == 0u"
message: "fixed64 value is not even"
}];
}

extend buf.validate.SFixed32Rules {
optional bool sfixed32_even_proto2 = 1161 [(buf.validate.shared.field).cel = {
id: "sfixed32.even.proto2"
expression: "this % 2 == 0"
message: "sfixed32 value is not even"
}];
}

extend buf.validate.SFixed64Rules {
optional bool sfixed64_even_proto2 = 1161 [(buf.validate.shared.field).cel = {
id: "sfixed64.even.proto2"
expression: "this % 2 == 0"
message: "sfixed64 value is not even"
}];
}

extend buf.validate.BoolRules {
optional bool bool_false_proto2 = 1161 [(buf.validate.shared.field).cel = {
id: "bool.false.proto2"
expression: "this == false"
message: "bool value is not false"
}];
}

extend buf.validate.StringRules {
optional bool string_valid_path_proto2 = 1161 [(buf.validate.shared.field).cel = {
id: "string.valid_path.proto2"
expression: "!this.matches('^([^/.][^/]?|[^/][^/.]|[^/]{3,})(/([^/.][^/]?|[^/][^/.]|[^/]{3,}))*$') ? 'not a valid path: `%s`'.format([this]) : ''"
}];
}

extend buf.validate.BytesRules {
optional bool bytes_valid_path_proto2 = 1161 [(buf.validate.shared.field).cel = {
id: "bytes.valid_path.proto2"
expression: "!string(this).matches('^([^/.][^/]?|[^/][^/.]|[^/]{3,})(/([^/.][^/]?|[^/][^/.]|[^/]{3,}))*$') ? 'not a valid path: `%s`'.format([this]) : ''"
}];
}

extend buf.validate.EnumRules {
optional bool enum_non_zero_proto2 = 1161 [(buf.validate.shared.field).cel = {
id: "enum.non_zero.proto2"
expression: "int(this) != 0"
message: "enum value is not non-zero"
}];
}

extend buf.validate.RepeatedRules {
optional bool repeated_at_least_five_proto2 = 1161 [(shared.field).cel = {
id: "repeated.at_least_five.proto2"
expression: "uint(this.size()) >= 5u"
message: "repeated field must have at least five values"
}];
}

extend buf.validate.DurationRules {
optional bool duration_too_long_proto2 = 1161 [(shared.field).cel = {
id: "duration.too_long.proto2"
expression: "this <= duration('10s')"
message: "duration can't be longer than 10 seconds"
}];
}

extend buf.validate.TimestampRules {
optional bool timestamp_in_range_proto2 = 1161 [(shared.field).cel = {
id: "timestamp.time_range.proto2"
expression: "int(this) >= 1049587200 && int(this) <= 1080432000"
message: "timestamp out of range"
}];
}

message SharedFloatRuleProto2 {
optional float val = 1 [(buf.validate.field).float.(float_abs_range_proto2) = 1.0];
}

message SharedDoubleRuleProto2 {
optional double val = 1 [(buf.validate.field).double.(double_abs_range_proto2) = 1.0];
}

message SharedInt32RuleProto2 {
optional int32 val = 1 [(buf.validate.field).int32.(int32_even_proto2) = true];
}

message SharedInt64RuleProto2 {
optional int64 val = 1 [(buf.validate.field).int64.(int64_even_proto2) = true];
}

message SharedUInt32RuleProto2 {
optional uint32 val = 1 [(buf.validate.field).uint32.(uint32_even_proto2) = true];
}

message SharedUInt64RuleProto2 {
optional uint64 val = 1 [(buf.validate.field).uint64.(uint64_even_proto2) = true];
}

message SharedSInt32RuleProto2 {
optional sint32 val = 1 [(buf.validate.field).sint32.(sint32_even_proto2) = true];
}

message SharedSInt64RuleProto2 {
optional sint64 val = 1 [(buf.validate.field).sint64.(sint64_even_proto2) = true];
}

message SharedFixed32RuleProto2 {
optional fixed32 val = 1 [(buf.validate.field).fixed32.(fixed32_even_proto2) = true];
}

message SharedFixed64RuleProto2 {
optional fixed64 val = 1 [(buf.validate.field).fixed64.(fixed64_even_proto2) = true];
}

message SharedSFixed32RuleProto2 {
optional sfixed32 val = 1 [(buf.validate.field).sfixed32.(sfixed32_even_proto2) = true];
}

message SharedSFixed64RuleProto2 {
optional sfixed64 val = 1 [(buf.validate.field).sfixed64.(sfixed64_even_proto2) = true];
}

message SharedBoolRuleProto2 {
optional bool val = 1 [(buf.validate.field).bool.(bool_false_proto2) = true];
}

message SharedStringRuleProto2 {
optional string val = 1 [(buf.validate.field).string.(string_valid_path_proto2) = true];
}

message SharedBytesRuleProto2 {
optional bytes val = 1 [(buf.validate.field).bytes.(bytes_valid_path_proto2) = true];
}

message SharedEnumRuleProto2 {
enum EnumProto2 {
ENUM_PROTO2_ZERO_UNSPECIFIED = 0;
ENUM_PROTO2_ONE = 1;
}
optional EnumProto2 val = 1 [(buf.validate.field).enum.(enum_non_zero_proto2) = true];
}

message SharedRepeatedRuleProto2 {
repeated uint64 val = 1 [(buf.validate.field).repeated.(repeated_at_least_five_proto2) = true];
}

message SharedDurationRuleProto2 {
optional google.protobuf.Duration val = 1 [(buf.validate.field).duration.(duration_too_long_proto2) = true];
}

message SharedTimestampRuleProto2 {
optional google.protobuf.Timestamp val = 1 [(buf.validate.field).timestamp.(timestamp_in_range_proto2) = true];
}
Loading
Loading