Refactor JSON deserialization to assist code generation #474
Conversation
|
|
||
| /// Represents the location of a token | ||
| #[derive(Debug, Eq, PartialEq, Copy, Clone)] | ||
| pub struct TokenOffset(pub usize); |
There was a problem hiding this comment.
I would probably just name this Offset. We're already in a module named Token
| expect_fn!(expect_start_array, StartArray); | ||
|
|
||
| /// Expects a string or null token, and if its a string, returns its unescaped value. | ||
| pub fn expect_string_or_null( |
There was a problem hiding this comment.
I was a little surprised to find no expect_string.
There was a problem hiding this comment.
I need to think carefully about the null handling. Right now, it just passes off the Option to the builder's set function, and later, if that field was actually required, the fallible builder will fail. I'm not certain if this works correctly for, say, a primitive int field.
| expect_fn!(expect_start_object, StartObject); | ||
| expect_fn!(expect_start_array, StartArray); | ||
|
|
||
| /// Expects a string or null token, and if its a string, returns its unescaped value. |
There was a problem hiding this comment.
| /// Expects a string or null token, and if its a string, returns its unescaped value. | |
| /// Expects a string or null token. If the value was a string, its **unescaped** value will be returned |
| ) -> Result<Option<String>, Error> { | ||
| match token.transpose()? { | ||
| Some(Token::ValueNull { .. }) => Ok(None), | ||
| Some(Token::ValueString { value, .. }) => Ok(Some(value.to_unescaped()?.to_string())), |
There was a problem hiding this comment.
I guess you know you're definitely going to need a string at this point so there is no point in keeping at a Cow?
There was a problem hiding this comment.
Exactly. The only time this gets called is when we're moving a string value into a builder.
| #[test] | ||
| fn skip_simple_value() { | ||
| let mut tokens = json_token_iter(b"null true"); | ||
| skip_value(&mut tokens).unwrap(); |
There was a problem hiding this comment.
so it seems like json_token_iter accepts invalid JSON? not necessarily a problem, just worth documenting what is accepted and what is not
There was a problem hiding this comment.
I don't think that's necessarily invalid, but rather, multiple values. I could make it only read one value and then fail if there's more, but that's effectively what the codegen doing with it.
| Ok(Token::ValueBool { | ||
| offset, | ||
| value: false, | ||
| }) | ||
| } | ||
| _ => unreachable!(), |
There was a problem hiding this comment.
probably worth commenting why this is unreachable or putting a message in unreachable
| _ => unreachable!(), | |
| _ => unreachable!("this function MUST only be called when the next byte is t or f"), |
| /// Validation is done on the fly, so it is possible for it to parse an invalid JSON document | ||
| /// until it gets to the first [Error]. | ||
| /// | ||
| /// JSON string values are left unescaped in the [Token::ValueString] as an [EscapedStr], |
More work towards #161. These changes to the rust-runtime facilitate the JSON parser generator by making it possible to create custom errors, and also to expect certain value types or skip entire values in the token stream.
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.