Implement TokenFlags stored on each Token (#11578)

## Summary

This PR implements the `TokenFlags` which will be stored on each `Token`
and certain flags will be set depending on the token kind. Currently,
it's equivalent to `AnyStringFlags` but it will help in the future to
provide additional information regarding certain tokens like
unterminated string, number kinds, etc.

The main motivation to add a `TokenFlags` is to store certain
information related to the token which will then be used by downstream
tools. Currently, the information is only related to string tokens. The
downstream tools should not be allowed access to the flags directly,
it's an implementation detail. Instead, methods will be provided on
`Token` to query certain information. An example can be seen in the
follow-up PR (#11592).

For example, the `Stylist` and `Indexer` uses the string flags stored on
`String`/`FStringStart` token to get certain information. They will be
updated to use these flags instead, thus removing the need for `Tok`
completely.

Prior art in TypeScript:
https://github.com/microsoft/TypeScript/blob/16beff101ae1dae0600820ebf22632ac8a40cfc8/src/compiler/types.ts#L2788-L2827

crates/ruff_python_ast/src/str_prefix.rs

@@ -150,45 +150,6 @@ impl AnyStringPrefix {
     }
 }
 
-impl TryFrom<char> for AnyStringPrefix {
-    type Error = String;
-
-    fn try_from(value: char) -> Result<Self, String> {
-        let result = match value {
-            'r' => Self::Regular(StringLiteralPrefix::Raw { uppercase: false }),
-            'R' => Self::Regular(StringLiteralPrefix::Raw { uppercase: true }),
-            'u' | 'U' => Self::Regular(StringLiteralPrefix::Unicode),
-            'b' | 'B' => Self::Bytes(ByteStringPrefix::Regular),
-            'f' | 'F' => Self::Format(FStringPrefix::Regular),
-            _ => return Err(format!("Unexpected prefix '{value}'")),
-        };
-        Ok(result)
-    }
-}
-
-impl TryFrom<[char; 2]> for AnyStringPrefix {
-    type Error = String;
-
-    fn try_from(value: [char; 2]) -> Result<Self, String> {
-        let result = match value {
-            ['r', 'f' | 'F'] | ['f' | 'F', 'r'] => {
-                Self::Format(FStringPrefix::Raw { uppercase_r: false })
-            }
-            ['R', 'f' | 'F'] | ['f' | 'F', 'R'] => {
-                Self::Format(FStringPrefix::Raw { uppercase_r: true })
-            }
-            ['r', 'b' | 'B'] | ['b' | 'B', 'r'] => {
-                Self::Bytes(ByteStringPrefix::Raw { uppercase_r: false })
-            }
-            ['R', 'b' | 'B'] | ['b' | 'B', 'R'] => {
-                Self::Bytes(ByteStringPrefix::Raw { uppercase_r: true })
-            }
-            _ => return Err(format!("Unexpected prefix '{}{}'", value[0], value[1])),
-        };
-        Ok(result)
-    }
-}
-
 impl fmt::Display for AnyStringPrefix {
     fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
         f.write_str(self.as_str())