Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove need for use_attr and use_proc #5

Merged
merged 6 commits into from
Jun 6, 2023
Merged

Remove need for use_attr and use_proc #5

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
a31c6a8
working implementation no doc
gui1117 Jun 4, 2023
a9e9d31
cargo fmt
gui1117 Jun 5, 2023
a74ffeb
more precise comments
gui1117 Jun 5, 2023
b1acdf4
Merge remote-tracking branch 'origin/main' into gui-recursive
gui1117 Jun 5, 2023
cf6d754
unused args
gui1117 Jun 5, 2023
469937b
typo
gui1117 Jun 5, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
246 changes: 109 additions & 137 deletions core/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,9 @@ mod keywords {
custom_keyword!(proc_macro_attribute);
custom_keyword!(proc_macro);
custom_keyword!(proc_macro_derive);

// WARNING: Must be kept same as in macro expansions
custom_keyword!(__private_macro_magic_tokens_forwarded);
}

/// Used to parse args that were passed to [`forward_tokens_internal`].
Expand Down Expand Up @@ -596,18 +599,25 @@ pub fn forward_tokens_inner_internal<T: Into<TokenStream2>>(tokens: T) -> Result
let parsed = parse2::<ForwardedTokens>(tokens.into())?;
let target_path = parsed.target_path;
let imported_tokens = parsed.item;
let combined_tokens = match parsed.extra {
Some(extra) => quote! {
#imported_tokens,
#extra
},
None => quote!(#imported_tokens),
};
Ok(quote! {
#target_path! {
#combined_tokens
}
})
let tokens_forwarded_keyword = keywords::__private_macro_magic_tokens_forwarded::default();
let pound = Punct::new('#', Spacing::Alone);
match parsed.extra {
// no extra, used by attr, so expand to attribute macro
Copy link
Contributor Author

@gui1117 gui1117 Jun 5, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sam0x17 what I wrote here looks a bit abusive to me, but as long as we use the extra token stuff only for attribute macros we are good.
but maybe it should be refactored together with extra token feature.

otherwise we could have forward_tokens_to_attribute_macro and forward_tokens_to_proc_macro.
the implementation would redirect to forward_tokens_inner(attr: true, ...) and forward_tokens_inner(attr: false, ...). and then same for forward_tokens_internal.

Copy link
Owner

@sam0x17 sam0x17 Jun 5, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah also if I remember correctly the main blocker to having a more general thing like a tt here is anything but an ident or a literal limits what types of position you can use forward_tokens stuff with because of how decl macros work. This is the same reason I have to do that crazy manual interpolation of idents with :: instead of just a path in the export tokens macro. Extra as a string works in all positions.

Maybe a tuple like this would be better though:

("extra item 1", "extra item 2", "extra item 3", "extra item 4")

I don't think that would mess up the position capabilities at all, and at least then we wouldn't have to do string interpolation and escaping

Copy link
Owner

@sam0x17 sam0x17 Jun 5, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But yes I think it is fine to expand to a proc macro here, and I'd be happy to merge it this way

gui1117 marked this conversation as resolved.
Show resolved Hide resolved
Some(extra) => Ok(quote! {
#pound [#target_path(
#tokens_forwarded_keyword
#imported_tokens,
#extra
)] type __Discarded = ();
}),
// no extra, used by proc, import_tokens, etc, so expand to proc macro
None => Ok(quote! {
#target_path! {
#tokens_forwarded_keyword
#imported_tokens
}
}),
}
}

/// The internal implementation for the `#[with_custom_parsing(..)` attribute macro.
Expand Down Expand Up @@ -729,6 +739,7 @@ pub fn import_tokens_attr_internal<T1: Into<TokenStream2>, T2: Into<TokenStream2
let orig_sig = proc_macro.proc_fn.sig;
let orig_stmts = proc_macro.proc_fn.block.stmts;
let orig_attrs = proc_macro.proc_fn.attrs;
let orig_sig_ident = &orig_sig.ident;

// inner macro
let inner_macro_ident = format_ident!("__import_tokens_attr_{}_inner", orig_sig.ident);
Expand All @@ -743,51 +754,65 @@ pub fn import_tokens_attr_internal<T1: Into<TokenStream2>, T2: Into<TokenStream2
#(#orig_attrs)
*
pub #orig_sig {
pub #inner_sig {
let __combined_args = #mm_path::__private::syn::parse_macro_input!(#attr_ident as #mm_path::mm_core::AttrItemWithExtra);
let (#attr_ident, #tokens_ident) = (__combined_args.imported_item, __combined_args.extra);
let #attr_ident: proc_macro::TokenStream = #attr_ident.to_token_stream().into();
let (#tokens_ident, __source_path, __custom_tokens) = {
use #mm_path::mm_core::unescape_extra;
let extra = #tokens_ident.value();
let mut extra_split = extra.split("~~");
let (tokens_string, foreign_path_string, custom_parsed_string) = (
unescape_extra(extra_split.next().unwrap()),
unescape_extra(extra_split.next().unwrap()),
unescape_extra(extra_split.next().unwrap()),
);
let foreign_path: proc_macro::TokenStream = foreign_path_string.as_str().parse().unwrap();
let tokens: proc_macro::TokenStream = tokens_string.as_str().parse().unwrap();
let custom_parsed_tokens: proc_macro::TokenStream = custom_parsed_string.as_str().parse().unwrap();
(tokens, foreign_path, custom_parsed_tokens)
};
#(#orig_stmts)
*
}

use #mm_path::__private::*;
use #mm_path::__private::quote::ToTokens;
use #mm_path::mm_core::*;
let attached_item = syn::parse_macro_input!(#tokens_ident as syn::Item);
let attached_item_str = attached_item.to_token_stream().to_string();
#path_resolver
let extra = format!(
"{}~~{}~~{}",
escape_extra(attached_item_str),
escape_extra(path.to_token_stream().to_string().as_str()),
escape_extra(custom_parsed.to_token_stream().to_string().as_str())
);
quote::quote! {
#mm_override_path::forward_tokens! {
#pound path,
#inner_macro_ident,
#mm_override_path,
#pound extra
}
}.into()
}

#[doc(hidden)]
#[proc_macro]
pub #inner_sig {
let __combined_args = #mm_path::__private::syn::parse_macro_input!(#attr_ident as #mm_path::mm_core::AttrItemWithExtra);
let (#attr_ident, #tokens_ident) = (__combined_args.imported_item, __combined_args.extra);
let #attr_ident: proc_macro::TokenStream = #attr_ident.to_token_stream().into();
let (#tokens_ident, __source_path, __custom_tokens) = {
use #mm_path::mm_core::unescape_extra;
let extra = #tokens_ident.value();
let mut extra_split = extra.split("~~");
let (tokens_string, foreign_path_string, custom_parsed_string) = (
unescape_extra(extra_split.next().unwrap()),
unescape_extra(extra_split.next().unwrap()),
unescape_extra(extra_split.next().unwrap()),
syn::custom_keyword!(__private_macro_magic_tokens_forwarded);

let mut cloned_attr = #attr_ident.clone().into_iter();
let first_attr_token = cloned_attr.next();
let attr_minus_first_token = proc_macro::TokenStream::from_iter(cloned_attr);

let forwarded = first_attr_token.map_or(false, |token| {
syn::parse::<__private_macro_magic_tokens_forwarded>(token.into()).is_ok()
});

if forwarded {
#inner_macro_ident(attr_minus_first_token)
} else {
let attached_item = syn::parse_macro_input!(#tokens_ident as syn::Item);
let attached_item_str = attached_item.to_token_stream().to_string();
#path_resolver
let extra = format!(
"{}~~{}~~{}",
escape_extra(attached_item_str),
escape_extra(path.to_token_stream().to_string().as_str()),
escape_extra(custom_parsed.to_token_stream().to_string().as_str())
);
let foreign_path: proc_macro::TokenStream = foreign_path_string.as_str().parse().unwrap();
let tokens: proc_macro::TokenStream = tokens_string.as_str().parse().unwrap();
let custom_parsed_tokens: proc_macro::TokenStream = custom_parsed_string.as_str().parse().unwrap();
(tokens, foreign_path, custom_parsed_tokens)
};
#(#orig_stmts)
*
quote::quote! {
#mm_override_path::forward_tokens! {
#pound path,
#orig_sig_ident,
#mm_override_path,
#pound extra
}
}.into()
}
}

})
}

Expand All @@ -812,6 +837,7 @@ pub fn import_tokens_proc_internal<T1: Into<TokenStream2>, T2: Into<TokenStream2
let orig_sig = proc_macro.proc_fn.sig;
let orig_stmts = proc_macro.proc_fn.block.stmts;
let orig_attrs = proc_macro.proc_fn.attrs;
let orig_sig_ident = &orig_sig.ident;

// inner macro
let inner_macro_ident = format_ident!("__import_tokens_proc_{}_inner", orig_sig.ident);
Expand All @@ -830,60 +856,42 @@ pub fn import_tokens_proc_internal<T1: Into<TokenStream2>, T2: Into<TokenStream2
#(#orig_attrs)
*
pub #orig_sig {
#inner_sig {
#(#orig_stmts)
*
}

use #mm_path::__private::*;
use #mm_path::__private::quote::ToTokens;
let source_path = match syn::parse::<syn::Path>(#tokens_ident) {
Ok(path) => path,
Err(e) => return e.to_compile_error().into(),
};
quote::quote! {
#mm_override_path::forward_tokens! {
#pound source_path,
#inner_macro_ident,
#mm_override_path
}
}.into()
}

#[doc(hidden)]
#[proc_macro]
pub #inner_sig {
#(#orig_stmts)
*
}
})
}
syn::custom_keyword!(__private_macro_magic_tokens_forwarded);

/// Internal implementation for the `#[use_proc]` and `#[use_attr]` attribute macros
pub fn use_internal<T1: Into<TokenStream2>, T2: Into<TokenStream2>>(
attr: T1,
tokens: T2,
mode: ProcMacroType,
) -> Result<TokenStream2> {
parse2::<Nothing>(attr.into())?;
let orig_stmt = parse2::<BasicUseStmt>(tokens.into())?;
let orig_path = orig_stmt.path.clone();
let orig_attrs = orig_stmt.attrs;
let vis = orig_stmt.vis;
let ident = &orig_stmt
.path
.segments
.last()
.expect("path must have at least one segment")
.ident;
let hidden_ident = match mode {
ProcMacroType::Normal => format_ident!("__import_tokens_proc_{}_inner", ident),
ProcMacroType::Attribute => format_ident!("__import_tokens_attr_{}_inner", ident),
ProcMacroType::Derive => unimplemented!(),
};
let mut hidden_path: Path = orig_stmt.path.clone();
hidden_path.segments.last_mut().unwrap().ident = hidden_ident;
Ok(quote! {
#(#orig_attrs)
*
#vis use #orig_path;
#[doc(hidden)]
#vis use #hidden_path;
let mut cloned_tokens = #tokens_ident.clone().into_iter();
let first_token = cloned_tokens.next();
let tokens_minus_first = proc_macro::TokenStream::from_iter(cloned_tokens);

let forwarded = first_token.map_or(false, |token| {
syn::parse::<__private_macro_magic_tokens_forwarded>(token.into()).is_ok()
});

if forwarded {
#inner_macro_ident(tokens_minus_first)
} else {
use #mm_path::__private::*;
use #mm_path::__private::quote::ToTokens;
let source_path = match syn::parse::<syn::Path>(#tokens_ident) {
Ok(path) => path,
Err(e) => return e.to_compile_error().into(),
};
quote::quote! {
#mm_override_path::forward_tokens! {
#pound source_path,
#orig_sig_ident,
#mm_override_path
}
}.into()
}
}
})
}

Expand Down Expand Up @@ -1058,42 +1066,6 @@ mod tests {
.is_err());
}

#[test]
fn test_parse_use_stmt() {
assert!(use_internal(
quote!(),
quote!(
use some::path;
),
ProcMacroType::Attribute,
)
.is_ok());
assert!(use_internal(
quote!(),
quote!(
use some::path
),
ProcMacroType::Normal,
)
.is_err());
assert!(use_internal(
quote!(),
quote!(
use some::
),
ProcMacroType::Attribute,
)
.is_err());
assert!(use_internal(
quote!(),
quote!(
pub use some::long::path;
),
ProcMacroType::Attribute,
)
.is_ok());
}

#[test]
fn test_snake_case() {
assert_eq!(to_snake_case("ThisIsATriumph"), "this_is_a_triumph");
Expand Down
68 changes: 12 additions & 56 deletions macros/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -377,66 +377,22 @@ pub fn with_custom_parsing(attr: TokenStream, tokens: TokenStream) -> TokenStrea
}
}

/// Can be used to properly import and re-export attribute macros that were created using
/// [`macro@import_tokens_attr`].
///
/// You should use this if you ever need to import or re-export macros created using this
/// facility since it will ensure the hidden helper macros are imported and/or re-exported as
/// well.
///
/// This attribute only supports simple, non-tree-based use statements consisting of `use [vis]
/// [path];` and will fail if you attempt to provide a more complex use statement.
///
/// ## Examples
///
/// A simple import:
/// ```ignore
/// #[use_attr]
/// use my_crate::some_attribute;
/// ```
///
/// A re-export:
/// ```ignore
/// #[use_attr]
/// pub use my_crate::some_other_attribute;
/// ```
/// Deprecated: No-op
#[deprecated(
note = "`use_attr` is no longer needed for importing or re-exporting, implementation is no-op, it can be removed safely"
)]
#[proc_macro_attribute]
pub fn use_attr(attr: TokenStream, tokens: TokenStream) -> TokenStream {
match use_internal(attr, tokens, ProcMacroType::Attribute) {
Ok(tokens) => tokens.into(),
Err(err) => err.to_compile_error().into(),
}
pub fn use_attr(_attr: TokenStream, tokens: TokenStream) -> TokenStream {
tokens
}

/// Can be used to properly import and re-export proc macros that were created using
/// [`macro@import_tokens_proc`].
///
/// You should use this if you ever need to import or re-export macros created using this
/// facility since it will ensure the hidden helper macros are imported and/or re-exported as
/// well.
///
/// This attribute only supports simple, non-tree-based use statements consisting of `use [vis]
/// [path];` and will fail if you attempt to provide a more complex use statement.
///
/// ## Examples
///
/// A simple import:
/// ```ignore
/// #[use_proc]
/// use my_crate::some_proc_macro;
/// ```
///
/// A re-export:
/// ```ignore
/// #[use_proc]
/// pub use my_crate::some_other_proc_macro;
/// ```
/// Deprecated: No-op
#[deprecated(
note = "`use_proc` is no longer needed for importing or re-exporting, implementation is no-op, it can be removed safely"
)]
#[proc_macro_attribute]
pub fn use_proc(attr: TokenStream, tokens: TokenStream) -> TokenStream {
match use_internal(attr, tokens, ProcMacroType::Normal) {
Ok(tokens) => tokens.into(),
Err(err) => err.to_compile_error().into(),
}
pub fn use_proc(_attr: TokenStream, tokens: TokenStream) -> TokenStream {
tokens
}

/// A helper macro used by [`macro@import_tokens`]. Hidden from docs.
Expand Down
2 changes: 0 additions & 2 deletions tests/isolated_crate/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,7 @@
#![cfg(test)]

#[middle_crate::use_attr]
use middle_crate::distant_re_export_attr;

#[middle_crate::use_proc]
use middle_crate::distant_re_export_proc;

#[distant_re_export_attr(middle_crate::ForeignItem)]
Expand Down
Loading