Allow interchangeability of KANLinear with nn.Linear #6
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently KANLinear strictly checks the dimensions are (batch_size, in_features). This does not allow compatibility with other input shapes where the feature sizes of the same. On the other hand, nn.Linear allows for any shape where the last dimension matches feature shape, eg: (, in_features), and it will reshape it to match and reshape back to the same (, out_features) on returning. Because of this difference, KANLinear and nn.Linear are currently not interchangeable, prevent people from swapping MLPs with KANs in existing code to explore differences in the two.
This change would allow for better shape compatibility, and therefore allow people to interchange KANLinear (or KAN) with nn.Linear layers.