Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What are the main contributions of p tuning? #76

Open
2catycm opened this issue May 2, 2024 · 1 comment
Open

What are the main contributions of p tuning? #76

2catycm opened this issue May 2, 2024 · 1 comment

Comments

@2catycm
Copy link

2catycm commented May 2, 2024

If it is just an implementation of existing methods, which is not novel, why the conference of p tuning paper is top CCF-A and the paper is widely cited?

So I wonder what is the core difference between p tuning and prefix tuning and deep soft prompt tuning.

From my literatur review, it seems preprending K and V is not proposed in prefix tuning, but many papers wrongly think prefix tuning is changing K V.
So is it actually your inventions? to my knowledge,prefix tuning is like deep visual prompt tuning in jia's paper, which proposed to prepend the x at each layer,not KV.

I found it worth noting that your work is utilizing KV cache that hf transformefs would have as an important implementation predicate. is it also a contribution?

@2catycm 2catycm changed the title What is the main contributions of p tuning? What are the main contributions of p tuning? May 2, 2024
@2catycm
Copy link
Author

2catycm commented May 4, 2024

i have read your paper, but i am not familiar with nlp terms, so i cannot understand your contributions. in the paper,it seems your method is exactly the same with prefix tuning and p tuningv1,just changing the evaluation dataset from nlp to nlu. In your methods section, you made a table to clarify your contribution, saying that your method have Reparam.
Deep PT
Multitask
No verb

But i got confused because it is not directly explained in the paper about what these terms are.
I have the following questions:

  • to my best knowledge,soft prompt tuning methods are not “reparameterizable” in the terms of lora paper, but it seems your reparameterizable has a different definition,and what is that based on?
  • why p tuningv1 and prefix cannot multitask
  • what is differnt not having a verbalizer in nlp

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant