You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
We have signature like below: class InputToScreenSignature(dspy.Signature): """Your task is to get project (name and description as input) and requirements (screen description and fields) and return a json document based on context document provided in the input on markdown format. """ context = dspy.InputField(default=None) project = dspy.InputField() requirements = dspy.InputField() screen_json = dspy.OutputField(desc="answer as json") # key-value pairs
Context is a document around 20 pages long and contains all design construction (json product), structure, components relation and what component could be parent, what could be child, and on, and on.
We are optimizing it using BootstrapFewShotWithRandomSearch optimizer with a considerable dataset (~500 rows) gold (human written) rows.
When used compiled module, and generate screen_json, we usually get half document. Our initial thoughts are that we're hitting max_tokens (4096) and we don’t get a full generation for a screen. If this is a problem (happened to others as well), how do we pass this issue, is there any approach to ask model to continue "part-2" of generation? Or finetuning with whole document doesn't require us to send to forward function full document always (that would minimize max_tokens spent :( ). Or any other idea at all...
I'd really love to hear your feedback and opinions on how to approach this issue!
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Hey there,
We have signature like below:
class InputToScreenSignature(dspy.Signature): """Your task is to get project (name and description as input) and requirements (screen description and fields) and return a json document based on context document provided in the input on markdown format. """ context = dspy.InputField(default=None) project = dspy.InputField() requirements = dspy.InputField() screen_json = dspy.OutputField(desc="answer as json") # key-value pairs
Context is a document around 20 pages long and contains all design construction (json product), structure, components relation and what component could be parent, what could be child, and on, and on.
We are optimizing it using BootstrapFewShotWithRandomSearch optimizer with a considerable dataset (~500 rows) gold (human written) rows.
When used compiled module, and generate screen_json, we usually get half document. Our initial thoughts are that we're hitting max_tokens (4096) and we don’t get a full generation for a screen. If this is a problem (happened to others as well), how do we pass this issue, is there any approach to ask model to continue "part-2" of generation? Or finetuning with whole document doesn't require us to send to forward function full document always (that would minimize max_tokens spent :( ). Or any other idea at all...
I'd really love to hear your feedback and opinions on how to approach this issue!
Thanks in advance!
Beta Was this translation helpful? Give feedback.
All reactions