Langfuse batch prompts api should return prompt configs as well

[ashutoshsaboo]

Describe the feature or potential improvement

Currently GET /api/public/v2/prompts only returns a list of prompts, but doesn't return configs with it. It becomes tough to build applications with it, as the config can contain crucial metadata. For example, if I want a entire list of prompts that apply to a user based on say their country code, and multiple other params which are included in the metadata, then I can't do that. First getting all prompts, and then doing a specific prompt detail lookup GET /api/public/v2/prompts/{promptName} will be too costly.

Can this be please added? @marcklingen

Additional information

No response

[marcklingen]

The GET /api/public/v2/prompts by design only returns versions/labels of a specific prompt to map which prompt versions are available. As config varies between versions of a prompt and can be large, we did not include it in the response.

I understand that being able to filter would be really useful here, what do you think about GET /api/public/v2/prompts/{promptName}/versions to fetch all versions of a single prompt including the prompt and config?

[marcklingen]

@ashutoshsaboo aiming to merge the PR later today or tomorrow, feel free to share any feedback you might have, hope this solves for your use case (which sounds really exciting)!

[ashutoshsaboo]

@marcklingen just replied on your linked PR. First up, thanks a bunch for all your prompt responses above. I truly appreciate it! 😄

One additional suggestion, is it possible to add a date filter query param to the above api (which relies on the same lastUpdatedDate for each prompt)? Basically if say the client passes, ?datetime=<5th august 2024, 11:50:50 hrs (in either epoch timestamp/datetime format however you persist it in LF's Postgres)>, then essentially LF should only return matching prompts which were last updated on or after the above timestamp. Similarly if ?datetime=xyz&label=production is passed, then it should retrieve only the prod prompts which were last updated >= xyz timestamp whichever was passed. I.e, the datetime query filter should work with other already existing filters - label/version/both whichever the client might want to pass. I believe once this is added, this should make it feasible for almost any high-scale prod usecase to be easily built on top of LF prompts. 😄

The idea is to cache prompts locally on the client (which we already do with our existing system, where we mange all our prompts before we move to LF), and not invoke LF unnecessarily for the already fetched prompts. Again to give you clarity - the major constraint is that, because this is going to be invoked for each user - so at extremely high DAU/concurrent calls, even the best of LF deployments can have major down-times (plus, why to really query Postgres unnecessarily when it's not even needed!). I thought of mentioning this here, rather than in the PR as I wasn't aware if you'd want this to be part of the same PR, or if you would want a separate PR for the same.

[marcklingen]

thanks, added this to the pr

[marcklingen]

merged, thanks again for your feedback here!

[ashutoshsaboo]

🚀 🚀

[ashutoshsaboo]

Hey I'll check the PR tomorrow. Thanks!

…

On Mon, Aug 5, 2024 at 20:30 Marc Klingen ***@***.***> wrote: @ashutoshsaboo <https://github.com/ashutoshsaboo> aiming to merge the PR later today or tomorrow, feel free to share any feedback you might have, hope this solves for your use case (which sounds really exciting)! — Reply to this email directly, view it on GitHub <#2684 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACLRHEKAVINIVAZSXQA2NKDZP6HRBAVCNFSM6AAAAABLEJGXRWVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAMRUGQZDOOA> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[ashutoshsaboo]

Hey @marcklingen , one follow up thing related to the above - one sort of a bit (major) miss that I see in Langfuse is that anyone can promote prompts to prod (like even non admins can promote prompts to prod). Ideally there should be an access control mechanism to only allow "admins" to promote prompts to prod. Else it can be a bit too risky. I might want to grant access to multiple people to play around with our data in Langfuse, but promoting things to prod and such critical access mechanisms should probably only be reserved for select members, be it admins or a new role.

What do you think? Can this be added please?

[marcklingen]

This is great feedback, can you create a new idea thread for this? https://langfuse.com/ideas

We plan to add more customizable role-based access control and this seems to be a great use-case here.