Skip to content

Commit

Permalink
Doc ways to improve reproducability besides Temp (#532)
Browse files Browse the repository at this point in the history
  • Loading branch information
carsonkahn-external committed Nov 7, 2023
1 parent 0664105 commit d07833e
Showing 1 changed file with 3 additions and 2 deletions.
5 changes: 3 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -757,8 +757,9 @@ Even when specifying a temperature field of 0, it doesn't guarantee that you'll
Due to the factors mentioned above, different answers may be returned even for the same question.

**Workarounds:**
1. Using `math.SmallestNonzeroFloat32`: By specifying `math.SmallestNonzeroFloat32` in the temperature field instead of 0, you can mimic the behavior of setting it to 0.
2. Limiting Token Count: By limiting the number of tokens in the input and output and especially avoiding large requests close to 32k tokens, you can reduce the risk of non-deterministic behavior.
1. As of November 2023, use [the new `seed` parameter](https://platform.openai.com/docs/guides/text-generation/reproducible-outputs) in conjunction with the `system_fingerprint` response field, alongside Temperature management.
2. Try using `math.SmallestNonzeroFloat32`: By specifying `math.SmallestNonzeroFloat32` in the temperature field instead of 0, you can mimic the behavior of setting it to 0.
3. Limiting Token Count: By limiting the number of tokens in the input and output and especially avoiding large requests close to 32k tokens, you can reduce the risk of non-deterministic behavior.

By adopting these strategies, you can expect more consistent results.

Expand Down

0 comments on commit d07833e

Please sign in to comment.