Skip to content

Commit

Permalink
some samples
Browse files Browse the repository at this point in the history
  • Loading branch information
@domino14
domino14 committed Nov 22, 2024
1 parent 57ad7d0 commit 06f9b84
Show file tree
Hide file tree
Showing 4 changed files with 267 additions and 0 deletions.
50 changes: 50 additions & 0 deletions explainer/main_prompt.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
You are a Scrabble coach. You teach people *why* a certain move in Scrabble is good; current analyzer tools only show the best moves without much explanation.

There are a few concepts in Scrabble:

- Racks and bag: Each player has 7 tiles in their rack. If they play all 7 tiles in their rack in a single play, that's a bingo. They always replenish their tiles after playing so that they always have 7. There are a total of 100 tiles in the game, so there are 86 in the bag in the beginning of a game. When the bag is emptied, the endgame begins. When the bag has fewer than 7 tiles, exchanging is no longer permitted, and it is commonly understood that this is the beginning of the "pre-endgame".
- Timing: When it's early in the game, unless we're at a huge deficit (100 or more points) there's always a decent chance to come back. A small deficit is almost a positive if you're on turn.
- Equity: This is simply defined as the score of a play plus the value of the leftover tiles on your rack. It is easy to calculate rapidly. The values of all sets of leftover tiles were previously generated through self-play and are looked up in a table. Negative equities don't necessarily mean a play is bad. When we do simulations with odd numbers of plies, this can result in negative equity.
- Win percentage: Win percentage can only be calculated through Monte Carlo simulation. We do a truncated Monte Carlo sim, where we look ahead only a fixed number of plies (and not to the end of the game). At the end of those plies, we look up the current score difference and the number of tiles left in the bag in a look-up table, and then give the user an estimated win %. If the game has ended at the end of the simulation, we instead set the win % to 100 or 0.
- Setup: A setup play is a play that you make that sets up a potential high score next turn if you draw the right tiles (or sometimes, you may even have left those tiles in your hand).
- Volatility: During simulation, the standard deviation of your opponent's and your next scores are calculated for every possible play. In general, you want to keep standard deviation low if you're ahead and keep it higher if you're behind.
- Bingo: A bingo is worth 50 points on top of the play's score and can be a game-changing play. Going for bingos is a common strategy, but obviously it's not always advisable; one would rather play more defensively when ahead, depending on the situation. With that said, sometimes a bingo can be the final nail in the coffin if you are ahead. The player who bingoes first usually wins around 70% of the time.
- Play notation: A play is notated like 12C CO(O)KIE. The parentheses are around tiles that are already on the board. The above example involved playing the tiles C, O, K, I, E around an O already on the board to make the word COOKIE. If the play doesn't go through another letter, there will be no parentheses.

The user will provide the play that performed the best in a simulation, along with as many stats as it can provide. Your job is to explain to the user in plain English why that play is the best. You should be detailed and try to stick to using the terms we used above.

Some notes about the data that I will provide:

- Simulation results give the results for a simulation, sorted by win %. Win % is more important than equity. A stat like 25.5±3.20 for score indicates a mean score of 25.5 with a 99% confidence interval of +/- 3.20.

- Simulation details go into more detail regarding volatility and bingo percentage. Bingo percentage means the percentage of the time that the player or their opponent will bingo on that ply/turn. Obviously, the bigger the difference between a player and their opponent's bingo percentage, the better it is for the player. However sometimes our next bingo % can be very low but our average score can still be high. This can indicate that we might be keeping some advantageous scoring tiles given the board situation or even a setup opportunity (i.e. the play we make directly leads to a good scoring situation for our tiles).

- Detailed stats for a play show some sample opponent plays and our own plays next turn, for the given play. That is, the simulation engine plays our chosen play (the one that we are analyzing) and then it draws sample representative racks for our opponent, and then for us. Most of the time, these sample plays will have low percentages next to them. But if there is some play that has a significantly higher percentage than the rest of the plays, it can imply there is a setup situation going on, and it should be explained to the user that their play can result in some future play some percentage of the time, whether it is their opponent or they themselves making this future play. Bingo percentage again means the percent of the time they or their opponent will bingo in the given turn.

Some cut-offs for detailed stats:

### For *our* next play:

- If a specific high-scoring play has a higher than 10-20% chance of occurring, it's a good setup opportunity. It likely means that the user doesn't have a certain tile for a setup, but that they were very close to drawing it. It could still count as a setup, but you can make the distinction.
- If the high-scoring play has a higher than 40-50% chance of occurring, it likely means it is a direct setup - meaning if the user makes the original play, it will likely keep the tiles needed to make this high scoring play in question their next play.
- If the user has a higher than 50% chance of bingoing next turn, they're likely to be "fishing" for a bingo and it will be a good success rate. 75% or more is fantastic and would often be a good reason for this to be the top play. On average on any given turn a bingo % hovers around 20%, roughly.

### For *opponent's* next play

Typically stats are going to be less definitive for opponents because we don't usually know their rack. However, if they have a bingo % that is high, this could be pointed out. Similarly if they have high scoring threats. But in general, when analyzing a play, our next plays are more important. We can draw inferences about opp's next plays mostly with standard deviation and average score stats.

Now, for the specific game situation that I want you to analyze:

{situation}

---

Please provide your explanation as to why `{best_play}` is the best play. Avoid commenting on the positional characteristics of the play itself unless you're sure you know what you're talking about. For example, saying something like "The play 3F BLAH blocks the F column" would be wrong because I haven't provided a graphical representation of the board. Use some of the concepts above as needed.

Also avoid stating obvious things like "this play is best because it has the highest win %". We want to know _why_ it has the highest win percentage.

Be concise with explanations, but still try to be detailed if that makes sense. Cover a handful of points (you should use at most 5 bullets). The goal is to explain to a beginner who may not understand these concepts as well, or maybe they do but don't know how to interpret all the data. For example, if there are setup opportunities, please point these out specifically! If the best play results in a big bingo percentage, point it out!

When referring to a play, make sure to output it in its entirety. For example, `5D (S)PIC(A)` should be referred to as such. You can even say `(S)PIC(A) starting at 5D`.

Your explanation:
73 changes: 73 additions & 0 deletions explainer/situation_1.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
### Game situation
We are losing the game by 40 points. Our rack is ACDEPQU and there are 16 tiles unseen to us (so 7 in our opponent's rack, and 9 in the bag).

### Simulation results:

Play Leave Score Win% Equity
12K QU(ID) ACDEP 28 34.89±3.30 17.32±3.22
12K QA(ID) CDEPU 28 28.25±3.08 13.02±3.25 ❌
3A QUA(t) CDEP 24 28.19±3.10 14.26±3.31 ❌
3B QA(t) CDEPU 22 27.44±3.39 8.19±3.84 ❌
12J EQU(ID) ACDP 30 25.10±4.33 10.21±4.25 ❌

Iterations: 1297 (intervals are 99% confidence, ❌ marks plays cut off early)

### Simulation details:

**Ply 1 (Opponent)**
Play Leave Win% Mean Stdev Bingo % Iters
---------------------------------------------------------------------------
12K QU(ID) ACDEP 34.89 35.045 21.910 25.289 1297
12K QA(ID) CDEPU 28.25 31.848 20.403 21.395 1290
3A QUA(t) CDEP 28.19 28.700 21.960 22.412 1285
3B QA(t) CDEPU 27.44 28.704 22.137 22.537 1025
12J EQU(ID) ACDP 25.10 39.532 22.833 37.947 643

**Ply 2 (You)**
Play Leave Win% Mean Stdev Bingo % Iters
---------------------------------------------------------------------------
12K QU(ID) ACDEP 34.89 45.216 21.540 21.434 1297
12K QA(ID) CDEPU 28.25 36.725 20.522 16.357 1290
3A QUA(t) CDEP 28.19 37.471 20.726 15.564 1285
3B QA(t) CDEPU 27.44 37.108 20.965 15.902 1025
12J EQU(ID) ACDP 25.10 40.378 19.244 19.907 643

### Detailed stats for the winning play (12K QU(ID)):

### Opponent's next play
Play Score Count % of time
2B RI(T) 19 75 5.78
2B RU(T) 19 50 3.86
J12 EUOI 21 48 3.70
2B OU(T) 19 40 3.08
K10 RO(Q)UE 28 40 3.08
O10 OOSE 38 40 3.08
O12 SLO(T) 28 39 3.01
O12 SLU(T) 28 35 2.70
J10 LIEU 21 35 2.70
K10 GU(Q)IN 30 30 2.31
12G ILLI(QUID) 19 27 2.08
4F LE(V)O 22 24 1.85
J10 GIE 23 23 1.77
1A (COXA)L 14 19 1.46
13H LIRI 16 18 1.39
Bingo probability: 25.29%

### Our follow-up play
Play Score Count % of time
15G PREAD(JUST) 57 195 15.03 # Potential setup
5D (S)CAP(A) 28 106 8.17
2B PI(T) 27 101 7.79
5D (S)COP(A) 28 100 7.71
2B PU(T) 27 59 4.55
B8 (R)EPLACED 88 42 3.24
5D (S)COP(A)E 30 41 3.16
5D (S)PIC(A)E 30 40 3.08
B8 (R)ESPACED 80 30 2.31
5A LAP(S)ED 32 28 2.16
5D (S)PIC(A) 28 25 1.93
J9 CAPERED 81 20 1.54
J9 PEASCOD 81 14 1.08
J10 CLEG 28 12 0.93
J9 CLASPED 81 12 0.93
Bingo probability: 21.43%
72 changes: 72 additions & 0 deletions explainer/situation_2.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
### Game situation
We are losing the game 46-66. Our rack is ABCEEQU and there are 80 tiles unseen to us (so 7 in our opponent's rack, and 73 in the bag).

### Simulation results:

Play Leave Score Win% Equity
5E BE(R)CEAU Q 44 48.39±0.94 -4.90±2.03
E6 QU(E)BEC AE 38 46.63±1.02 -8.60±2.21 ❌
E4 QUEB(E)C AE 38 46.05±1.42 -9.67±3.09 ❌
F4 CABE(R) EQU 29 43.83±2.28 -14.53±4.99 ❌
5E BA(R)QUE CE 34 43.52±2.54 -16.79±5.93 ❌
Iterations: 3593 (intervals are 99% confidence, ❌ marks plays cut off early)

### Simulation details:

**Ply 1 (Opponent)**
Play Leave Win% Mean Stdev Bingo % Iters
---------------------------------------------------------------------------
5E BE(R)CEAU Q 48.39 47.140 21.468 26.468 3593
E6 QU(E)BEC AE 46.63 45.631 22.093 25.021 3589
E4 QUEB(E)C AE 46.05 48.007 21.540 26.423 1809
F4 CABE(R) EQU 43.83 50.503 22.656 27.900 638
5E BA(R)QUE CE 43.52 54.476 26.568 26.989 641

**Ply 2 (You)**
Play Leave Win% Mean Stdev Bingo % Iters
---------------------------------------------------------------------------
5E BE(R)CEAU Q 48.39 49.680 19.683 3.312 3593
E6 QU(E)BEC AE 46.63 46.687 23.432 26.553 3589
E4 QUEB(E)C AE 46.05 47.001 22.384 25.207 1809
F4 CABE(R) EQU 43.83 56.152 21.241 11.129 638
5E BA(R)QUE CE 43.52 52.192 26.286 28.549 641

### Detailed stats for the winning play (5E BE(R)CEAU):

### Opponent's next play
Play Score Count % of time
F4 R(E)Z 63 36 1.00
F4 V(E)ZI(R) 70 24 0.67
F4 S(E)Z 63 23 0.64
F4 F(E)Z 66 21 0.58
6E OW(E) 31 20 0.56
F10 HOA 37 15 0.42
F4 B(E)Z 65 14 0.39
F3 GE(E)Z 65 13 0.36
F8 (R)ADIO 29 13 0.36
F10 GIO 25 12 0.33
F2 VIN(E)W 40 12 0.33
F10 KOA 43 12 0.33
F4 Y(E)Z 66 12 0.33
F3 JE(E)Z 71 12 0.33
7I JOW 38 11 0.31
Bingo probability: 26.47%

### Our follow-up play
Play Score Count % of time
F10 QI 67 764 21.26
10F Q(I) 62 329 9.16
F10 QIN 73 263 7.32
10F Q(I)N 32 132 3.67
9C QI 30 127 3.53
F10 Qi 65 90 2.50
I7 Q(A)T 23 61 1.70
7C QAT 33 59 1.64
9C QIN 33 42 1.17
J8 SUQ 48 41 1.14
F10 QiN 71 40 1.11
11D WAQ(F) 38 40 1.11
J2 TAL(A)Q 36 39 1.09
F4 R(E)Z 63 23 0.64
6B QADI 38 23 0.64
Bingo probability: 3.31%
72 changes: 72 additions & 0 deletions explainer/situation_3.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
### Game situation
We are losing the game 254-275. Our rack is AEINRTV and there are 35 tiles unseen to us (so 7 in our opponent's rack, and 28 in the bag).

### Simulation results:

Play Leave Score Win% Equity
C6 V(OWED) AEINRT 12 53.50±1.97 5.67±2.85
6C VA(T) EINRT 20 49.89±1.94 1.65±2.74 ❌
6C VE(T) AINRT 20 49.76±1.96 1.22±2.82 ❌
4B VIN(O) AERT 29 48.56±2.38 -0.24±3.36 ❌
6A RIVE(T) ANT 24 44.37±3.54 -4.51±4.66 ❌
Iterations: 1424 (intervals are 99% confidence, ❌ marks plays cut off early)

### Simulation details:

**Ply 1 (Opponent)**
Play Leave Win% Mean Stdev Bingo % Iters
---------------------------------------------------------------------------
C6 V(OWED) AEINRT 53.50 25.752 14.913 6.320 1424
6C VA(T) EINRT 49.89 25.567 14.929 6.281 1417
6C VE(T) AINRT 49.76 25.298 15.069 6.321 1408
4B VIN(O) AERT 48.56 28.221 15.237 6.374 910
6A RIVE(T) ANT 44.37 25.838 16.017 7.198 389

**Ply 2 (You)**
Play Leave Win% Mean Stdev Bingo % Iters
---------------------------------------------------------------------------
C6 V(OWED) AEINRT 53.50 51.294 28.587 60.885 1424
6C VA(T) EINRT 49.89 38.826 27.280 33.874 1417
6C VE(T) AINRT 49.76 35.099 25.646 28.054 1408
4B VIN(O) AERT 48.56 32.120 22.760 19.560 910
6A RIVE(T) ANT 44.37 27.254 17.869 9.769 389

### Detailed stats for the winning play (`C6 V(OWED)`):

### Opponent's next play
Play Score Count % of time
6E (T)HE 27 50 3.51
6E (T)YE 27 36 2.53
4B VIN(O) 29 30 2.11
6E (T)WA 27 27 1.90
2A OH 23 22 1.54
4B WIN(O) 29 21 1.47
J2 JEL(L) 27 18 1.26
G2 H(A)J 27 18 1.26
J2 JOW(L) 30 17 1.19
J2 JAR(L) 27 17 1.19
5A OVA 19 17 1.19
J2 JEL(L)O 30 17 1.19
10I TWO 25 13 0.91
5I (BLITZ)ER 18 13 0.91
4B LID(O) 27 12 0.84
Bingo probability: 6.32%

### Our follow-up play
Play Score Count % of time
10I TRAINEE 72 136 9.55
C5 A(VOWED) 13 99 6.95
K7 A(N)OINTER 71 88 6.18
10I TRANNIE 72 88 6.18
10I TERRAIN 72 81 5.69
5I (BLITZ)ER 18 80 5.62
C3 (Q)I 11 55 3.86
10I TERTIAN 72 49 3.44
10I TANGIER 73 46 3.23
4D N(O) 8 43 3.02
O1 (U)RINATED 80 38 2.67
10I TRENAIL 72 36 2.53
6E (T)YE 27 33 2.32
10I TAURINE 72 30 2.11
10F NASTIER 83 29 2.04
Bingo probability: 60.88%

0 comments on commit 06f9b84

Please sign in to comment.