Support postponed conflict check

[ekexium]

ref: pingcap/tidb#36579

What's changed:
Add a flag NeedConstraintCheckInPrewrite. It's a temporary flag, which means any read or write to a key with this flag will result in unsetting the flag. It's implemented by copying a new value log in the latest stage, so that later InspectStage could find the key. While it brings side effects to getValue, so we need to figure out where the side effect is needed.
An alternative way could be recording the changed value log addresses in a separate map, and resetting them every time stage changes. I chose not to do so for better compatibility (e.g. with savepoints).

I suppose it is unnecessary to encode flags in memBufferMutations, and CommitterMutationFlags could be removed. We can just read from the mem buffer for flags.
But for simplicity, I followed the routine.

[sticnarf]

Considering the possible impact to getValue, do you think it is an option to create a new variant of memDB (which may be a wrapper of the current one)? And the type of memDB is decided by the session variable when initializing?

[ekexium]

Considering the possible impact to getValue, do you think it is an option to create a new variant of memDB (which may be a wrapper of the current one)? And the type of memDB is decided by the session variable when initializing?

Great idea!

[cfzjywxk]

/cc @TonsnakeLin

[cfzjywxk]

An alternative way could be recording the changed value log addresses in a separate map, and resetting them every time stage changes. I chose not to do so for better compatibility (e.g. with savepoints).

Perhaps it would make the original checkpoint-related implementation much more complex.
BTW if the key flags are stored separately would it be possible to avoid the copy?

[ekexium]

BTW if the key flags are stored separately would it be possible to avoid the copy?

The key here is to let InspectStage be able to find the change. IIUC as long as the impl doesn't change and flags are individual from stages it won't help.

[cfzjywxk]

BTW if the key flags are stored separately would it be possible to avoid the copy?

The key here is to let InspectStage be able to find the change. IIUC as long as the impl doesn't change and flags are individual from stages it won't help.

Maybe we could record some future enhancement tasks for the MemDB like separating the flags.

[ekexium]

Maybe we could record some future enhancement tasks for the MemDB like separating the flags.

Yes if it shows obvious increment in memory footprint. Let's see it when benchmarking.

[ekexium]

"Compensating locks on read" can't totally avoid the anomaly of reading results that break uniqueness constraint.
Consider there are multiple unique keys, a select for update statement uses uk1 to read, and it locks uk1. However there are uniqueness conflict on uk2, but we are not able to know it until commit.
A more concrete case

set @@ tidb_constraint_check_in_place_pessimistic=1
create table t2 (id int primary key, uk int, unique key i1(uk))
insert into t2 values (1, 1)
begin pessimistic
insert into t2 values (2, 1)
select * from t2 use index(primary) for update -- (1,1) and (2,1)
commit -- fail

I'm considering falling back to the option of only removing the temporary flag when a key is modified. Or we can stick to the current design, but do not guarantee the integrity of read results.

[sticnarf]

I'm considering falling back to the option of only removing the temporary flag when a key is modified. Or we can stick to the current design, but do not guarantee the integrity of read results.

In my opinion, it is not so meaningful to stick to the current design if it cannot totally avoid the problem.

[TonsnakeLin]

maybe flagConstraintCheckInPrewrite is more consistency with flagKeyLocked and flagNeedConstraintCheckInPrewrite

[ekexium]

Personally I prefer need constraint check in prewrite. It clearly describes our intention.

[MyonKeminta]

Since the dependency to TiDB is replaced to a fork, should we merge this PR when the corresponding PR in TiDB is ready?

[sticnarf]

Since the dependency to TiDB is replaced to a fork, should we merge this PR when the corresponding PR in TiDB is ready?

We can accept this. The TiDB dependency is only used for integration test.