Running Workload D and E in Parallel

[ntrhieu89]

Hi,

I am trying to run workloads D and E in parallel (using multiple nodes to issue actions).
The problem is the insert operation. How to prevent two different nodes to insert the same new user to the data store?

This link https://github.com/brianfrankcooper/YCSB/wiki/Running-a-Workload-in-Parallel gives an instruction for loading the data store with multiple nodes but doesn't mention anything about the run phase.

Thanks,
Hieu Nguyen

[busbey]

the insertstart and insertcount properties are used to constrain the keys chosen for all operations, including when workloads are in the run phase.

[busbey]

closing as stale. if you're still having trouble please reopen.

[perdelt]

Hi, I still have this problem. I load data with

recordcount=1000000
operationcount=500000
workload=site.ycsb.workloads.CoreWorkload

readallfields=true

readproportion=0
updateproportion=0
scanproportion=0.95
insertproportion=0.05

requestdistribution=zipfian

maxscanlength=100

scanlengthdistribution=uniform

insertstart=0
insertcount=500000

and

recordcount=1000000
operationcount=500000
workload=site.ycsb.workloads.CoreWorkload

readallfields=true

readproportion=0
updateproportion=0
scanproportion=0.95
insertproportion=0.05

requestdistribution=zipfian

maxscanlength=100

scanlengthdistribution=uniform

insertstart=500000
insertcount=500000

without a problem. I then have two processes for workload E
First

recordcount=2000000
operationcount=500000
workload=site.ycsb.workloads.CoreWorkload

readallfields=true

readproportion=0
updateproportion=0
scanproportion=0.95
insertproportion=0.05

requestdistribution=zipfian

maxscanlength=100

scanlengthdistribution=uniform

insertstart=1000000
insertcount=500000

Second

recordcount=2000000
operationcount=500000
workload=site.ycsb.workloads.CoreWorkload

readallfields=true

readproportion=0
updateproportion=0
scanproportion=0.95
insertproportion=0.05

requestdistribution=zipfian

maxscanlength=100

scanlengthdistribution=uniform

insertstart=1500000
insertcount=500000

and I receive a lot of PK violation errors.
Can you help me, where am I doing something wrong?
Many thanks!

[seybi87]

Hi @perdelt

the problems with the PV violations are caused by the specified recordcount values in the parallel RUN phase. The recordcount also specifies the key range of the inserted records, i.e. in the RUN phase you have specified for both workloads recordcount=2000000.

In consequence, the Workloads E1 and E2 will insert records from the key range insertstart to recordcount:

• Workload E-1 1000000 to 2000000
• Workload E-2 1500000 to 2000000

Setting the recordcount=1500000 for Workload E-1 should solve the issue.