Working with pre-simulated data

[fbkarsdorp]

Hi there, I was looking for some guidance to work with pre-simulated data. Is there by any chance some documentation or a tutorial available on that?

Thanks,

Folgert

[michaeldeistler]

See here, you can just load theta and x from disk instead of generating them with theta, x = simulate_for_sbi(simulator, proposal=prior, num_simulations=2000)

[fbkarsdorp]

Of course, I see! I guess my main issue was with x not fitting memory. In that case, something like

inference = NPE(prior=prior)
inference = inference.append_simulations(theta, x)

doesn't work, am I right? The most straightforward solution would then to write a custom training loop. Something like:

for i in range(iterations):
    theta, x = get_data_batch(i) # or something like that
    optimizer.zero_grad()
    losses = inference.loss(theta, condition=x)
    loss = torch.mean(losses)
    loss.backward()
    optimizer.step()

[michaeldeistler]

Yes, that is right. In such cases you have to write a custom training loop, which is explained here.