Skip to content

[BUG] Make DeepARModel deterministic #802

Closed
1 task done
Mr-Geekman opened this issue Jul 15, 2022 · 1 comment · Fixed by #1251
Closed
1 task done

[BUG] Make DeepARModel deterministic #802

Mr-Geekman opened this issue Jul 15, 2022 · 1 comment · Fixed by #1251
Assignees
Labels
bug Something isn't working

Comments

@Mr-Geekman
Copy link
Contributor

Mr-Geekman commented Jul 15, 2022

🐛 Bug Report

Currently, DeepARModel during each forecast predicts different values.
It seems like it is not a general problem for neural nets: TFTModel predicts the same values each time.

We can probably make forecast deterministic if we fix n_samples parameter for prediction in init of DeepARModel. For deterministic behaviour n_samples should be set to None.

Expected behavior

Each time DeepARModel predicts the same values.

After fixing you should

  1. Init DeepARModel in test_inference as deterministic.
  2. Remove torch random initialization from test_inference._test_forecast_out_sample_prefix.

How To Reproduce

from pandas.util.testing import assert_frame_equal
from pytorch_forecasting.data import GroupNormalizer

from etna.datasets import TSDataset
from etna.datasets import generate_ar_df
from etna.models.nn import DeepARModel
from etna.pipeline import Pipeline
from etna.transforms import PytorchForecastingTransform


def main():
    # load data
    df = generate_ar_df(periods=100, n_segments=3, start_time="2020-01-01", freq="D", random_seed=0)
    ts = TSDataset(df=TSDataset.to_dataset(df), freq="D")

    # fit pipeline
    model = DeepARModel(max_epochs=5, learning_rate=[0.01])
    pf_transform = PytorchForecastingTransform(
        max_encoder_length=5,
        max_prediction_length=5,
        time_varying_known_reals=["time_idx"],
        time_varying_unknown_reals=["target"],
        target_normalizer=GroupNormalizer(groups=["segment"]),
    )

    pipeline = Pipeline(model=model, horizon=5, transforms=[pf_transform])
    pipeline.fit(ts)

    # forecast
    ts_forecast_1 = pipeline.forecast()
    ts_forecast_2 = pipeline.forecast()
    assert_frame_equal(ts_forecast_1.to_pandas(), ts_forecast_2.to_pandas())


if __name__ == "__main__":
    main()

Environment

No response

Additional context

No response

Checklist

  • Bug appears at the latest library version
@Mr-Geekman Mr-Geekman added the bug Something isn't working label Jul 15, 2022
@Mr-Geekman Mr-Geekman moved this to Specification in etna board Jul 15, 2022
@Mr-Geekman Mr-Geekman moved this from Specification to Todo in etna board Jul 15, 2022
@Mr-Geekman Mr-Geekman added the good first issue Good for newcomers label Jul 18, 2022
@Mr-Geekman
Copy link
Contributor Author

Try to use functions from pytorch_lightning.utilities.seed.

@Mr-Geekman Mr-Geekman removed the good first issue Good for newcomers label Apr 27, 2023
@Mr-Geekman Mr-Geekman self-assigned this Apr 28, 2023
@Mr-Geekman Mr-Geekman moved this from Todo to In Progress in etna board Apr 28, 2023
@Mr-Geekman Mr-Geekman moved this from In Progress to In Review in etna board Apr 28, 2023
@github-project-automation github-project-automation bot moved this from In Review to Done in etna board May 3, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug Something isn't working
Projects
Status: Done
Development

Successfully merging a pull request may close this issue.

1 participant