N-BEATS Missing Stack Input Residual Link

[rschwarz15]

Hey Daniel,

Absolutely loved your Tensorflow 2.0 course. Thanks a bunch for putting it together.

There was one small thing I noticed with the N-BEATS implementation where the stack input residual link is missing. The current architecture:

However the implementation in the paper has a subtraction of the stack input and the block 1 backcast:

This just requires the following additional line to add it:

# 3. Create initial backcast and forecast input (backwards predictions are referred to as residuals in the paper)
backcast, forecast = nbeats_block_layer(stack_input)
 
# ADDITIONAL LINE for input residual link
residuals = layers.subtract([stack_input, backcast], name=f"subtract_00") 
 
# 4. Create stacks of blocks

It doesn't actually improve the performance of the model :(

Thanks again for the course.

[mrdbourke]

Hey Ronen,

Thank you so much for the correction, I've just added it to the N_BEATS notebook - https://github.com/mrdbourke/tensorflow-deep-learning/blob/main/10_time_series_forecasting_in_tensorflow.ipynb

And thank you for the kind words, I really appreciate it!

[ananyajoshi2301]

Hi Daniel,
Thanks for putting up the implementation of Tensorflow NBeats.

I faced an error when running the following block of code:

2. Create input to stacks

stack_input = layers.Input(shape=(INPUT_SIZE), name="stack_input")

3. Create initial backcast and forecast input (backwards predictions are referred to as residuals in the paper)

residuals, forecast = nbeats_block_layer(stack_input)

Add in subtraction residual link, thank you to: #174

residuals = layers.subtract([stack_input, backcast], name=f"subtract_00")

4. Create stacks of blocks

for i, _ in enumerate(range(N_STACKS-1)): # first stack is already created in (3)

5. Use the NBeatsBlock to calculate the backcast as well as block forecast

backcast, block_forecast = NBeatsBlock(
input_size=INPUT_SIZE,
theta_size=THETA_SIZE,
horizon=HORIZON,
n_neurons=N_NEURONS,
n_layers=N_LAYERS,
name=f"NBeatsBlock_{i}"
)(residuals) # pass it in residuals (the backcast)

The error being NameError: name 'backcast' is not defined

in ####3 of code line: residuals = layers.subtract([stack_input, backcast], name=f"subtract_00")

which you might not have faced since you ran the dummy forecast and backcast earlier and if I do not use that then I will get this error since the backcast was never initialized.

A possible solution, please?

[STD3155]

In the added substract layer you need to feed in stack_input and the backcast from one line before. So you just need to replace "residuals" with "backcast" one line earlier.

[mrdbourke]

@STD3155 is correct

Can you check your line:

# 3. Create initial backcast and forecast input (backwards predictions are referred to as residuals in the paper)
residuals, forecast = nbeats_block_layer(stack_input)

Should be:

# 3. Create initial backcast and forecast input (backwards predictions are referred to as residuals in the paper)
backcast, forecast = nbeats_block_layer(stack_input)

[ananyajoshi2301]

Thanks! @STD3155 @mrdbourke

[dbarrau]

@mrdbourke, I also appreciate a lot the time and dedication to teaching us in your course!! Really amazing material to learn from!!

With the implemented line of code from this issue, MAE falls to 562.93317 (with the random seed set to the meaning of life). I would have liked to hear your comments about model_7 with the fix. Keep up your great work and greetings from Germany!