GARCH model with exogenous regressors not functioning?

[KenRoytman]

Hello,

I'm struggling to figure out how to properly use this package to fit a GARCH(1,1) model with an exogenous variable. Here's an example Jupyter notebook to illustrate what I'm trying to do.

In short, using the canonical example of daily S&P 500 returns, I'm trying to add a dummy variable to a GARCH(1,1) model to examine the effect of Mondays. If I fit a model like so, arch_model(returns, x=mondays).fit().summary(), the summary output does not include any information for the dummy variable, making me think the variable was ignored altogether.

Looking through the ARCH documentation, I found a page specifying that I may need to specify a mean model for exogenous regressors. If I explicitly specify the mean model to be HARX, like so, arch_model(returns, x=mondays, mean='HARX').fit().summary(), I receive the following exception: ValueError: x must be nobs by n, where nobs is the same as the number of elements in y.

Am I missing something simple here? I'm trying to use this package for a graduate class in econometrics, and this question is causing some serious pain. Any help would be greatly appreciated. Thanks!

[dimab0]

Unfortunately, I think the arch package seems to support only exogenous variables in calculating the mean, not in estimating the variance. I don't think the package supports what you are looking for, which is a regression for variance as a function of previous returns, previous variance, and this additional dummy value.

[KenRoytman]

ARCH contributors, is the comment from @dimab0 true? I'm trying to fit the following model:

[bashtage]

What @dimab0 said is correct - there is not support for exogenous variance regressors in the current version.

[KenRoytman]

@bashtage, thanks for the clarification. Do you know of a Python or R package that does support that functionality?

[bashtage]

I don't know much R package availability, and as far as I know there are none for Python. pyflux would be the other place to look.

…

On Tue, Feb 21, 2017 at 12:17 PM Ken ***@***.***> wrote: @bashtage <https://github.com/bashtage>, thanks for the clarification. Do you know of a Python or R package that does support that functionality? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#160 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AFU5RbAJqmUiAalAWm7cgDHqSRVF2vnKks5retW-gaJpZM4MGv68> .

[KenRoytman]

Interesting. The professor for the class recommended that we use EViews for class assignments; I was hoping to build my skill set on more widely used platforms (python/R).

Is there a reason why exogenous variance regressors are not currently supported? Is this a tool that's not heavily used in the GARCH/financial econometrics space?

[bashtage]

In no particular order:

• They aren't incredibly common in practice
• General support for exogenous regressors can be more difficult to use to ensure variances stay > 0, unless restricted
• It takes time to implement anything, which means not doing other work

[KenRoytman]

Understood. I'm happy to submit a pull request. I'm just trying to understand the area a little better.

Thanks for the quick replies!

[bashtage]

PRs welcome 👍

[lawofearth]

Is there any Makov-Switching GARCH out there on Python?

[bashtage]

It has not been coded in this toolbox and I haven't come across it.

[murphywe]

Is it still true that there is no support for exogenous variance regressors in this package?

[bashtage]

I have not added exogenous regressors for volatility.

[lawofearth]

@bashtage do you came across any Vuong test on Python yet?

[camontanezp]

Dear all,

I am trying to use an ARX model (with exogenous variables) but I don't know how to pass the exogenous variables to the forecast method. I have tried adding a parameter x or exogp with the numpy array of the regressors to the following:
forecasts = am.forecast(horizon=n_periods, start=params['lags'][-1]-1)

But I get the error that x or exgop don't exist.

How can I get this problem solved?

The model is difined as follows:

params = {'lags': [11,14,20,22,28,30,32,37,47], 'volatility': HARCH([1]), 'dist': Normal()}
am = HARX(y_obs, lags=params['lags'])
am.volatility = params['volatility']
am.distribution = params['dist']

And I only need exogenous regressors for the mean.

Many thanks!!

[bashtage]

@camontanezp I think you are missing an argument to forecast. When I do the following:

from arch.univariate import HARX, HARCH, Normal
params = {'lags': [11,14,20,22,28,30,32,37,47], 'volatility': HARCH([1]), 'dist': Normal()}
am = HARX(y_obs, lags=params['lags'])
am.volatility = params['volatility']
am.distribution = params['dist']
res = am.fit()

fcasts = am.forecast(res.params, horizon=n_periods, start=params['lags'][-1]-1)

it works as expected.

You are missing res.params from my code above.

[quantvol]

@bashtage, thanks for the clarification. Do you know of a Python or R package that does support that functionality?

Not sure if you still care, but rugarch in R does support external regressors for variance.

[johnukfr]

Hi,

I am struggling with this exact issue, and I am working on a solution.
At the moment, the following is the closest thing to a solution I found online, I am working from there to try and create a class that allows you to estimate cofactors in any GARCH, GJR-GARCH or EGARCH models with or without exogenous variables and provide forecasts for any number of ‘steps’ ahead; I’ll post it on GitHub and link it here when I finish it:

https://github.com/duffau/RNN_GARCH/blob/28d39ed677bcf0b276a526b81e8dfe6b197f0933/vol_model/GARCH.py#L7

[bashtage]

The design of arch makes this a bit trickier than ideal since a VolatiltiyProcess usually takes inputs from the Mean model. I think I know how to solve it in a reasonable way to make GARCH-X and EGARCH-X models.

[shengnan92]

I go back the arch_model() function code. It needs the "x" to be a 2-dimensional array. If not, then it shows the same error. However, I do not know why "x" should be a 2-dimensional array. After I reshape "x" as a [79,1] array (because I have 79 observations), it yields the following output. If someone is familiar in this field, please give me some comments. Thank you. @bashtage

AR-X - EGARCH Model Results                          
==============================================================================
Dep. Variable:                      y   R-squared:                      -0.084
Mean Model:                      AR-X   Adj. R-squared:                 -0.098
Vol Model:                     EGARCH   Log-Likelihood:               -134.641
Distribution:                  Normal   AIC:                           279.283
Method:            Maximum Likelihood   BIC:                           291.130
                                        No. Observations:                   79
Date:                Fri, May 08 2020   Df Residuals:                       74
Time:                        03:05:56   Df Model:                            5
                                 Mean Model                                 
============================================================================
                 coef    std err          t      P>|t|      95.0% Conf. Int.
----------------------------------------------------------------------------
Const         -0.3382      0.117     -2.885  3.918e-03     [ -0.568, -0.108]
x0         9.0525e-05  4.479e-06     20.213  7.499e-91 [8.175e-05,9.930e-05]
                              Volatility Model                             
===========================================================================
                 coef    std err          t      P>|t|     95.0% Conf. Int.
---------------------------------------------------------------------------
omega          0.0413  4.952e-02      0.834      0.404 [-5.576e-02,  0.138]
gamma[1]      -0.3810      0.101     -3.763  1.681e-04    [ -0.579, -0.183]
beta[1]        1.0000  1.944e-02     51.441      0.000    [  0.962,  1.038]
===========================================================================

[johnukfr]

Hi shengnan92, X can be forced to be a 2 dimensional array and still work with only one set of observations because you (the code in this case) can set the 1st array to be zeros. This is a somewhat common thing to do because it allows for generalisation in adding a constant to your model. E.g.: if you want a constant in your model, the 1st X row would be made fully of 1s.

[bashtage]

The issue with the exog shape has been fixed in master. Master and the next release will accept 1d exog as long as it has the right number of observations.

[woneric]

Hi, I am also having problems with the exogenous model forecast using ARX function in the arch_model library. I have one exogenous continuous variable S&P500 I want to use to help predict the mean log returns of another variable using ARX mean model. I am having issues getting a multi-step forecast horizon. I have tried many fixes based upon the comments in this chain, but nothing is working. Here is a copy of the code that get me one step ahead forecasts.

import numpy as np
import pandas as pd
import datetime 
month_df = pd.read_csv('quant_strat_monthly.csv',index_col=[0,1])

Dataset = month_df.loc[month_df.index.get_level_values(0) ==1341,['SP500','log_return']]
Dataset =month_df[['SP500','log_return']].xs(1341, level = 'portfolioID')

random_state = np.random.RandomState(1)
X = Dataset[['SP500']]
X = X*100 # manual suggest scaling returns by 100 
y = Dataset[['log_return']]
# y = y*100 y should already be scaled by 100
volatility = GARCH(1,0,1)
dist = StudentsT(random_state = random_state)
mod = ARX(y=y,x=X,  lags = [1], volatility = volatility, distribution = dist )
res = mod.fit(disp = 'off')
forecasts = res.forecast(start = '2019-12-31',horizon = 6 )
forecasts.mean

I'm hoping you can tell me how to correct my code to get multi-step forecast results when I include an exogenous variable.

[bashtage]

There is no support for multistep forecasts with exogenous regressors, so you will need to roll your own.

[kcho5820]

Hi @bashtage

I'm too having trouble with GARCH model with exogenous regressors.

Below is my code and I keep having the value error.

Was wondering if you might know where I got it wrong?

data_c1 = yf.download("RIO.AX", start='2010-01-01', end='2022-03-09')
data_c3 = yf.download("CPU.AX", start='2010-01-01', end='2022-03-09')

p_rio = data_c1['Adj Close']
p_cpu = data_c3['Adj Close']

r_rio = 100 * np.log(p_rio).diff().dropna()
r_cpu = 100 * np.log(p_cpu).diff().dropna()

last_in = '2020-03-18'
first_fc = '2020-03-19'
nf = len(r_cpu[first_fc:])
r_window = len(r_cpu[:first_fc])

y = r_cpu[1:]
x = r_rio[:-1]

def qt(p, df):
    return stats.t.ppf(p, df) * np.sqrt((df - 2) / df)

s_garch = pd.Series(index=r_cpu[first_fc:].index, dtype='float64')
var25_garch=pd.Series(index=r_cpu[first_fc:].index, dtype='float64')

for i in tqdm(range(nf)):
    y_train = y[i:i+r_window]
    x_train = x[i:i+r_window]
    x_fc = x_train[i+r_window:i+r_window+1] #shape of 1x1
    
    ar1_garch_t = arch_model(y_train, x=x_train, mean='ARX', vol='GARCH', p=1, q=1, lags=1, dist='StudentsT')
    ar1_garch_t_fit = ar1_garch_t.fit(first_obs=i, last_obs=i+r_window, disp='off')
    
    fc = ar1_garch_t_fit.forecast(horizon=1, x=x_fc, reindex=False)
    
    s = np.sqrt(fc.variance)
    s_garch[i]=s.loc[s_garch.index[i]]

    var25 = fc.mean + s * qt(0.025, ar1_garch_t_fit.params['nu'])
    var1 = fc.mean + s * qt(0.01, ar1_garch_t_fit.params['nu'])

[bashtage]

I still think exog variables haven't been implemented yet.

[julietaylor9911]

Are there any other standard garch implementations in Python? Hard to believe there isn't a package in Python with support for exogenous variance variables

[filanarfahz]

Hi @kcho5820 , it's a long shot but did you manage to fix this problem? I am having the same issue at the forecasting step.

[billbwu]

The design of arch makes this a bit trickier than ideal since a VolatiltiyProcess usually takes inputs from the Mean model. I think I know how to solve it in a reasonable way to make GARCH-X and EGARCH-X models.

Hi @bashtage, since exogenous regressors are still not supported in the variance step, could you elaborate a bit on what you meant by the above re a reasonable way to make GARCH-X models work using arch? Just trying to see if there can be a relatively quick/simple workaround on this.