This project implements a deep learning framework applied to stock portfolio management. Using the top 20 stocks of FTSE (Financial Times Stock Exchange) top 100 by market share, 10 portfolios are constructed. At first, historical stock data is downloaded, cleaned, transformed and then, using RFE key features are chosen. Then 2 models are used for predicting the stock returns, an existing LSTM architecture and a designed and developed time series transformer. Finally, using Sharpe ratio based portfolio optimisation, portfolio returns are calculated and the performance of each model is evaluated. The model was tested against an LSTM architecture proposed by Sen et al. for the same task and yielded much better performance. Repository contains all the files related to the project.
- Overview
- Data Loading & Preprocessing
- Feature Engineering
- Feature selection
- Data transformation for supervised learning
- Model architecture
- Portfolio optimization model
- Results
- Conclusion & future work
- Acknowledgments
For data acquisition the widely used libraries are pandas_datareader and yfinance. Pandas_datareader is an extension of pandas library to access and retrieve available financial data from Yahoo, Tiingo, IEX, FRED, etc. But, Yahoo has been used as the main data source in most recent research due to ease of use, free and accessible, integration with pandas and active maintenance compared to libraries such as pandas_datareader (Lynn, 2021). Using yfinance, ‘Closing Price’ data for all the assets in the portfolio were downloaded for the period ‘2010-01-01’ to ‘2023-12-31’.
Incorporating sentiment analysis scores of financial news relating to each stock as an input feature and checking its impact on the model performance could be interesting. There are many off the shelf models available today for sentiment analysis. To perform sentiment analysis on historical financial news, a model that has been trained on financial data/text is vital since, the accuracy of these readily available models could change based on the data they have been trained on. FinBERT is the most widely used model for sentiment analysis on financial texts. It is based on BERT language model, and it has been further developed and trained in the finance domain (Araci, 2019). It uses a transformer architecture with only an encoder similar to the time series transformer architecture that has been proposed for this project. The output of the model has 2 values, a label and a score. The label could be either negative, positive or neutral and the score is a softmax value. In order to extract financial news for each asset, an open-source API, NewsAPI was used. Then, daily top 25 news headlines of each asset were saved in json files, for the queried time period. Then, using FinBERT a sentiment score for each news headline is calculated. If the label is ‘neutral’ the softmax score is multiplied by 0, if it’s ‘negative’ then it is multiplied by -1 and it is ‘positive’ it is multiplied by 1. Then a daily mean score is calculated per each stock and this value is saved in the database along with the other features. Technical implementation was done using Hugging Face implementation of FinBERT (ProsusAI/finbert · Hugging Face, 2023) However, ultimately sentiment score could not be incorporated into the input since, NewsAPI free subscription only provides 2 weeks of data. After making a request I was able to get access for Perigon API and query historical financial news data for a period of 2 years but, the data was inconsistent and sparse. The technical implementation was completed even though it could not be incorporated.
It is vital that data is cleaned properly for time series prediction tasks. When a database is created by downloading stock prices for a long period of time, there could be inconsistencies in the data. Therefore, data needs to be cleaned by dropping the missing values, removing outliers, standardizing the data, transforming data etc. Before dropping the missing values and removing outliers, it is important to know the impact it could have on the overall quality and quantity of the data. First the number of missing values and outliers were checked and in this case with respect to the number of entries in the database, the number of missing values and outliers were negligible. Therefore, the missing values and outliers could be easily removed from the database.
Even though ‘Closing Price’ is the most prominent feature in predicting the stock returns, other features such as RSI (Relative Strength Index), EMA (Exponential Moving Average), MoM (Momentum Indicator) and simple moving averages (5day rolling average, 10day rolling average, 20day rolling average) could contribute towards predicting better returns for the model. Therefore, the above attributes were calculated for each asset. The attributes are explained in detail below,
- RSI
It is a momentum indicator that is used in technical analysis which measures the magnitude and speed of an asset’s recent price to understand whether it is over or undervalued. The calculation has 2 steps as shown below,
$$\text{RSI}_{\text{step one}} = 100 - \left[ \frac{100}{1 + \left( \frac{\text{Average gain}}{\text{Average loss}} \right)} \right]$$
The standard number of periods used to calculate the initial RSI value is 14. Therefore, a period of 14 days will be considered for the RSI calculation. The second will be calculated as below (Relative Strength Index (RSI) Indicator Explained With Formula, 2021),
-
EMA It is a weighted moving average which gives more priority to recent price data of the asset. It can be calculated using the below equation,
$${\text{EMA} = \left( \frac{{\text{Price}{\text{current}} \times 2}}{{N+1}}\right) + \text{EMA}{\text{previous}} \times \left( 1 - \frac{2}{{N+1}}\right)}$$
Here, the N indicates the number of days chosen for EMA and the weighting given to the most recent price is greater for a short period of EMA than for a longer period of EMA (What is EMA? How to Use Exponential Moving Average With Formula, 2023).
-
MOM This attribute is used to determine the momentum of an asset when it is gaining or falling in price in the market. It simply compares the current price of an asset with the previous price of it from a given number of periods ago. It can be calculated as below,
$$\text{MOM} = \text{Closing Price}{\text{current}} - \text{Closing Price}{\text{n periods ago}}$$
-
Simple Moving Averages (SMA) It is a moving average calculated by adding the recent prices of an asset and dividing it by the number of time periods to get an average. For an example, to calculate the 5-day simple moving average of a stock, the last 5 closing prices of that particular stock will be added together and divided by 5. As input features, 5-day, 10-day and 20-day moving averages will be chosen and they will be calculated as below,
$$\text{5-day SMA} = \frac{1}{5} \sum_{t=1}^{5} \text{Closing Price}$$ $$\text{10-day SMA} = \frac{1}{10} \sum_{t=1}^{10} \text{Closing Price}$$ $$\text{20-day SMA} = \frac{1}{20} \sum_{t=1}^{20} \text{Closing Price}$$ The above attributes were calculated for each asset and saved into a pandas data frame along with the closing prices and stock ticker value. The stock ticker value represents an integer value between 1-10. This is used for converting stock names into numerical values and identifying different assets for training purposes. For example, if a portfolio consists of [AAPL, MSFT, TSLA, …], their stock ticker values would be [1, 2, 3, …]
Initially, 8 features were chosen for the model. The nature of the application is time series prediction and there are numerous feature selection techniques that could be used for selecting the vital features. Namely, RFE (Recursive Feature Elimination), correlation with the target variable or other predictors, Principal Component Analysis (PCA) etc. however, RFE is the most common feature selection method that is used for time series prediction tasks. For this task RFE is used with RF (Random Forest) to select the 5 most important features by setting the threshold to 0.3. This value was previously used by Wang et al. for a time series feature selection in their research (Wang et al., 2020). sklearn package was used for implementing feature selection. Results of RFE with RF proved that Closing price, stock ticker value, RSI, 5-day SMA, 10-day SMA and 20-day SMA are the key features for the model.
In order to prepare the dataset for a time series prediction task, it has to be transformed. At first, based on the results of the feature selection, the less important features will be dropped from the database (pandas data frame) and then the whole database is converted to a NumPy array. This resulted in an array which has number of rows equal to the number of trading days and columns equal to the number of features considered. Then the array was reshaped in such a way that each row consists of data pertaining to 5 trading days. This arrangement was made because the model predicts the returns of assets for the following week, given the current weeks’ data. Then the data will be split into training and test sets. A split of 0.6 is used for training, 0.2 for validation and 0.2 is used for testing.
There are several models that have been used in past research for stock return predictions. Recurrent Neural Networks is a popular method which has been used in multiple occasions and the LSTM (Long Short Term Memory) is the most widely used architecture (Fischer and Krauss, 2018; Wang et al., 2020). However, due to the introduction of attention mechanism in 2017, Transformers became wildly popular. Multiheaded attention mechanism became the base of any transformers model. Even though, initially it was introduced for Natural Language Processing tasks, now it has been adapted to time series prediction tasks as well (Vaswani et al., 2017). For this project, I will be using the LSTM architecture proposed by Sen et al. due to its frequent usage in literature and a time series transformer to compare the return predictions.
All models have been designed using TensorFlow. Pytorch and TensorFlow are widely used and supported deep learning packages for python. Contrary to Pytorch, TensorFlow provides better visualization, therefore debugging is more straightforward, and the implementation is easier. The program has been written using Object Oriented Programming (OOP) concepts. Separate classes have been created for time embeddings, ETL, predictions, evaluations, etc.
LSTMs are made of an input layer, one/multiple hidden layers, and an output layer. The number of neurons in the input layer is equal to the number of input features.
The special characteristic of LSTM is the capability of maintaining memory cells. To enable this, 3 gates are used in a LSTM memory cell, - Forget gate: Defines the information which needs to be removed from the memory - Input gate: Defines the information which needs to be added to the memory. - Output gate: Defines which information to use as the output
Sen et al. designed an LSTM architecture to predict the future stock prices of top five assets from nine different sectors of the Indian Stock Market. The model uses only the ‘Daily close prices’ of the past 50 days of each stock. Therefore, the input layer takes the shape (50,1). Then, the model receives this data and forwards it to the first LSTM layer which contains 256 nodes. The LSTM layer has a shape of (50,256) at the output. The first LSTM layer is followed by a dropout layer which switches off 30 percent of the nodes to avoid overfitting and then another LSTM layer (50,256) receives the output coming from the dropout layer and which is followed by another dropout layer of the same dropout rate. Then finally, a dense layer with 256 nodes receives the output of the second LSTM layer and outputs a single node which yields the predicted value of the close price. The architecture diagram of the network is shown below.
However, the layout of the of the above architecture has to be adjusted to fulfill the purpose of the task. Therefore, instead of taking the previous close prices of the past 50 days, last weeks’ data is considered and instead of predicting the close price of the next day, the closing prices for the next are predicted using the model. The model was implemented using TensorFlow.
Attention concept was first introduced by Vaswani et al. to improve the performance of neural machine translation applications. Transformers use attention to enhance the training speed of the application. The transformers have 2 main components, the encoder part, and the decoder part. However, transformer models do not necessarily need to have both. Below figure shows the Transformer model architecture proposed by Vaswani et al. (Vaswani et al., 2023)
As shown in below figure, there is an encoding section, a decoding section and the connections between them. The encoders are all identical in structure and they are comprised of 2 important sub layers, namely the Self-Attention layer and Feed-Forward layer. The Decoder has both these layers, but it contains an additional attention layer in between them to set focus on relevant parts of the input sequence. This is visualized below.
However, for the time series transformer, the decoder part is irrelevant as it only consists of an encoder. Input embeddings are fed into the encoder block along with the input vector as shown in above figure and it only happens in the first encoder block. Input embeddings are time embeddings in the context of a time series transformer, and it is further explained later in this section. This input flows through a self-attention layer and then the output of this is fed to a feed forward neural network.
Time is a vital feature for sequential data, and in some cases, time is fed as an input feature. Recent research suggests that developing a learnable vector representation or embedding for time instead of using it merely as an input feature can improve model performance (Kazemi et al., 2019).
For transformers, input embeddings are crucial, especially in the context of time series transformers where time embeddings play a critical role. Therefore, the Time2Vec representation proposed by Kazemi et al. was utilized in this project. It is represented by the equation:
Where ( t2v(\tau)[i] ) denotes the ith element of the Time2Vec vector for time (\tau). ( F ) is a periodic activation function, and a sine function was chosen as the activation function. ( \omega_i ) and ( \phi_i ) are learnable parameters: ( \omega_i ) and ( \phi_i ) for the case ( i = 0 ), and frequency and phase-shift of the sine function for ( 1 \leq i \leq k ).
The Python implementation of this function in Keras was used in the program (Ntakouris, 2021)
In the proposed architecture, transformed input for the supervised learning task is first fed to a time distributed layer, a linear and non-linear trend will be generated for each input feature, and it is concatenated to the input. Then it will be passed through multiheaded attention layer and then the result is added back to the input as shown in figure 10 through residual connections. Then the result is passed through a normalization layer before it is fed to the feed forward layer. In feed forward, 2 convolution layers are used. Dropout is used after the multiheaded layer and in between the convolution layers in order to stabilize the network and increase efficiency.
The output of the transformer block (i.e. encoder) is passed through a global average pooling layer, and it is applied to reduce the dimension of the network. Finally, 2 dense layers of 256 neurons and 5 neurons are used to get the predictions for the following week.
Keras time series transformer implementation was taken as the base code for implementing the model architecture.
Non-accelerated gradient descent optimization techniques do not work well with transformers. Therefore, as the optimizer Adam has been used. A crucial part of the multi-headed attention in order to lead towards greater stability is the learning rate warmup. In order to facilitate this, initially a small learning rate is chosen and then it gradually incremented till it reaches the base value and then decreased again. For the loss function Mean Absolute Percentage Error (MAPE) was chosen.
As discussed in the literature review, there are multiple portfolio building methods. According to MV model, variance of a given portfolio is reliant on variances of its individual stocks and the covariances among each pair as given by,
Here, the w_i is the weight assigned to each stock, s_i is the standard deviation of each stock, the minimum risk portfolio is the portfolio which yields the lowest value for variance. Even though in theory minimum variance portfolio is deemed as the best portfolio, in a real-world, investors prefer to undertake higher risks, in order to attain higher profits. Therefore, to serve this purpose Sharpe Ratio is used. Based on the return predictions of the model, the most optimised portfolio will be calculated. The criteria chosen to find the most optimised portfolio is maximizing the expected portfolio return while minimizing the risk. Sharpe ratio is calculated using the below equation,
$$\text{Sharpe Ratio} = \frac{R(x) - R_f}{\sigma}$$
Where R(x) is the average rate of return, Rf is the best available risk-free rate of return and the standard deviation of R(x) is given by σ. Sharpe ratio of 0 denotes that the investment does not yield any excess returns, ratio of 1 is considered good for investment and greater than 1 is deemed highly appropriate. The sharpe ratio is maximized based on the below equation,
Subject to:
The numerator of the function calculates the excess returns, w_i is the weight assigned to each asset and μ_i is the return of each asset and Rf is the risk-free asset. The denominator calculates the portfolio risk which is the standard deviation of its returns and σ_ijis the covariance matrix of returns. In order to solve this via python, Sequential Least Squares Quadratic Programming (SLSQP) algorithm, as implemented in scipy.optimize package is used (refer Appendix 6)
For each portfolio, 10 assets were chosen out of the top 20 stocks of the FTSE 100 by market share and returns were calculated using proposed time series architecture and Sen et al’s LSTM architecture. For each portfolio, weights of each asset, Maximum Sharpe ratio, Annualized risk and Expected portfolio returns were calculated and for each asset Mean Absolute Percentage Error and Variance ratios were calculated. The top 20 stocks of FTSE top 100 by market cap are given in table
| Stock Ticker | Company Name |
|---|---|
| SHEL.L | Shell |
| LIN | Linde |
| HSBA.L | HSBC UK |
| ULVR.L | Unilever |
| RIO.L | Rio Tinto |
| BP.L | British Petroleum |
| GSK.L | Glaxo Smith Kline |
| DGE.L | Diageo |
| REL.L | RELX |
| BATS.L | British American Tobacco |
| LSEG.L | London Stock Exchange Group |
| AON | Aon |
| RKT.L | Reckitt Benckiser |
| CPG.L | Compass Group |
| FERG.L | Ferguson |
| LLOY.L | Lloyds Banking Group |
| RR.L | Rolls-Royce Holdings |
| AAL.L | Anglo American |
| BA.L | BAE Systems |
| NG.L | National Grid |
| Portfolio | Assets |
|---|---|
| 1 | ['FERG.L', 'SHEL.L', 'RIO.L', 'AON', 'ULVR.L', 'BP.L', 'RKT.L', 'DGE.L', 'LSEG.L', 'RR.L'] |
| 2 | ['RKT.L', 'CPG.L', 'SHEL.L', 'DGE.L', 'NG.L', 'GSK.L', 'LLOY.L', 'HSBA.L', 'REL.L', 'BA.L'] |
| 3 | ['CPG.L', 'NG.L', 'BP.L', 'FERG.L', 'BATS.L', 'LIN', 'RKT.L', 'ULVR.L', 'SHEL.L', 'LLOY.L'] |
| 4 | ['HSBA.L', 'LLOY.L', 'BATS.L', 'RKT.L', 'AAL.L', 'NG.L', 'LIN', 'REL.L', 'RIO.L', 'RR.L'] |
| 5 | ['LLOY.L', 'BP.L', 'FERG.L', 'LSEG.L', 'RKT.L', 'NG.L', 'HSBA.L', 'SHEL.L', 'CPG.L', 'BA.L'] |
| 6 | ['LLOY.L', 'AON', 'BP.L', 'LIN', 'LSEG.L', 'BATS.L', 'SHEL.L', 'AAL.L', 'DGE.L', 'HSBA.L'] |
| 7 | ['LSEG.L', 'BP.L', 'ULVR.L', 'GSK.L', 'AAL.L', 'RR.L', 'RIO.L', 'REL.L', 'BATS.L', 'HSBA.L'] |
| 8 | ['REL.L', 'BA.L', 'LIN', 'RIO.L', 'BATS.L', 'BP.L', 'LLOY.L', 'LSEG.L', 'RKT.L', 'DGE.L'] |
| 9 | ['RKT.L', 'LSEG.L', 'GSK.L', 'ULVR.L', 'AON', 'RR.L', 'FERG.L', 'SHEL.L', 'DGE.L', 'REL.L'] |
| 10 | ['HSBA.L', 'AON', 'BA.L', 'LLOY.L', 'BATS.L', 'RIO.L', 'NG.L', 'RKT.L', 'AAL.L', 'SHEL.L'] |
The results of the 2 models discussed in Results section can be summarized as below,
| Portfolio | No of assets | Average MAPE | Actual return | Predicted return |
|---|---|---|---|---|
| 1 | 5 | 0.0277 | 25.701 | 26.556 |
| 2 | 10 | 0.0710 | 16.382 | 16.426 |
| 3 | 6 | 0.0649 | 27.398 | 28.656 |
| 4 | 5 | 0.0477 | 26.360 | 26.568 |
| 5 | 6 | 0.0708 | 25.1075 | 24.221 |
| 6 | 5 | 0.0255 | 19.104 | 19.593 |
| 7 | 3 | 0.0254 | 26.344 | 25.133 |
| 8 | 5 | 0.0379 | 21.736 | 22.085 |
| 9 | 5 | 0.0227 | 23.408 | 24.965 |
| 10 | 6 | 0.0580 | 17.828 | 19.281 |
Transformer Results Summary
As seen in above table, portfolio 3 yielded the highest actual return with an average MAPE of 0.0486. Despite literature suggesting 10 assets are optimal for portfolio construction, portfolios with 6 assets often yielded higher returns, with zero weights assigned to the remaining assets (as discussed in the Results section). The majority of portfolios consisted of either 5 or 6 assets.Therefore, the composition of the best portfolio built using 10,000 GBP,
| Company | Stock ticker | Weight (%) | Amount | Expected return |
|---|---|---|---|---|
| Compass Group | CPG.L | 4.382 | £438.2 | £558.26 |
| National Grid | NG.L | 1.085 | £108.5 | £138.23 |
| British Petroleum | BP.L | 18.906 | £1890.6 | £2408.59 |
| Ferguson | FERG.L | 28.053 | £2805.3 | £3573.90 |
| Shell | SHEL.L | 25.613 | £2561.3 | £3263.04 |
| Linde | LIN | 21.962 | £2196.2 | £2797.91 |
| Total Expected Return | - | - | - | £12739.93 |
Best Portfolio
| Portfolio | No of assets | Average MAPE | Actual return | Predicted return |
|---|---|---|---|---|
| 1 | 5 | 0.5044 | 25.701 | 29.317 |
| 2 | 10 | 0.5117 | 16.382 | 15.230 |
| 3 | 6 | 0.5134 | 27.398 | 20.160 |
| 4 | 5 | 0.4852 | 26.360 | 22.362 |
| 5 | 6 | 0.4926 | 25.1075 | 27.890 |
| 6 | 4 | 0.4503 | 19.104 | 19.714 |
| 7 | 10 | 0.4986 | 26.344 | 14.477 |
| 8 | 5 | 0.4900 | 21.736 | 24.196 |
| 9 | 4 | 0.5132 | 23.408 | 26.880 |
| 10 | 6 | 0.4764 | 17.828 | 24.782 |
LSTM Results Summary
As shown in Table 8, while portfolio 3 achieved the highest actual return, portfolio 1 had the highest predicted return despite a higher average MAPE of 0.5044. The poor performance of LSTM models in predicting stock returns may be attributed to their use of multiple features compared to the simpler approach of using only closing prices, as noted by Sen et al. Transformers, with their multiheaded attention mechanism and time embeddings, are better suited for capturing complex patterns.
The best performance was achieved with the following hyperparameters:
head_size: 128number of heads: 8ff_dim: 2 (filter dimension of convolution layers)number of encoder blocks: 1mlp_units: 256 (number of hidden nodes in ANN layer)mlp_dropout: 0.1 (dropout rate for ANN layer)dropout: 0.1 (dropout rate for encoder)
These settings provided optimal results without increasing model complexity unnecessarily.
In this project, historical financial asset related data were downloaded, processed and transformed to feed an existing LSTM model (Sen, Dutta and Mehtab, 2021) and a Time series transformer inspired by the time series classification proposed by Ntakouris (Ntakouris, 2021). Although in the proposal, LSTM was picked as the desired model for stock return calculations, further research into the latest neural network architectures proved time series transformers could be a better approach for the task. Therefore, an existing LSTM architecture and a designed and developed transformer architecture were considered. Using the models, stock return predictions were calculated for each portfolio, assuming there were no transaction costs and no portfolio rebalancing, therefore, all the assets had the same holding period. However, during the project sharpe ratio-based portfolio optimisation was chosen over typical Markovitz model due to high-risk high reward nature of the real-world portfolios. Finally, using the return predictions, expected portfolio returns were calculated for each portfolio.
In hindsight, I believe I was able to achieve all objectives of the project. I spent a substantial amount of time learning about transformers and training and tuning them. It was challenging to design a suitable architecture and tune them due to the lack of research with regards to transformers in the time series domain. Even though, substantial amount of time was spent and technical implementation of incorporating sentimental analysis of financial news as an input feature was completed, it could not be incorporated due to lack of free APIs. Even the paid APIs like NewsAPI did not provide data beyond 2 years and as per the requirement of this project, historical data was needed for at least 10 years.
However, the topic of portfolio optimisation is very broad, therefore, while I was working on the project, I stumbled across few interesting things that I could not incorporate in this project scope due to time constraint which hindered me from implementing real world portfolio optimisation model. Therefore, there were some limitations to the project as well. Here all the assets are held for a fixed period and returns are calculated at the end of the period. In an actual scenario, techniques such as portfolio rebalancing are used to manage the portfolio where holding periods are defined in advance and the assets are sold/bought based on the returns. This model can be further improved by incorporating such a model and dynamically changing the portfolio with time and incorporating sentiment analysis of financial news as an input to the model.