This epigraph is divided into two sub-headings in which the results obtained are briefly described. In the first of the sub-headings the predictions are described and analysed. The second sub-heading presents the results of applying quadratic programming for portfolio composition based on the predictions obtained.
2.6.1 Predictions
A more detailed explanation of the code used during the procedure described in this subsection can be found in Annex. 4 - Predictions.
As explained previously, while the model was being trained, the predictions were obtained. As was done with the artificial neural network models, the predictions were computed for the different observations using the arithmetic mean of the observations. The arithmetic mean was used because it is one of the most frequently used measures as an indicator of possible future behaviour in the study of financial time series.
The predictions will be evaluated by computing the \(MSE\) and the real values and the \(R^2\) of the results obtained by the artificial neural network models and the arithmetic means.
As explained by Glen (2023) the \(MSE\) tells you how close a regression line is to a set of points. It does this by taking the distances from the points to the regression line (these distances are the “errors”) and squaring them. Squaring is necessary to remove any negative signs. It also gives more weight to larger differences. It is called the root mean square error since you are finding the average of a set of errors. The lower the \(MSE\), the better the forecast, shown by Equation 1 as calculated.
\[ \begin{aligned} MSE &= \frac{1}{n} * \sum_{i=1}^{n}{(Y_i-\hat{Y_i})^2} \\ \end{aligned} \tag{1}\]
Where:
\(n\): number of observations
\(Y_i\): real value
\(\hat{Y_i}\): expected value
As explained by Nandakumar (2020) \(R^2\) is commonly used to explain how well a model does compared to the total mean of the observations, Equation 2:
\[ \begin{aligned} R^2 &= 1-\frac{SSR}{SST}\\ R^2 &= 1-\frac{\sum_{i=1}^{n}{(Y_i-\hat{Y_i})^2}}{\sum_{i=1}^{n}{(Y_i-\tilde{Y})^2}}\\ \end{aligned} \tag{2}\]
Where:
\(\tilde{Y}\): arithmetic mean of all observations
But this can be an unfair indicator of the performance of a regression model since it is assumed that all observations over which a mean is computed are known, and as mentioned above this is not the case for neural network models. artificial animals trained using the walk forward validation methodology. Due to this, the calculation of \(R^2\) was modified, as has been done in other investigations such as Gu, Kelly, and Xiu (2018), so that the model with which the results obtained by the ANNs used are compared is that comprised by the arithmetic means of the observations earlier than the one predicted.
Next, the different results obtained by the different models built will be briefly described. It should be noted that, although 3 different models of each of them were proposed, 10 were built, with the aim of standardizing the results obtained, since the process of building and training neural networks contains a random factor. Therefore, the results described below are the average results obtained by the various models built.
2.6.1.1 An observation
The results obtained by those models that were trained with input vectors that had one observation from each series showed, as seen in Figure 8, that in the first periods the models presented better predictions than those obtained by the arithmetic mean. . It can be observed that the effectiveness of the models in comparison with the means decays as the model progresses in time and learns from the new observations. It is also clearly seen that in most periods the \(R^2\) of this model is negative. In addition, a peak is observed in the \(MSE\) of the model at the beginning of 2020, which is understood as a loss of effectiveness of the model. This loss of effectiveness of the model could be related to sudden market movements resulting from the economic effects of Covid-19.
The previous analysis of the behaviour of the indicators of these models by period gives us an overview of how these models performed but given that the results obtained in the companies are essential for the composition of the portfolio, we will now analyse the behaviour observed in the results obtained by the 20 companies that presented the best and worst results, based on the \(R^2\) obtained as a criterion.
Observing the results of the indicators exposed in the Table 4, those companies that presented a worse \(R^2\) also present a low \(MSE\), which indicates that it is less likely that the composition of the portfolio will be altered by the results obtained by these companies. companies. On the other hand, among the companies that obtained a better \(R^2\) there are some that obtained a high \(MSE\) accompanied by a \(R^2\) greater than 5%. This indicates that differences could be generated between the compositions of the portfolios due to the differences in the predictions and that these are companies that do not have a good \(MSE\).
The results described in the previous paragraph are similar for the cases of the models built with two and three observations, respectively.
2.6.1.2 Two observations
The results obtained by those models that were trained with input vectors that had two observations from each series, it was found, as seen in Figure 9, that in the first periods the models presented better predictions than those obtained by the mean. arithmetic. It can be seen that the effectiveness of the models compared to the means decays as the model progresses in time, but they decay at a slower rate than those models trained with input vectors with one observation. It is also clearly seen that the \(R^2\) of these models has less variation than the \(R^2\) of the previously analysed models, seeing that for these models the \(R^2\) is positive in most of the periods. In addition, as in the case of the models analysed previously, a peak is also observed in the \(MSE\) of the model at the beginning of 2020.
2.6.1.3 Three observations
The results obtained by those models that were trained with input vectors that had three observations of each series, it was found, as seen in Figure 10, that in the first periods the models presented better predictions than those obtained by the mean. arithmetic. It can be seen that the effectiveness of the models compared to the means decays as the model progresses in time, but they decay at a slower rate than those models trained with input vectors with one observation. It is clearly seen that the R2 of these models has a greater variation than the R2 of the previously analysed models, observing how this variation decreases for those predictions after 2015. These models, like the first ones, presented a negative R2 in most of them. of the periods. In addition, as in the previous cases, a peak is also observed in the \(MSE\) of the model at the beginning of 2020.
2.6.2 Portfolios composition
A more detailed explanation of the code used during the procedure described in this subsection can be found in Annex. 4 - Portfolios composition.
This sub-heading describes the results obtained after applying quadratic programming to determine the composition of the portfolio. This, as well as the predictions, was made period by period with the aim of emulating a real situation in which the techniques were applied as a whole. Therefore, the present analysis focuses on the behaviour observed when using the various models and the comparison of these results with those obtained with the use of means.
As can be seen in Figure 11, the portfolios made from the predictions obtained by the neural network models that had one observation generally obtained better results than the portfolios made from the predictions using the mean. It is observed that both portfolio groups presented a lower return than the index, IBEX, in the period between 2009 and 2016.
When carrying out the analysis of the behaviour of the returns obtained by the models with two input observations, Figure 12, it is observed: the behaviour of the returns obtained by the different models varies less than those previously analysed; In this case and contrary to the previous case, the returns remain similar in the period between 2009 and 2016; and although the final result is far from the result obtained by the averages, it is lower than that obtained by the previous models, the latter being due to the fact that the evaluation of the models in this case begins in a period prior to those of the previously analysed models.
Observing the results obtained by the latest models, Figure 13, it is observed: a distribution of returns higher than those trained with two observations but lower than those trained with one observation; it is observed that the returns begin to exceed those of the index after 2013 instead of 2016 as in previous years; and it is also observed that the returns of the RNA models are higher than those of the averages and also constitute the maximum returns obtained between the different structures of RNA models.