Automated Portfolio Optimization by Combining AI and Risk Methods


The optimized allocation of financial assets under risk/return aspects is one of the most demanding tasks in Finance. Besides the property of high-yield assets to come along with high risks, the optimization is also complicated by the multitude of “tuning parameters“.

Procedures from the areas Machine Learning (ML) / Artificial Intelligence (AI) and Financial Risk Management (FRM) are very convenient here, since they enable the prediction of possible outcomes as well as the estimation of the respective risks. By combining these knowledge areas and adding optimization routines, we developed a tool for the automated optimization of financial assets.

With that, point predictions for given portfolios and time horizons are also possible as risk estimations. By performing a multi-dimensional search, optimized portfolios under given Value at Risk (VaR) constraints are calculated.

Optimized allocation of financial assets

According to the efficient market hypothesis, all available information is already included in the price of each asset. Also – according to the theory of rational expectations (homo oeconomicus) – this information is the sole basis of the price.
In contrast, markets are not always rational, tend to panic and speculation and tend to have different phases. Hence, it is advisable to search for possible patterns for better predictions and thus a better “alpha“.

In addition, financial assets are always “risky“, i.e. investing in those assets may lead to losses. One important class of risks – the here considered market risks – stem from unpredictable price fluctuations.
As it has been shown by modern portfolio theory (MPT), the higher the return of an asset, the higher the risk. This makes it necessary to set a risk tolerance for each investment and use risk figures (like VaR) as key ingredient for each investment decision.

AI / ML and FRM methods as basis for an automated portfolio optimization

Machine Learning

Recent years have seen tremendous achievements in the are of data science, which lead to new insights into various patterns.
Here, the sub-area machine learning (ML) provides diverse regression algorithms which can be used also for predictions. Particularly worth mentioning here is the “Elastic Net“ algorithm, a regression algorithm which penalizes too high numbers as well as values of parameters and can be used for high-quality predictions.
The sub-area deep learning / artificial intelligence (AI) goes one step further by using complex (“deep“) neural nets suitable for even complex behaviours.

Financial Risk Management

Financial markets are characterized by inherent risks which cannot be predicted and remain as residual risks, even when advanced prediction models are used.
However, financial risk management (FRM) provides powerful methods – like Monte Carlo and Historical simulations – to quantify and thus handle those risks.
Based on the approach used, distributions of the possible asset values can be calculated which are in turn the basis for risk figures like VaR.

Optimized Allocation

In general, an optimized portfolio of assets in the sense used here must generate the highest possible return and at the same time not exceed a specified risk for a given investment horizon.
This can be only achieved if trustworthy and accurate predictions of the relevant market data and – additionally – reliable risk estimations are available. To bring this goal closer, a combination of ML and FRM methods is necessary.
In order to find an optimized portfolio, an additional multi-dimensional search process is necessary. During that, expectations and risks must be calculated for several asset combinations in order to find more and more optimized solutions.

One possible optimization method is the “stochastic gradient descent“ algorithm which tends to converge to (at least local) optima. Also, it is possible to use reinforcement learning to train a neural network to find optimized portfolios for given market data time series; this requires an immense amount of data, however.

Our approach for a portfolio optimization under risk-return aspects

Taking into account the above, we have developed a Python-based solution for the automated portfolio optimization. Regarding the sub-tasks involved, we proceeded as follows.

Enhanced Predictions

Based on given time series of endogenous (like stock prices) and exogenous data (like interest rates), returns and differences (for interest rates) are calculated.
The Elastic Net regression algorithm (as well as in-sample and out-of-sample tests) is applied to these data to find possible predictions. If no patterns are evident, more simple procedures are used. The returns are recalculated to absolute values in order to obtain point predictions.

Residual Risks

No prediction is perfect, as there are always random fluctuations in financial data. In order to estimate this risk, our tool analyzes the distribution of the model residuals (compared to reality).
Based on the residuals, market data and from those value scenarios are calculated.
The distribution of the value scenarios, in turn, is used to calculate the Value at Risk (VaR) at a given confidence level for each portfolio.

Automated Optimization

Our program calculates the estimated return and VaR for each given portfolio
On these calculations, a stochastic gradient descent optimization is applied. The result is a portfolio with an optimized return and a VaR (with given confidence level) not higher than a specified money amount.
The whole procedure can be applied to deliberate investment horizons (like days, weeks, months), as long as sufficient data is available.
Every other parameter in the model – market data used, VaR confidence level, VaR amount, machine learning model – is freely adjustable. In addition, it can be set whether short positions are allowed or not.

Thus, we provide an automated portfolio optimizer under deliberately customizable risk-return aspects.

Advantages of our automated portfolio optimizer

Resource Efficiency

Our Python-based application has no specific hardware requirements and runs on usual laptops and desktops.
Python itself and the used libraries are freely available. When everything is set up and the market data are provided in an appropriate form, the use requires only very limited time resources.
Thus, portfolio experts are significantly relieved from tedious detail calculations.

Speed Increase

With market data series provided in a proper form (e.g. Excel sheets), every step – from return calculation to model estimation and risk computation – runs fully automatic.
Though the routines are partially quite CPU/GPU-intensive, calculations like the ones used in the described example (see below) can be processed in minutes using today’s standard computers.
Timely market reactions and calculation reruns are unproblematic.


Trading strategies are often not only a well-guarded secret, but also often hard to justify.
In contrast, our tool is based on clear specifications: just finding the portfolio with best return under a given risk restriction.
Also, the calculations run not in a black box. Each single step is visible and validatable.
Discussions with investors and auditors can thus be considerably simplified.


Due to its modular structure, our tool can be easily enhanced, if desired. The models of the Python libraries are developed by an active community and free.

Further developments may include

  • the consideration of (Bloomberg or Twitter) text messages for which we already offer separate models
  • or the use of advanced neural networks which we can implement efficiently within the scope of a project

Usecase: Automated Optimization of Sample Portfolio

Considered Assets

For the sample usecase described here, we considered a fictitious portfolio for an investor with USD as local currency. The “current“ day is always 07/06/2018.
The investor has available a total amount of approx. $18,700 and may choose between investing in USD, EUR, GBP, CHF, Gold and the Dow Jones (as index). No short positions are allowed in this sample case.
The basis for the analysis are the according daily time series of the last 10 years (see diagram below).

Development of the considered time series
Development of the considered time series

Preprocessing and Analysis

As it can be seen in the logarithmic plot right above, the assets evolved quite differently in the last years.
Apparently, investing in EUR and GBP was not a good choice for USD investors. The plot below shows the according risk-return interrelationships at a daily basis.
The data were all analyzed with an Elastic Net Model. Based on that, predictions were made if possible; otherwise the general trend was used
The risk scenarios were obtained as historic residuals and used as basis for the optimization loop.

Historic risk-return interrelationship of the available assets
Historic risk-return interrelationship of the available assets

Based on the sample data and methods described, we run several portfolio optimizations. At this, we let several parameters constant while varying others in order to depict the evolvement of the portfolio composition.

Sample portfolio optimization, case 1
Sample portfolio optimization, case 1

In the first considered case (shown above), we defined an upper VaR barrier of $500 at a confidence level of 95%. The varied parameter was the investment horizon (y axis).
As it can be seen, our algorithm “suggests“ gold in this case for smaller horizons while shifting to the Dow for longer considered periods.
Leaving a part of the money in USD is considered as appropriate for intermediate horizons.

Sample portfolio optimization, case 2
Sample portfolio optimization, case 2

In the second case (see above), the time horizon is constantly 10 days and the total VaR barrier amount is $500. However, the VaR confidence level is changed this time (y axis).
The main shift here is – as it would reasonably be expected – the shift from gold (for low confidence levels) to the secure USD (for high confidence levels). For high confidence levels, CHF plays also a role as it is apparently seen as a more secure alternative to gold. Some Dow positions are added for diversification reasons.

Sample portfolio optimization, case 3
Sample portfolio optimization, case 3

In the above-shown 3rd case, we left the VaR confidence level constant at 95% and kept the investment horizon at 10 days. The varied parameter was the VaR amount (in USD), this time.
As it can be clearly seen, USD is the asset of choice when no money is allowed to be lost. For higher tolerances, the weighting shifts to gold. CHF and Dow play only a minor role, EUR and GBP are – as also to be expected from the time series – not chosen at all.

Our offer for an on-site implementation

According to B2B customer requirements, we offer the following development stages for an on-site implementation of our solution.

Stage 1

  • Introduction to the methodology for forecasting and risk assessment of
    financial assets
  • Handover and installation of the existing Python solution for time
    series-based return forecasting, risk estimation, and portfolio optimization – or, depending on customer requirements, support of the on-site implementation
  • Transfer and documentation of visualization and evaluation techniques

→ Customer is able to independently use and further develop methodology for calculation and optimization.

Stage 2

  • Stage 1 and additionally
  • Analysis of the concrete market data, processes, and customer
    objectives for identifying an optimal use of the procedures
  • Adaptation of the models to the given asset time series and
    selection of the parameters
  • Development of processes to achieve the optimization goals
  • Communication and documentation of results to all stakeholders

Customer has available customized procedures and processes for market data forecasting, risk assessment and portfolio optimization

Stage 3

  • Stage 1, Stage 2, and additionally
  • Specification of all requirements for an automated, possibly web-based
    IT solution
  • Suggestion and contacting of possible providers
  • Support in vendor and tool selection
  • Support in planning the implementation
  • Technical and coordinative support of the implementation project
  • Professional support after implementation of the IT solution
  • Customer has automated IT solution for automated portfolio optimization according to risk/return aspects

→ Customer has automated IT solution for automated portfolio optimization according to risk/return aspects.

If desired, we can adapt our offer flexibly. We would be happy to work out a detailed project plan with you.


Dr. Dimitrios Geromichalos
Founder / CEO
RiskDataScience UG (haftungsbeschränkt)
Theresienhöhe 28, 80339 München
Telefon: +4989244407277, Fax: +4989244407001
Twitter: @riskdatascience