Expert Advisory A New Approach to Optimization – Analysis and Forecasting – December 22, 2024
summation
This essay argues that traditional EA development approaches that rely on overly long learning periods can lead to overfitting and hinder performance in dynamic markets. By focusing on short-term optimization and continuous adaptation, traders can create more powerful and profitable EAs. The key is to continuously improve the parameters of the EA based on recent market data, conduct rigorous out-of-sample testing, and implement a robust risk management strategy. This approach allows EA to better adapt to changing market conditions, improving performance and reducing risk.
introduction
Expert advisors (EAs) aim to capture the unique behavioral characteristics of trading instruments. Effective EA relies on an accurate understanding of these characteristics, which requires continuous learning from historical data. However, common practice in the MQL5 community emphasizes overly long learning periods, often spanning several years. While this approach may seemingly provide a sense of security, it can lead to overfitting and hinder adaptability to evolving market dynamics.
Risks of long-term learning:
overfitting
Long learning periods increase the risk of overfitting, where the EA over-adapts to past market conditions, including anomalies and noise. This can lead to poor performance when market conditions change.
a false sense of security
Presenting decades of backtest results as a seemingly stable asset curve can create the illusion of safety. However, these results may not accurately reflect actual performance, especially in volatile or rapidly changing markets.
Reading history, not predicting the future
EA’s who have been trained for too long often become “history readers” who effectively remember past price movements rather than identifying and adapting to evolving market patterns.
High risk of losing your account due to large stop losses
A significant portion of MQL5 users do not properly test or optimize their EAs. Consider EA, which has lost up to $1400 over the past five years. This ideally represents the maximum acceptable risk. If this EA incurs a significant loss, you must follow the Stop Loss (SL) order until the maximum loss of $1400 is reached or exceeded. However, human psychology often tempts us to stay in a position longer than we should in the hope of recovery.
What happens if long-term backtesting is inaccurate and EA’s actual maximum loss exceeds $1400? This can result in significant, unexpected losses and potentially put your entire trading account at risk. This scenario carries significant risk that could result in significant account losses.
By carefully considering your risk parameters and conducting thorough backtesting, you can work to minimize stressful situations and improve your trading experience.
Short-term optimization example:
Adaptability to evolving markets
Focusing on shorter learning periods, such as 5-6 months, allows EA to more effectively adapt to recent market trends, including short-term cycles, news-driven volatility, and changes in market sentiment.
reduce risk
By focusing on recent market trends, EA can better assess and mitigate current risks, such as sudden market changes or unexpected events. This can lead to more realistic risk management and reduced losses.
improved performance
By continuously adapting to changing market conditions, short-term optimization can lead to improved performance and potentially higher returns compared to EA trained on static long-term data.
Additional considerations:
The financial market is constantly evolving. Factors such as the behavior of market participants, advancements in trading technology, and changes in economic conditions are constantly in flux. It is unrealistic to expect a single trading algorithm to consistently capture the characteristics of a trading instrument over a long period of time, such as 5 or 10 years.
Even if an algorithm can achieve consistent long-term performance, it may require significant constraints to mitigate the risk of overfitting to historical data. This rigorous approach can significantly reduce potential returns and result in an unfavorable risk-reward profile.
This study proposes a new approach to optimizing professional advisors, with the goal of improving their performance and improving risk management.
Let’s explore this concept further by examining the characteristics of short-term cycles.
Brief description of short-term cyclical characteristics
It is a short-term cyclical characteristic that is influenced by a variety of factors, including macroeconomic data releases, market sentiment, geopolitical events, and central bank policy decisions. These cycles are often driven by trader psychology, market liquidity, and algorithmic trading strategies. Common characteristics and periods are as follows:
1. Intraday cycle
continue: From a few hours to a day.
characteristics:
It is usually determined by the market session (e.g. Asian, European and US trading hours).
Volatility spikes during major market openings and key economic data announcements (e.g. non-farm payrolls, ECB announcements, or Federal Reserve interest rate decisions).
Patterns often include range trading during low volume hours and breakouts during high volume sessions.
2. Cycle of multiple days
continue: 2~5 days.
characteristics:
They are often associated with short-term changes in sentiment, including positioning ahead of major economic or geopolitical events.
These include patterns such as the “Monday effect” or reactionary movements following weekend news.
These cycles may reflect strong trends for a particular technology level or corrective actions following consolidation.
3. Weekly or biweekly cycle
continue: 1~3 weeks.
characteristics:
In particular, in the case of the ECB or Federal Reserve, it can be adjusted according to the central bank meeting cycle.
It reflects market adjustments to changes in monetary policy expectations or changes in macroeconomic data.
Traders often refer to this as part of a “mini trend” within a broader trend.
4. Seasonal cycle
continue: From several weeks to several months.
characteristics:
Seasonal trends may be caused by recurring economic factors, such as fiscal year-end flows, tax deadlines, or corporate repatriations. Mid-year and year-end periods often show distinct trading patterns associated with portfolio rebalancing or hedging activities.
By analyzing the short-term nature of price movements, key cyclical patterns can be identified. If you choose a sufficiently long learning period, your EA can potentially learn through the following pattern, typically:
daily cycle
give multiple days
Weekly or biweekly cycle
seasonal cycle
These cycles can provide valuable insight into market behavior and present potential trading opportunities. However, focusing on historical data from eight years ago may not be relevant to current market conditions. We must prioritize learning from the most recent price action to adapt to evolving market dynamics.
methodology:
1- Study period definition:
Determine the appropriate study period. The above research suggests that a learning period of 5 to 6 months is generally sufficient. This can be shortened taking into account the desired transaction frequency and typical cycle length of the product.
2- Optimization:
Optimize EA parameters within a defined learning window.
3- Out-of-sample testing:
We perform rigorous out-of-sample testing, including forward and rewind tests, to evaluate the performance of the EA on data not used in the optimization process.
4 – Regular re-optimization:
Re-optimize your EA on a regular basis, ideally monthly or bi-weekly, or more frequently for high-frequency trading strategies, to ensure continuous adaptation to evolving market conditions.
application
If today is December 21st, you could set up an optimization routine like this:
Applying this approach to a trading algorithm results in the following asset curve: Upon inspection, this set file is accepted because it performs well both in and out of sample testing.
How should we manage risk?
Significant news events or economic data releases can suddenly change market sentiment, potentially exceeding the scope of an EA’s study period.
Implementing Stop Loss (SL) orders is very important for risk management. SL levels should be determined carefully to avoid overly strict settings, which may lead to frequent early liquidations, or overly loose settings, which may not adequately protect capital in adverse market conditions.
Ideally, SL should be set to limit potential losses to an amount that does not exceed average daily profits. For example, if your average daily profit is $40, your SL should not exceed this amount.
While some flexibility may be possible when trading exclusively with EA, it is generally recommended to limit potential losses to no more than your 3-day average profit.
Therefore, your EA parameters and position sizes should be tailored to these risk management guidelines.
In our specific example, we need to implement a stop-loss order when the withdrawal amount (DD) exceeds $45, with some buffer for additional safety. It is important to note that this professional advisor’s long-term maximum drawdown (DD) can potentially reach $700 or even $800. By focusing on short-term optimization and adapting to recent market conditions, we have significantly reduced the potential for material losses. This approach prioritizes risk management and aims to minimize the impact of unexpected market events on your trading account.
conclusion
By embracing short-term optimization and focusing on recent market action, traders can improve their EA’s adaptability, performance and risk management. This approach requires a more proactive and dynamic approach to EA management, but can ultimately lead to a more robust and profitable trading system.
