Why Hong Kong Funds Need Post-Forecast Attribution

Forecast Accuracy Requires More Than a Headline Hit Rate

Investment forecasts are usually most visible before an important market move. Research teams publish outlooks, portfolio managers discuss potential turning points, and clients assess whether the institution appears prepared for what may come next. Once the forecast period ends, however, many firms move quickly to the next prediction. Successful calls may remain prominent, while inaccurate or ambiguous forecasts receive little structured attention. For Hong Kong funds, post-forecast attribution can turn this weakness into a competitive advantage. It creates a disciplined process for comparing the original forecast with the market outcome, identifying what worked, measuring what failed and determining whether the research contributed useful information. This approach supports stronger investment governance because it replaces selective memory with documented evidence. A headline hit rate is only one part of forecast accuracy investment research. Before calculating it, the fund must define what counts as a forecast and what qualifies as a successful result. A forecast should identify the relevant asset, expected direction or behaviour, publication time, evaluation period, probability, tolerance window and invalidation condition. Without these fields, almost any market movement can be interpreted as confirmation after it occurs. Directional accuracy measures whether the market moved in the expected direction. If a report forecast a rise in the Hang Seng Index during the following ten trading sessions, the review should compare the specified starting and ending values. The forecast should not automatically receive full credit if the index initially fell sharply, briefly moved higher after the evaluation period and then weakened again. Magnitude measures whether the size of the movement was reasonably consistent with the forecast. Calling for a major volatility expansion when the market rises by only a fraction of a percentage point may be directionally correct but economically weak. Similarly, forecasting a mild correction before a severe drawdown suggests that the model identified the direction but underestimated the level of risk. Timing error measures the distance between the projected window and the actual market event. A forecast of a turning point during one week should not be classified as precisely correct when the reversal occurs a month later. The team should establish an acceptable tolerance before seeing the outcome. This prevents forecast windows from being extended until a favourable movement eventually appears. A transparent review can therefore distinguish an exact hit, a directionally correct but poorly timed forecast, a correctly timed move with the wrong magnitude, an unconfirmed hypothesis and a complete miss. This classification provides CIOs and allocators with more meaningful information than a single success percentage.

Separate Signal Quality From the Execution Result

A good forecast does not guarantee a profitable trade, and a profitable trade does not prove that the underlying forecast was good. Hong Kong funds should evaluate signal quality and execution results separately because each depends on a different part of the investment process. Signal quality concerns the research hypothesis itself. Did the model identify the relevant asset, direction, horizon and market behaviour? Did the predicted timing window contain a meaningful change in trend, volatility, liquidity or relative performance? Did the forecast outperform a reasonable baseline? These questions should be answered without considering whether the portfolio manager acted on the research. Execution attribution begins after the signal is issued. The fund should examine whether confirmation appeared, whether the investment team responded, when the position was entered, how it was sized and whether the trade followed the approved risk rules. Slippage, transaction costs, liquidity, hedging, currency movements and exit discipline can all affect the final result. A research team might correctly forecast a recovery window for Hong Kong technology shares, but the portfolio could still lose money if it entered before confirmation, concentrated too heavily in one security or held the position after the thesis had been invalidated. In this case, the signal may have contained useful information while execution weakened the result. The opposite can also occur. A portfolio manager may profit from a position even though the forecast was inaccurate. An unexpected policy announcement, corporate event or global liquidity shift may move prices favourably for reasons that were absent from the original thesis. The return should remain recorded, but the institution should not present it as evidence that the forecast model worked. Confirmation deserves its own attribution. A timing window may identify a period of elevated opportunity, while price, breadth, turnover, volatility, Stock Connect activity or USD/CNH fail to validate the hypothesis. If the documented process requires confirmation, avoiding the trade can represent a successful decision even when the market later moves in the anticipated direction. The framework protected the portfolio from acting on incomplete evidence. A complete attribution record should therefore contain separate assessments for forecast quality, confirmation quality, portfolio decision, implementation and financial outcome. This separation helps CIOs identify whether performance problems arise from research design, interpretation, decision delays, position sizing or execution. It also protects research teams from being evaluated solely through portfolio returns that may reflect many decisions beyond the original signal.

Compare Every Forecast With a Naïve Baseline

A forecast can appear impressive until it is compared with what would have happened under a simple alternative. Hong Kong funds should benchmark their forecasts against naïve baselines to determine whether the research genuinely adds information rather than merely describing common market behaviour. The appropriate baseline depends on the forecast. A directional equity call may be compared with always forecasting a positive return, reflecting the long-term upward tendency of many equity markets. A short-horizon forecast can be compared with assuming that the previous trend will continue, that the market will remain within its recent volatility range, or that no change in regime will occur. Suppose a model correctly predicts that Hong Kong equities will rise in 62% of its forecasts. That figure may appear useful. However, if the market rose during 64% of the same evaluation periods, the model did not outperform the simplest bullish assumption. Conversely, a 56% hit rate might contain value if the model focused on difficult transition periods in which the baseline achieved only 45%. Volatility forecasts require a different comparison. A model predicting volatility expansion should be evaluated against historical volatility persistence, option-implied expectations or a rule that assumes the current regime will continue. Sector-rotation research may be compared with equal-weight exposure, the broader Hang Seng Index or a strategy that simply holds the previous period’s strongest sector. Baseline comparison should also consider economic usefulness. A small improvement in hit rate may be valuable if it helps the portfolio avoid severe drawdowns, improve entry pacing or reduce unnecessary turnover. A higher hit rate may offer little benefit if the correct forecasts concern minor movements while the misses occur during major market dislocations. Hong Kong funds should report uncertainty around these results. A 70% hit rate based on ten forecasts is less reliable than a similar result produced across hundreds of consistently defined observations. Confidence intervals, sample size and the number of independent market episodes should accompany performance statistics. Repeated forecasts issued during the same trend should not be presented as entirely independent successes. The baseline itself should be documented before the review. Changing the benchmark after seeing the outcome creates another form of hindsight bias. A research process adds credible value only when it performs better than a clearly stated alternative after accounting for its complexity, data requirements, turnover and implementation costs.

Publish Misses and Explain Regime Changes

Transparent post-forecast attribution requires unsuccessful forecasts to remain visible. Publishing only correct calls creates an exaggerated picture of accuracy and prevents the investment team from learning where the framework is weakest. Hong Kong funds can build greater trust by reviewing misses with the same structure used for successes. A miss should not automatically be treated as evidence that the entire model has failed. The review should determine whether the error came from the forecast, the data, the assumed market regime or the evaluation rules. A forecast may fail because the expected relationship no longer operates, because an unexpected event overwhelms the signal or because the model was applied outside the environment in which it historically performed well. Regime analysis is particularly important for Hong Kong funds because local assets respond to several overlapping forces. US interest rates, Hong Kong-dollar liquidity, Mainland policy, USD/CNH, Stock Connect flows, global technology valuations and geopolitical risk can influence the same market at different times. A model that performs well during stable liquidity conditions may weaken when currency pressure, policy intervention or market stress changes normal relationships. The attribution process should classify the relevant environment as far as possible. Useful categories may include expansion, slowdown, disinflation, inflation shock, liquidity easing, liquidity tightening, low volatility, volatility transition and market stress. The team can then compare forecast quality across regimes instead of relying only on aggregate accuracy. Methodology changes should also remain transparent. If the team modifies a model after repeated misses, the new version should receive a fresh identifier and implementation date. Historical results produced by the previous model should not be blended silently with the updated framework. Backtested improvements should remain separate from live, out-of-sample evidence. Publishing misses does not require releasing proprietary formulas or sensitive portfolio information. A client-facing report can state the original hypothesis, the expected period, the actual outcome, the main reason for the difference and any process change that follows. Internal committees can review more detailed model diagnostics and execution records. This process changes the purpose of forecast review. The goal is not to defend every call or find an explanation that preserves the model’s reputation. It is to determine whether the forecast contained useful information, whether the decision framework responded appropriately and whether the same error is likely to recur. A fund that acknowledges uncertainty and demonstrates learning can build more durable credibility than one that promotes a flawless record no realistic forecasting system can maintain.

Create a Monthly Post-Forecast Attribution Review

A monthly attribution process can provide Hong Kong funds with a consistent rhythm for evaluating research. The review should begin with a complete register of forecasts that reached the end of their stated evaluation periods during the month. Forecasts should not be excluded because they were unsuccessful, unconfirmed or never translated into trades. Each record should identify the asset, forecast publication time, expected direction or behaviour, probability range, horizon, timing window, magnitude expectation, confirmation requirements and invalidation conditions. The review should then document the actual market direction, realised magnitude, date of the relevant movement, maximum favourable move, maximum adverse move and whether confirmation appeared. The assessment should classify directional accuracy, magnitude accuracy and timing error separately. A concise conclusion can state whether the forecast was an exact hit, partial hit, unconfirmed signal, false positive, missed opportunity or clear miss. The assigned category should follow fixed rules so that analysts cannot apply different standards to different outcomes. The next part should compare the forecast with its predefined naïve baseline. The committee should examine whether the model provided additional information, whether that advantage remained meaningful after costs and whether the result was consistent with its claimed use. A forecast designed to identify volatility risk should not be judged solely by directional return. Execution attribution should explain whether the portfolio acted, delayed, rejected or modified the signal. Where a trade occurred, the review should capture entry timing, position size, exit, transaction costs, slippage, hedge effectiveness and contribution to return. Where no trade occurred, the record should show whether that decision followed the established confirmation and suitability rules. The monthly report should then summarise results across all completed forecasts. It can present hit rate, directional accuracy, average timing error, magnitude accuracy, false-positive frequency, confirmation rate and baseline-relative performance. These figures should be segmented by asset, forecast horizon, signal type and market regime whenever the sample size supports a meaningful comparison. The final section should focus on learning and governance. It should explain the principal misses, emerging regime changes, data problems, model revisions and any adjustments to confirmation or invalidation rules. Every change should have a responsible owner, approval status and effective date. The next review should check whether the agreed action was completed. For CIOs, research directors and allocators, post-forecast attribution provides more than a performance score. It creates an auditable connection between what the fund predicted, what the market did, how the portfolio responded and what the institution learned. When Hong Kong funds measure direction, magnitude and timing, separate signals from execution, compare results with naïve baselines and publish their misses, transparent forecast review becomes a genuine source of trust and competitive advantage.