ASIA_X Multi-Horizon RL Research

ASIA_X Multi-Horizon RL Research research project details:

The project analyzes ASIA_X (Asia Composite Index) quote data using multi-horizon neural networks to estimate the directional outcome of different future time horizons from historical price/quote sequences.

It trains a neural network to evaluate WAIT, BUY, or SELL actions across six prediction horizons: 5s, 15s, 30s, 45s, 60s, and 120s. The model considers relationships between input state and historical outcomes, rather than using manually written trading rules.

The current multi-head neural network architecture includes 128-unit layers, 64-unit layer, and three actions (WAIT, BUY, SELL) per horizon, resulting in 18 total outputs. The discount factor is set to 0.95 and learning rate to 3e-4.

Although using the name DQN, the implementation is more accurately described as direct offline reward prediction, learning to predict expected historical reward for each action/horizon combination.

Raw quote data is provided in a specific format, divided into training segments (0-20) and unseen evaluation segments (21-25) for separate testing. The state representation consists of a 22-dimensional vector constructed from recent quote history and market information. Rewards for each action are determined by comparing future prices to entry price, with a winning reward of +0.85 and losing reward of -1.0.

Training uses segments 0-20 and evaluates on separate segments 21-25 to avoid overly optimistic performance estimates. Repository includes Python scripts for training and evaluation, model checkpoint, and requirements.