dynamic_axes in ONNX export: when to use them, when not to

dynamic_axes is the most misunderstood parameter in torch.onnx.export (and the equivalents in tf2onnx and skl2onnx). Used right, it gives you flexibility — one model, many batch sizes. Used wrong, it ruins LSTM exports, creates phantom CPU fallbacks, and produces models with mysterious "Memcpy nodes added" warnings.

The rule, written once so you remember it: only the batch dimension should be dynamic. Everything else, fix at export time.

What dynamic_axes actually does

When you export a model, every dimension of every input and output has a size. By default, those sizes are frozen — whatever the dummy input's shape was, that's the only shape the exported model accepts. Pass shape (1, 120, 1) at export, and the model only accepts (1, 120, 1) inputs.

dynamic_axes tells the exporter "this dimension can vary at runtime — don't bake it as a constant." The exported model stores a symbolic name (e.g., "batch") instead of a number. At inference time, the runtime resolves the symbol to whatever shape you actually pass.

The "batch only" rule

For models destined for MQL5, the only dimension that should be dynamic is the batch dimension (axis 0). Everything else — sequence length, feature count, hidden size — should be frozen at export.

the right way

torch.onnx.export( model, dummy, "model.onnx", input_names=["input"], output_names=["output"], opset_version=17, dynamic_axes={ "input": {0: "batch"}, # ONLY batch "output": {0: "batch"}, }, )

Consequences of dynamic seq_len

If you also make sequence length dynamic, three things may happen:

LSTM export fails or silently breaks. Documented in the LSTM article. The exporter has to capture the time loop as a runtime Loop op; the capture either crashes or produces wrong outputs.
Shape-manipulation ops appear in the graph. The model needs to figure out tensor shapes at runtime, which means ops like Shape, Slice, Reshape. These often run on CPU even with CUDA enabled — producing the Memcpy nodes warning.
Inference is slower. Even when it works, the runtime can't pre-compile a graph specialized for your specific shape. Each OnnxRun does extra dispatch work.

For trading models, sequence length is essentially always known at design time. A 1-hour timeframe EA looking at the past 5 days uses seq_len=120 always. There's no reason to keep it dynamic.

How dynamic axes interact with MQL5

Every dynamic axis in the exported model must be resolved with OnnxSetInputShape before OnnxRun can execute. Specifically:

resolving on the MQL5 side

// Model exported with input axis 0 (batch) dynamic, axis 1 (seq) static at 120 const long input_shape[] = {1, 120, 1}; // batch=1 resolves the dynamic axis OnnxSetInputShape(ExtHandle, 0, input_shape);

If a dimension is static in the exported model, you can pass a matching number or even leave it — the runtime knows. If a dimension is dynamic, you must resolve it.

Cases where you might break the rule

Two narrow exceptions:

Inference on multiple batch-sizes simultaneously. If you batch up multiple symbols (or timeframes) and pass them through together to amortize the per-call overhead, you genuinely need a dynamic batch dim. This is the canonical use case — and the only one you should hit in retail trading.
You really do need variable sequence lengths. Rare in MT5. The cleaner alternative is padding to a fixed maximum and using a mask — see the variable-length pattern in the LSTM article.

// related

general

Export PyTorch to ONNX

specific

PyTorch LSTM — the seq_len trap

reference

Which opset to target

downstream

Memcpy nodes warning