Symbolic shape inferer

[justinchuby]

I created https://github.com/justinchuby/onnx-shape-inference with good data propagation. Combined with microsoft/onnxscript#2807 we are able to fully optimize this model which has a bunch of redundant shape ops https://huggingface.co/justinchuby/BirdNET-onnx/blob/main/model.onnx.

I tested with onnxslim and it seems some shapes can be better inferred. I wonder if there is interest in adopting the shape inference project?

Rightly I am hosting it as a personal project, not as part of onnx-ir because it is too large to be properly reviewed (code I guided AI to create).

[inisis]

Great! It's very happy to see onnx supports this feature and I believe it should be part of onnx's foundamental infra. I'm curious that when doing torch.export, there should already have some symbolic shape, which you might be already working on, it should be the same except for the ir.

[inisis]

I can reporduce your result, that's really impressive. but when I further optimized with onnxslim, there's still room for optimization.

I believe that there should be an iteration.

[inisis]

and I find it's more like a constant folding issuse other than symbolic shape inference. onnxslim currently can not catch the info that split output is constant and that is directly feed to reshape node, and all these can be folded.

[justinchuby]

The shape inferencer is able to do partial data propagation and store the constant shape results for reshape. I can store it in the metadata if that’s helpful.

[inisis]

I will add an option to adopting the shape inference project, but will still improve my own implementation, I still enjoy it ❤️.

[inisis]

The shape inferencer is able to do partial data propagation and store the constant shape results for reshape. I can store it in the metadata if that’s helpful.

This is something that I long want to do, this is not symbolic shape inference but value inference, so the shape inferencer will change the graph, and make reshape input a constant?

[justinchuby]

It won't change the graph, but it can attach a hint for downstream tools

[justinchuby]

You can now find this information in the "pkg.onnx_shape_inference.sym_data" metadata key in the ValueInfo obejcts.