Use args.class_map for labels in inference.py

[JosuaRieder]

Addresses #1800

[rwightman]

@JosuaRieder thanks for the PR ... might have gathered from my comment in the bug related to class map foder filtering though it's relatred, the class map should be treated separately from the label information. class map is used for filtering, it can actually remove class labels from a dataset while remapping to a different classifier layout.

Assigning descriptions, label names should be separate.

[JosuaRieder]

Assigning descriptions, label names should be separate.

In that case, I may have misinterpreted what the intended use case of --class-map is.

What is the intended way to train an / infer with an efficientnet with custom classes?
I used --class-map class_map.txt --model efficientnet_b7 --num-classes 8 and it generally worked, both for training from scratch as well as finetuning.