fix: convert_dcp_to_hf.py correctly loads saved tokenizer from DCP if detected

[trias702]

What does this PR do ?

Fixed convert_dcp_to_hf.py to load the tokenizer directory when detected.

Usage

• You can potentially add a usage example below

uv run python examples/converters/convert_dcp_to_hf.py --config /path/to/config.yaml --dcp-ckpt-path /path/to/policy/weights --hf-ckpt-path /path/to/hf

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you run the unit tests and functional tests locally? Visit our Testing Guide for how to run tests
• [] Did you add or update any necessary documentation? Visit our Document Development Guide for how to write, build and test the docs.

Additional Information

• ...

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[yuki-97]

@RayenTian could you help to take a review?

[RayenTian]

@trias702 Thanks a lot for adding this! lgtm!
CC: @yuki-97

[yuki-97]

hi @trias702 , thanks for helping fix this! left some minor comments.

[yuki-97]

/ok to test 698757c

[yuki-97]

hi @trias702 , there's a lint check fail, could you help to fix?

[trias702]

hi @trias702 , there's a lint check fail, could you help to fix?

Sorry, I don't understand why it's failing. I'm afraid I don't know much about linting or how it works.

[RayenTian]

hi @trias702 , there's a lint check fail, could you help to fix?

Sorry, I don't understand why it's failing. I'm afraid I don't know much about linting or how it works.

Hi, @trias702. Lint check is a static code analysis tool that automatically scans your code for style issues, syntax problems, potential bugs, and violations of the project's coding standards.
You can refer to this doc to install pre-commit. Next time when you commit, the lint check will be triggered locally. Simply re-add the automatically fixed files and commit again. If any errors remain, the command output will show the specific files and rules that require manual adjustment.

Alternatively, if you prefer not to install pre-commit, you can run pre-commit run --all-files directly in the nemorl root workspace. This will also trigger the nemorl lint check and auto fix.

Feel free to let us know if you need help with any particular failure.

[yuki-97]

/ok to test 2382747

[yuki-97]

/ok to test 00c1c10