[FLINK-39240][runtime] Jobs share application-level jar when possible

[eemario]

Only the last commit is part of this pull request; the rest originate from the dependency.

What is the purpose of the change

This pull request ensures that jobs within an application can reuse the application-level user JAR (when provided), reducing redundant JAR upload overhead across multiple jobs in the same application.

Brief change log

• Record the application-level user JAR’s name and blobKey at the application level and propagate them to the StreamGraph, enabling the system to skip redundant JAR uploads.
• Allow jobs to fall back to the application-level JAR in BlobLibraryCacheManager when no job-specific JAR is present.
• Modify PermanentBlobCache to support application-level resource cleanup.

Verifying this change

This change added tests and can be verified as follows:

• Added tests for StreamGraph to validate the skipped jar uploads
• Added tests for BlobLibraryCacheManager to validate that jobs can reuse application-level blobs and the blobs can be properly cleaned up
• Updated tests for SlotManager to validate correct propagation of job–application association messages
• Manually verified the change by running a standalone cluster with zookeeper, submitting an application and confirming its job can use the application-level blob

Does this pull request potentially affect one of the following parts:

• Dependencies (does it add or upgrade a dependency): (no)
• The public API, i.e., is any changed class annotated with @Public(Evolving): (no)
• The serializers: (no)
• The runtime per-record code paths (performance sensitive): (no)
• Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (no)
• The S3 file system connector: (no)

Documentation

• Does this pull request introduce a new feature? (no)
• If yes, how is the feature documented? (not applicable)

[flinkbot]

CI report:

• abee15e Azure: SUCCESS

Bot commands

The @flinkbot bot supports the following commands:

• @flinkbot run azure re-run the last Azure build