Don't dedupe span metadata by id() (fix flaky test_collections_metadata)

[crusaderky]

Span.add_metadata used a class-level id-based de-duplication system that was tracking the id of already seen metadata. However, id is volatile and can be reused by the CPython interpreter as soon as the original object with that id is dereferenced. This caused legitimate new metadata to be occasionally dropped.
This deduplication also grew unboundedly over the lifetime of the scheduler.

The only deduplication actually needed is within a single observe_tasks call, where all tasks of one graph submission share the same metadata object.

[github-actions]

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

    19 files   -     21      19 suites   - 21   16h 34m 20s ⏱️ + 1h 33m 58s
 5 000 tests +   846   5 000 ✅ +1 025       0 💤  -   178  0 ❌  - 1 
95 000 runs  +14 159  85 000 ✅ +8 397  10 000 💤 +5 763  0 ❌  - 1 

Results for commit 6fa439b. ± Comparison against base commit ace5560.

This pull request removes 4154 and adds 5000 tests. Note that renamed tests count towards both.

distributed.cli.tests.test_dask_scheduler ‑ test_dashboard
distributed.cli.tests.test_dask_scheduler ‑ test_dashboard_allowlist
distributed.cli.tests.test_dask_scheduler ‑ test_dashboard_non_standard_ports
distributed.cli.tests.test_dask_scheduler ‑ test_dashboard_port_zero
distributed.cli.tests.test_dask_scheduler ‑ test_defaults
distributed.cli.tests.test_dask_scheduler ‑ test_hostport
distributed.cli.tests.test_dask_scheduler ‑ test_idle_timeout
distributed.cli.tests.test_dask_scheduler ‑ test_interface
distributed.cli.tests.test_dask_scheduler ‑ test_multiple_protocols
distributed.cli.tests.test_dask_scheduler ‑ test_multiple_workers
…

distributed.tests.test_spans ‑ test_collections_metadata[1-5000]
distributed.tests.test_spans ‑ test_collections_metadata[10-5000]
distributed.tests.test_spans ‑ test_collections_metadata[100-5000]
distributed.tests.test_spans ‑ test_collections_metadata[1000-5000]
distributed.tests.test_spans ‑ test_collections_metadata[1001-5000]
distributed.tests.test_spans ‑ test_collections_metadata[1002-5000]
distributed.tests.test_spans ‑ test_collections_metadata[1003-5000]
distributed.tests.test_spans ‑ test_collections_metadata[1004-5000]
distributed.tests.test_spans ‑ test_collections_metadata[1005-5000]
distributed.tests.test_spans ‑ test_collections_metadata[1006-5000]
…

♻️ This comment has been updated with latest results.