Support type checking with TY

docs/changes/newsfragments/8360.underthehood

@@ -0,0 +1,9 @@
+:class:`.Parameter` no longer replaces its own ``get_raw``/``set_raw`` methods
+with the implementation generated from ``get_cmd``/``set_cmd``. The generated
+implementation is stored on the parameter instead, and ``get_raw``/``set_raw``
+are now regular methods that dispatch to it. Assigning over the methods made
+static type checkers infer ``get_raw``/``set_raw`` to be instance attributes of
+:class:`.Parameter`, which in turn made every subclass implementing them as
+regular methods an invalid override. There is no change in behaviour; note only
+that ``parameter.get_raw`` is now always a bound method rather than, depending
+on the arguments, a ``Command`` instance.

docs/changes/newsfragments/8361.underthehood

@@ -0,0 +1,9 @@
+The ``TParameter`` type variable used by :meth:`.InstrumentBase.add_parameter`
+now defaults to ``Parameter[Any, Any]`` rather than to a bare ``Parameter``.
+When ``add_parameter`` is called without an explicit ``parameter_class`` the
+returned parameter is bound to the instrument it is added to, so the previous
+default (which expands to ``Parameter[Any, InstrumentBase | None]``) wrongly
+claimed that the instrument was ``InstrumentBase | None``. This made the result
+unassignable to the ``Parameter[SomeType, Self]`` annotations that drivers use.
+Code that relies on the inferred type of an unannotated
+``instrument.add_parameter("name")`` will now see ``Parameter[Any, Any]``.

docs/changes/newsfragments/8362.underthehood

@@ -0,0 +1,5 @@
+The legacy dataset importer now raises a clear :class:`ValueError` when a
+setpoint array has no ``array_id``, instead of passing ``None`` on to
+``add_result`` where a parameter name is expected. This was found by annotating
+``DataArray`` in ``qcodes_loop``, which also removes the need for a ``pyright``
+suppression on the array shape.

docs/changes/newsfragments/8363.underthehood

@@ -0,0 +1,6 @@
+The duck typed scale and offset conversions in ``ParameterBase`` have been
+factored out into dedicated module level helpers. The conversions assume the
+data type is numeric and rely on catching ``TypeError``, which does not fit the
+generic parameter data type. Giving them an explicit boundary lets the rest of
+the class stay properly typed and removes twelve type checker suppressions.
+There is no change in behaviour.

docs/changes/newsfragments/8364.underthehood

@@ -0,0 +1,4 @@
+``CombinedParameter.parameter`` is now a small dataclass holding the ``name``,
+``full_name``, ``label`` and ``unit`` of the combined parameter, replacing a
+lambda that had those attributes attached to it. It remains callable, and
+calling it returns ``None`` as before.

docs/changes/newsfragments/8365.underthehood

@@ -0,0 +1,6 @@
+Subclasses of ``ParameterBase`` that forward ``**kwargs`` on to their super
+class now carry a ``ty: ignore[invalid-argument-type]``. When a generic TypedDict
+declares a PEP 696 default for a type parameter, ty computes the upper bound of
+the synthesized ``Self`` as the default specialization, so every other
+specialization is rejected by the members that bind ``Self``, including expanding
+with ``**``. The reason is documented on ``ParameterBaseKWArgs``.

docs/changes/newsfragments/8366.underthehood

@@ -0,0 +1,5 @@
+The Infiniium driver now uses ``cast`` where it narrows ``root_instrument`` and
+``instrument`` to the concrete driver classes, and where ``pyvisa`` types the
+return of ``read_binary_values``/``query_binary_values`` as a ``Sequence[float]``
+regardless of the requested ``container``. This replaces five type checker
+suppressions and has no effect at runtime.

docs/changes/newsfragments/8367.underthehood

@@ -0,0 +1,5 @@
+``DataSet._finalize_res_dict_standalones`` now appends to its result list
+directly instead of building intermediate lists. The intermediate lists took
+their element type from the branch that built them rather than from the
+declaration, and ``dict`` is invariant in its value type, so the result was not
+assignable back. There is no change in behaviour.

docs/changes/newsfragments/8368.underthehood

@@ -0,0 +1,4 @@
+The Alazar DLL wrapper no longer assumes that the callable handed to a ctypes
+``errcheck`` has a ``__name__``, which a plain ``Callable`` does not guarantee.
+The error message falls back to the repr of the callable instead. This only
+affects the text of an error that should not occur in practice.

docs/changes/newsfragments/8370.underthehood

@@ -0,0 +1,5 @@
+``numpy_ints`` and ``numpy_floats`` in ``qcodes.utils.types`` are now annotated
+as tuples of ``type[np.integer]`` and ``type[np.floating]`` rather than of bare
+``type``. As a consequence ``_adapt_float``, which is registered as a sqlite
+adapter for the numpy float types as well as for ``float``, now declares that it
+accepts ``np.floating`` too. Its behaviour is unchanged.

docs/changes/newsfragments/8371.underthehood

@@ -0,0 +1,5 @@
+The ``__getattr__`` that provides backwards-compatible access to the old flat
+parameter names on the Tektronix AWG5014 now names its parameter ``key``,
+matching ``DelegateAttributes.__getattr__`` which it overrides and delegates to.
+Python only ever calls ``__getattr__`` positionally, so this has no effect at
+runtime.

docs/changes/newsfragments/8373.underthehood

@@ -0,0 +1,5 @@
+``json_template_linear`` and ``json_template_heatmap`` in
+``qcodes.dataset.json_exporter`` are now annotated as ``dict[str, Any]``. They
+are templates for a JSON document, so their values are deliberately
+heterogeneous, and without the annotation the inferred value type made indexing
+into them an error for callers filling the template in.

docs/changes/newsfragments/8374.improved

@@ -0,0 +1,6 @@
+The ``callback`` argument of :meth:`.DataSet.subscribe` is now typed as
+``Callable[..., None]``. The previous annotation described a callback taking
+exactly three arguments, which contradicted ``callback_kwargs``: those are bound
+onto the callback with ``functools.partial``, so a callback using them takes
+further arguments. ``_Subscriber``, which ``subscribe`` forwards to, already
+typed it this way.

docs/changes/newsfragments/8375.improved

@@ -0,0 +1,7 @@
+``Keysight34980A.module`` is now a ``dict`` of
+``Keysight34980ASwitchMatrixSubModule`` rather than one built with
+``dict.fromkeys``, whose values were typed as possibly ``None``. ``scan_slots``
+puts an entry in for every slot, either the driver for the installed module or a
+generic submodule, so the values were never ``None`` once the instrument was
+constructed. Code using ``instrument.module[slot]`` no longer has to account for
+a ``None`` that cannot occur.

docs/changes/newsfragments/8376.improved

@@ -0,0 +1,6 @@
+The return type of :func:`.parse_awg_file` has been corrected. The waveform and
+marker entries were declared as lists of dicts, but the parser returns the arrays
+from inside those dicts, and the loop counts and sequencing values were declared
+as possibly ``str`` when they are always ``int``. The type now matches the call
+signature of :meth:`.TektronixAWG5014.make_send_and_load_awg_file`, which the
+docstring already promised and which is how the function is meant to be used.

docs/changes/newsfragments/8377.underthehood

@@ -0,0 +1,3 @@
+``ty`` now also type checks the example notebooks in ``docs``. Unlike mypy and
+pyright it understands Jupyter notebooks, so this is coverage that the other two
+checkers do not provide.

docs/changes/newsfragments/8381.improved

@@ -0,0 +1,6 @@
+The Keysight B1500 example notebook called
+``b1500.run_iv_staircase_sweep.measurement_status()`` in the phase compensation
+section. ``IVSweepMeasurement`` has no such method, so the cell raised
+``AttributeError``. The surrounding text asks for all channel outputs to be
+enabled before performing phase compensation, so the cell now calls
+``b1500.enable_channels()``.

docs/changes/newsfragments/8382.improved

@@ -0,0 +1,7 @@
+``KeysightE4980AMeasurementPair`` now declares a ``__getattr__`` for type
+checkers. The two measured values are exposed as attributes named after the
+``names`` of the measurement function, for example ``capacitance`` for ``CPD``
+and ``inductance`` for ``LPD``, so which attributes exist is only known at
+runtime. The declaration lets this documented usage be written in typed code. It
+is not defined at runtime, so accessing an attribute that the current
+measurement function does not provide still raises the usual ``AttributeError``.

docs/changes/newsfragments/8384.improved

@@ -0,0 +1,4 @@
+The Stanford SR86x buffered readout example notebook now packs the waveforms for
+:meth:`.TektronixAWG70000Base.makeSEQXFile` in lists rather than wrapping them in
+further numpy arrays, which is the shape the method documents. The two forms
+behave the same at runtime.

docs/changes/newsfragments/8385.improved

@@ -0,0 +1,6 @@
+``connect_paths``, ``disconnect_paths`` and ``to_channel_list`` on the Keysight
+B220X switch matrix drivers now accept any iterable of paths rather than only a
+``Sequence``. They iterate the paths once and do not index them, so passing a
+set, as the example notebook does, is fine. Note that the order in which the
+paths appear in the channel list then follows the iteration order of the
+argument.

docs/changes/newsfragments/8386.improved

@@ -0,0 +1,6 @@
+The path arguments of ``connect_paths``, ``disconnect_paths``, ``are_closed``,
+``are_open`` and ``to_channel_list`` on the Keysight 34980A switch matrix
+submodules are now typed as a ``Collection`` rather than a ``list``, so a set or
+a tuple of paths is accepted as well. A ``Collection`` rather than an
+``Iterable`` because these methods walk the paths twice, once to validate them
+and once to build the channel list, which a one shot iterator would not survive.

docs/changes/newsfragments/8387.improved

@@ -0,0 +1,5 @@
+The live temperature plot helper in the two Lakeshore example notebooks keeps
+the appended y data in a local variable instead of reading it back with
+``Line2D.get_ydata``. The return of ``get_ydata`` is typed as ``ArrayLike``,
+which is not necessarily sized, so taking its length was a type error. This also
+avoids reading the data back from the line on every iteration.

docs/changes/newsfragments/8388.improved

@@ -0,0 +1,5 @@
+The ``colorbars`` argument of :func:`.plot_dataset` and :func:`.plot_by_id` now
+accepts a sequence that may contain ``None``. Both functions return a list of
+colorbars in which the entries for 1D plots are ``None``, so passing the result
+back in, which is how you plot into the same axes again, did not match the
+declared argument type. A sequence of colorbars is still accepted.

docs/changes/newsfragments/8389.improved

@@ -0,0 +1,3 @@
+The offline plotting tutorial now checks the optional values it gets back from
+:func:`.plot_dataset` before using them, and asks for the root figure when
+saving. ``Axes.figure`` may be a ``SubFigure``, which has no ``savefig``.

docs/changes/newsfragments/8390.improved

@@ -0,0 +1,5 @@
+``snapshot_raw`` is now part of :class:`.DataSetProtocol` and is available on
+:class:`.DataSetInMem` as well as on :class:`.DataSet`. It is documented as the
+way to get the snapshot of a run as a JSON string, and is used as such in the
+example notebooks, but it was only declared on one of the two dataset classes,
+so reading it from a dataset returned by a measurement did not type check.

docs/changes/newsfragments/8391.improved

@@ -0,0 +1,3 @@
+The snapshot example notebook now checks that the snapshots it reads back from
+the datasets are present before using them. A run only has a snapshot if one was
+recorded, so both ``snapshot`` and ``snapshot_raw`` are optional.

docs/examples/DataSet/Offline Plotting Tutorial.ipynb

@@ -477,6 +477,8 @@
    "outputs": [],
    "source": [
     "colorbar = colorbars[0]\n",
+    "# 2D plots have a colorbar, 1D plots do not, so the entries are optional\n",
+    "assert colorbar is not None\n",
     "colorbar.set_label(\"Correct science label\")"
    ]
   },
@@ -939,9 +941,11 @@
    "source": [
     "%%time\n",
     "axeslist, _ = plot_dataset(dataset)\n",
-    "axeslist[0].figure.savefig(\n",
-    "    Path.cwd().parent / \"example_output\" / f\"test_plot_dataset_{dataid}.pdf\"\n",
-    ")"
+    "# Axes.figure may be a SubFigure, which cannot be saved, so ask for the\n",
+    "# root figure\n",
+    "figure = axeslist[0].get_figure(root=True)\n",
+    "assert figure is not None\n",
+    "figure.savefig(Path.cwd().parent / \"example_output\" / f\"test_plot_dataset_{dataid}.pdf\")"
    ]
   },
   {
@@ -971,9 +975,11 @@
    "source": [
     "%%time\n",
     "axeslist, _ = plot_dataset(dataset, rasterized=False)\n",
-    "axeslist[0].figure.savefig(\n",
-    "    Path.cwd().parent / \"example_output\" / f\"test_plot_dataset_{dataid}.pdf\"\n",
-    ")"
+    "# Axes.figure may be a SubFigure, which cannot be saved, so ask for the\n",
+    "# root figure\n",
+    "figure = axeslist[0].get_figure(root=True)\n",
+    "assert figure is not None\n",
+    "figure.savefig(Path.cwd().parent / \"example_output\" / f\"test_plot_dataset_{dataid}.pdf\")"
    ]
   }
  ],

docs/examples/DataSet/Working with snapshots.ipynb

@@ -593,7 +593,9 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "snapshot_of_run = dataset.snapshot"
+    "snapshot_of_run = dataset.snapshot\n",
+    "# a run only has a snapshot if one was recorded, this one has\n",
+    "assert snapshot_of_run is not None"
    ]
   },
   {
@@ -602,7 +604,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "snapshot_of_run_in_json_format = dataset.snapshot_raw"
+    "snapshot_of_run_in_json_format = dataset.snapshot_raw\n",
+    "assert snapshot_of_run_in_json_format is not None"
    ]
   },
   {
@@ -881,7 +884,10 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "diff_param_values(dataset.snapshot, bad_dataset.snapshot).changed"
+    "snapshot_of_bad_run = bad_dataset.snapshot\n",
+    "assert snapshot_of_bad_run is not None\n",
+    "\n",
+    "diff_param_values(snapshot_of_run, snapshot_of_bad_run).changed"
    ]
   },
   {

docs/examples/driver_examples/Qcodes example with Keysight 34980A Switch Mainframe and Modules.ipynb

@@ -428,9 +428,10 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "switch_matrix.module[\n",
-    "    2\n",
-    "]._is_locked = True  # DO NOT perform this action in real situation"
+    "# DO NOT perform this action in a real situation. ``_is_locked`` is defined\n",
+    "# on the 34934A driver rather than on the shared submodule base class that\n",
+    "# ``module`` is typed as, hence the suppression.\n",
+    "switch_matrix.module[2]._is_locked = True  # ty: ignore[unresolved-attribute]"
    ]
   },
   {

docs/examples/driver_examples/Qcodes example with Keysight B1500 Parameter Analyzer.ipynb

@@ -288,7 +288,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "b1500.by_kind[\"SMU\"]"
+    "b1500.by_kind[constants.ModuleKind.SMU]"
    ]
   },
   {
@@ -331,8 +331,9 @@
     "# Selecting a module by channel number using the Enum\n",
     "m1 = b1500.by_channel[constants.ChNr.SLOT_01_CH1]\n",
     "\n",
-    "# Without enum\n",
-    "m2 = b1500.by_channel[1]\n",
+    "# Without enum. ChNr is an IntEnum, so a plain int is the same key at\n",
+    "# runtime, but the dict is typed as taking ChNr.\n",
+    "m2 = b1500.by_channel[1]  # ty: ignore[invalid-argument-type]\n",
     "\n",
     "# And we assert that we selected the same module:\n",
     "assert m1 is m2"
@@ -1118,7 +1119,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "b1500.run_iv_staircase_sweep.measurement_status()"
+    "# enable all channel outputs\n",
+    "b1500.enable_channels()"
    ]
   },
   {

docs/examples/driver_examples/Qcodes example with Lakeshore 325.ipynb

@@ -517,8 +517,9 @@
     "        text.value = f\"T = {channel_to_read.temperature()}\"\n",
     "\n",
     "        # Add new point to the data that is being plotted\n",
-    "        line.set_ydata(numpy.append(line.get_ydata(), channel_to_read.temperature()))\n",
-    "        line.set_xdata(numpy.arange(0, len(line.get_ydata()), 1) * read_period)\n",
+    "        ydata = numpy.append(line.get_ydata(), channel_to_read.temperature())\n",
+    "        line.set_ydata(ydata)\n",
+    "        line.set_xdata(numpy.arange(0, len(ydata), 1) * read_period)\n",
     "\n",
     "        ax.relim()  # Recalculate limits\n",
     "        ax.autoscale_view(True, True, True)  # Autoscale\n",

docs/examples/driver_examples/Qcodes example with Lakeshore 336 or 372 - Bluefors T control.ipynb

@@ -508,8 +508,9 @@
     "        text.value = f\"T = {channel_to_read.temperature()}\"\n",
     "\n",
     "        # Add new point to the data that is being plotted\n",
-    "        line.set_ydata(numpy.append(line.get_ydata(), channel_to_read.temperature()))\n",
-    "        line.set_xdata(numpy.arange(0, len(line.get_ydata()), 1) * read_period)\n",
+    "        ydata = numpy.append(line.get_ydata(), channel_to_read.temperature())\n",
+    "        line.set_ydata(ydata)\n",
+    "        line.set_xdata(numpy.arange(0, len(ydata), 1) * read_period)\n",
     "\n",
     "        ax.relim()  # Recalculate limits\n",
     "        ax.autoscale_view(True, True, True)  # Autoscale\n",

docs/examples/driver_examples/Qcodes example with Stanford SR86x with buffered readout.ipynb

@@ -683,8 +683,10 @@
     "# (3000 samples for 3000 S/s sample rate)\n",
     "waveform_ch1[1, :-1500] = 1  # falling from 1 to 0 (a.u.),\n",
     "# at 0.5s after the start of the waveform\n",
-    "elements = numpy.array([waveform_ch1])  # we only have one element in the sequence\n",
-    "waveforms = numpy.array([elements])  # we will use only 1 channel\n",
+    "# makeSEQXFile takes the waveform arrays packed in lists, per channel and\n",
+    "# then per element, rather than in a further numpy array\n",
+    "elements = [waveform_ch1]  # we only have one element in the sequence\n",
+    "waveforms = [elements]  # we will use only 1 channel\n",
     "\n",
     "# Create a sequence file from the \"waveform\" array\n",
     "seq_name = \"single_trigger_marker_1\"\n",
@@ -929,8 +931,10 @@
     "    n_trigger_pulses,\n",
     ")  # falling from 1 to 0 (a.u.) every 0.01s after the start of the waveform\n",
     "\n",
-    "elements = numpy.array([waveform_ch1])  # we only have one element in the sequence\n",
-    "waveforms = numpy.array([elements])  # we will use only 1 channel\n",
+    "# makeSEQXFile takes the waveform arrays packed in lists, per channel and\n",
+    "# then per element, rather than in a further numpy array\n",
+    "elements = [waveform_ch1]  # we only have one element in the sequence\n",
+    "waveforms = [elements]  # we will use only 1 channel\n",
     "\n",
     "# Create a sequence file from the \"waveform\" array\n",
     "seq_name = \"single_trigger_marker_1\"\n",

docs/examples/driver_examples/Qcodes example with Tektronix AWG5014C.ipynb

@@ -100,19 +100,23 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "# ``instrument.parameters`` is a dict of ``ParameterBase``, and not every\n",
+    "# parameter type carries a ``label``: ``MultiParameter`` has ``labels`` instead.\n",
+    "# Fall back to an empty string so this works for any parameter.\n",
+    "\n",
     "# Top-level parameters\n",
     "for name in sorted(awg1.parameters):\n",
-    "    print(name, \": \", awg1.parameters[name].label)\n",
+    "    print(name, \": \", getattr(awg1.parameters[name], \"label\", \"\"))\n",
     "\n",
     "# Channel parameters (e.g. ch1)\n",
     "print(\"\\nChannel 1 parameters:\")\n",
     "for name in sorted(awg1.ch1.parameters):\n",
-    "    print(f\"  ch1.{name}: \", awg1.ch1.parameters[name].label)\n",
+    "    print(f\"  ch1.{name}: \", getattr(awg1.ch1.parameters[name], \"label\", \"\"))\n",
     "\n",
     "# Marker parameters (e.g. ch1.m1)\n",
     "print(\"\\nChannel 1 Marker 1 parameters:\")\n",
     "for name in sorted(awg1.ch1.m1.parameters):\n",
-    "    print(f\"  ch1.m1.{name}: \", awg1.ch1.m1.parameters[name].label)"
+    "    print(f\"  ch1.m1.{name}: \", getattr(awg1.ch1.m1.parameters[name], \"label\", \"\"))"
    ]
   },
   {