cluster-spec-sheet: recreate the region for envd CPU changes

[aljoscha]

Motivation

The envd CPU sweeps have been failing against staging with failed to apply Cloud CPU override for environmentd via 'mz region': never got correct result ... FATAL: upstream server not available (SQL-596).

Nothing is wrong with the environment. mz region enable --environmentd-cpu-allocation on a region that still has durable state is a 0dt
rollout, so the new envd comes up read-only and only promotes after the
caught-up stability period:

06:25:01  cluster is caught up but not yet stable for the required period
          cluster_id=s1/s2/s3/u2  all_healthy=true  restart_total=0
          stable_for_ms=290025  required_period_ms=600000

Enabled at 06:19:48, caught up and healthy by 06:20:03, would have promoted at
~06:30. We wait 300s and give up at 06:25, and nothing serves SQL in between
because the soft disable took the old envd away. The
with_0dt_caught_up_check_stability_period=0s we pass via
--environmentd-extra-arg isn't in effect there (600000 is the compiled-in
default, and the region is hard-disabled at job start so there's no stale ALTER SYSTEM SET). That's being followed up separately, this PR makes the test not
depend on it.

Description

Hard-disable instead of soft-disable: with no state left there's no predecessor
generation to catch up with, so the new envd promotes as soon as it has booted
and the gate never applies. The sweep re-runs prepare per scale point, since
the wipe takes its state with it (one table, one insert, both envd workloads
have an empty setup()). Teardown's reset to the default allocation goes
through the same recreate, so it can't leave the region in read-only limbo for a
following scenario.

Readiness gets an honest check, which is what let this hide. After the recreate
we assert the region has no user tables, which can only hold for a fresh region
and so rules out a leftover envd, and wait_for_envd now wants several
consecutive probes instead of one, reports how long it waited, and allows 600s
instead of 300s. Neither mz region enable (which only checks that something
answers on the SQL port) nor a single query can tell old envd from new, and
before this change the wait was returning one to two seconds after the
re-enable, which is well inside a real bring-up.

The second commit retries the disable/enable pair, because a sweep makes a dozen
region API calls and staging returns the occasional 502 that the CLI does not
retry itself.

Verification

Test-only. Ran on staging in spec-sheet build 29, where
qps_envd_strong_scaling passed: all six scale points plus the teardown reset,
each region recreate serving SQL after 6-9s, roughly two minutes per point end
to end. That scenario has been failing since build 27. Measured QPS matches
build 26, the last green run, within a few percent at every point but the
32-core one, which is a plateau in the benchmark unrelated to this change, see
SQL-596 for that.

Closes SQL-596

[def-]

Oh well, sounds expensive too!