Avoid reinstalling env templates if Git source has not changed

[itowlson]

Fixes #3672.

The adopted fix is to record against each template the repo and tag that it was installed from, assume that tags and branches are immutable, and update only if the actual tag or branch listed in the env file has changed from the one used for installation.

But: two considerations:

1. This will result in missed updates if the env owner moves a tag or branch in their templates repo. For this to work, env owners have to consider refs immutable, and if they want to change their templates, they have to update their env with a new tag. I am not sure if this is a safe assumption. We could have a Plan B to re-fetch periodically, but either this kick in too often (and Thorsten would remain frustrated), or it would be too infrequent to be useful. Feedback welcome.

2. This does not address the delay to re-snapshot the environment, which happens if the environment is unversioned and our snapshot is more than 1 hour old (basically if no unversioned env has been used in the last hour). This is trickier because we can have absolutely no way to tell if the upstream has changed, without a network round-trip. We could increase the recency window to reduce the incidence of these delays, but again that just makes it more and more likely that a user will hit the window. Although we do have the spin targets update command to mitigate that, so maybe it would be okay.

So: feedback and discussion first, so we don't do the wrong thing; but at the same time, it would be nice to get any fix we are happy with into 4.1. So no mixed messages then.

(cc @ThorstenHans who flagged the original issue)

[fibonacci1729]

Suggested change

	pub fn is_installed_from(&self, expected_repo: &str, expected_tag: Option<&str>) -> bool {
	pub fn is_installed_from_git_ref(&self, expected_repo: &str, expected_tag: Option<&str>) -> bool {

Would help disambiguate from is_from_source_repo above.

[fibonacci1729]

I think maybe we should bail out here when the source branch is None. Untagged envs will keep today's behavior. This effectively makes the "refs are immutable" an opt-in contract the env author signs by pinning rather than an assumption imposed on people who deliberately didn't pin. The only downside here is that untagged envs get no speedup at all.

[fibonacci1729]

AFAICT spin targets update only refreshes the environment catalogue. To add an escape hatch when a ref moves (for whatever reason) I think we should either:

1. Refresh the installed env templates stores during target update
2. Add a --refresh-templates/--force flag to spin new -E
3. Or a TTL on the skip to bound staleness

[itowlson]

Ooh I like spin targets update refreshing the templates.

[itowlson]

Okay spin targets update updating the templates is going to need a wee bit more thought. update updates the entire catalogue which contains all envs, so we need to figure out which envs the user is actually using and update only those templates, which is new - we don't currently have a concept of "the user is using an environment," it is all on-demand. I think we can get away with "do we have cached templates from this env" but I would like to defer it to another PR so I can think about it separately.

[itowlson]

Updated with your suggestions @fibonacci1729 - please take a look - thanks!

[ThorstenHans]

What about retrofitting this into spin templates upgrade? From a user perspective, a template contributed by a target-environment is no different from a "regular" Spin template.

• If we know http-yolo is a template that's provided by the yolo environment, I would expect spin templates list to visualize this as well
• When running spin templates upgrade users should be able to select an environment and trigger upgrade of all templates contributed by the environment

Handing off the template management to those commands grouped under spin templates would allow spin new to remain the way it was prior to Spin 4.x. and have the lifecycle for templates decoupled from the actual scaffolding.

Installing templates defined by an environment should IMO be possible through two routes:

• spin templates install -E yolo: To install all templates defined by an environment called yolo
• spin targets use yolo: This would set the yolo environment as my personal default (e.g, SPIN_DEFAULT_TARGET_ENVIRONMENT=yolo) and install templates for that environment after user confirmation.

ℹ️: 2nd approach is just an idea that came into my mind