Dotenv dryrun fromuri

[JunAishima]

Update toward the template repo:

• dotenv to bring in the Tiled API key, instead of storing in Prefect Blocks
• dry_run - enable just running through a workflow without writing to files or Tiled. good for initial testing
• from_uri - consolidate the instantiation of Tiled clients, and use from_uri which enables running the workflows outside of the NSLS-II infrastructure. Another building block towards running test workflows in CI

[Copilot]

Pull request overview

This PR moves Tiled access toward a from_uri-based approach and starts threading a Tiled API key through Prefect flows/tasks to enable running workflows outside NSLS-II profile-based infrastructure.

Changes:

• Added get_run() (Tiled from_uri) and updated workflows/exporters to fetch runs via this helper and accept an api_key parameter.
• Removed Tiled profile mounts/env from the Prefect Docker deployment configuration.
• Updated end-of-run workflow to pass api_key through to downstream exporters/logging.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
data_validation.py	Adds from_uri-based get_run() and updates validation flow to use it.
xrf_hdf5_exporter.py	Switches run loading to get_run() and adds api_key plumb-through.
xanes_exporter.py	Switches run loading to get_run() and adds api_key parameters across exporters.
logscan.py	Starts switching to get_run() and adds api_key parameters (but still has profile/Secret globals).
end_of_run_workflow.py	Uses get_run() and passes api_key to downstream flows.
prefect.yaml	Removes Tiled profile env/volume from the deployment container configuration.

Comments suppressed due to low confidence (3)

logscan.py:86

• logscan() accepts api_key but doesn't pass it into logscan_detailed(), so the nested call will run with api_key=None and may fail to authenticate against Tiled. Thread the api_key argument through (logscan_detailed(ref, api_key=api_key)).

def logscan(ref, api_key=None):
    logger = get_run_logger()
    logger.info("Start writing logfile...")
    logscan_detailed(ref)
    logger.info("Finish writing logfile.")

logscan.py:11

• This module still initializes Secret.load(...) and from_profile(...) at import time (and defines tiled_client_raw) even though the actual code path now uses get_run(...) instead. With the Prefect deployment no longer mounting/setting Tiled profiles, this import-time from_profile('nsls2', ...) is likely to fail before the flow runs. Remove these globals (and the Secret/from_profile imports) or migrate them to from_uri and only construct clients inside functions when needed.

from prefect.blocks.system import Secret
from tiled.client import from_profile

from data_validation import get_run


api_key = Secret.load("tiled-srx-api-key", _sync=True).get()
tiled_client = from_profile("nsls2", api_key=api_key)["srx"]
tiled_client_raw = tiled_client["raw"]

xanes_exporter.py:340

• xanes_exporter() accepts api_key and uses it to determine scan_type, but it does not pass api_key into xas_step_exporter() / xas_fly_exporter(). As a result those tasks will run with api_key=None and may fail when they call get_run(...). Pass api_key through to the selected exporter task.

def xanes_exporter(ref, api_key=None):
    logger = get_run_logger()
    logger.info("Start writing file with xanes_exporter...")

    # Get scan type
    scan_type = (
        get_run(ref, api_key=api_key).start.get("scan", {}).get("type", "unknown")
    )

    # Redirect to correction function - or pass
    if scan_type == "XAS_STEP":
        logger.info("Starting xanes step-scan exporter.")
        xas_step_exporter(ref)
        logger.info("Finished writing file with xanes step-scan exporter.")
    elif scan_type == "XAS_FLY":
        logger.info("Starting xanes fly-scan exporter.")
        xas_fly_exporter(ref)
        logger.info("Finished writing file with xanes fly-scan exporter.")

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

api_key is now optional, but this task later assigns it into os.environ['TILED_API_KEY']. If api_key is None, os.environ will raise a TypeError (env values must be strings) and the export will fail. Either require api_key (raise a clear error if missing) or guard the env assignment and/or fall back to reading the key from the environment.

[Copilot]

The @slack decorator wraps the flow with an inner end_of_run_workflow(stop_doc) that does not accept/forward api_key. With the flow signature now including api_key, calling the flow with api_key=... will raise TypeError: unexpected keyword argument. Update the decorator wrapper to accept *args, **kwargs and pass them through to func(*args, **kwargs) (and ideally use functools.wraps to preserve the signature).

[JunAishima]

fixed in ddfa92d