Updates to website issue

[Will-Cooper]

Looking across several issues, updating versions and some documentation.
Should also close #180 with some logic changes, since underlying package updates.
Makes sqllite file downloadable, adds a link to the json.
Spectra also now downloadable from raw query.

[kelle]

This looks like progress -- great work! But I think review should really come from @dr-rodriguez .

Please keep in mind that SIMPLE should be using a database.toml file which specifies the reference tables. It's not doing that yet, but will be in the next month or so. (@ying2212 is working on it.)

Reference issue: SIMPLE-AstroDB/SIMPLE-db#672

[dr-rodriguez]

Looks like there are 4 functions added in utils.py, which has a test module. We should add tests for these 4 functions, particularly since they focus mainly on manipulating strings.

[dr-rodriguez]

This will break with the recent changes because the init file has been removed in favor of a database.toml file

[dr-rodriguez]

Change this instead to:

import tomllib
with open("database.toml", "rb") as f:
    settings = tomllib.load(f)
    REFERENCE_TABLES = settings['lookup_tables']

[dr-rodriguez]

Instead of importing simple, we want to update the cronjob so that it pulls the database.toml from from github and then imports it:

from astrodb_utils.loaders import DatabaseSettings
db_settings=DatabaseSettings(settings_file="database.toml")
REFERENCE_TABLES = db_settings.lookup_tables

[dr-rodriguez]

We don't want to use astrodb_utils- it requires too much of the setup and fails if we only have the database.toml file. Instead we'll use the built-in library tomlib

[dr-rodriguez]

The cronjob.sh needs to read:

cd ~/ROOT
wget -O SIMPLE.sqlite https://raw.githubusercontent.com/SIMPLE-AstroDB/SIMPLE-binary/main/SIMPLE.sqlite
wget https://raw.githubusercontent.com/SIMPLE-AstroDB/SIMPLE-db/refs/heads/main/database.toml
git pull

I recommend including this in this PR as per #212

[dr-rodriguez]

I tested this in a codespace but don't have permissions to push to this branch, let us know how you want to proceed with these @Will-Cooper

[Will-Cooper]

I've made the cronjob changes and included it in git. Database.toml wasn't as clear, as we also need a data folder as well as schema.yaml in the same directory too - I've included some logic to do so

[dr-rodriguez]

I've made the cronjob changes and included it in git. Database.toml wasn't as clear, as we also need a data folder as well as schema.yaml in the same directory too - I've included some logic to do so

We ran into this issue while testing and opted instead to use tomlib:

import tomllib
with open("database.toml", "rb") as f:
    settings = tomllib.load(f)
    REFERENCE_TABLES = settings['lookup_tables']

This uses the database.toml file without the extra requirements of the data folder or schema file.

[dr-rodriguez]

Merge Will-Cooper#9 first

[Will-Cooper]

Merge Will-Cooper#9 first

Thanks for doing that, sorry I didn't get around to it!

[dr-rodriguez]

Change this instead to:

import tomllib
with open("database.toml", "rb") as f:
    settings = tomllib.load(f)
    REFERENCE_TABLES = settings['lookup_tables']

[dr-rodriguez]

We don't want to use astrodb_utils- it requires too much of the setup and fails if we only have the database.toml file. Instead we'll use the built-in library tomlib