Skip to content

Implement micro service system + add and seperate services + development enviroment#5

Merged
LEEDASILVA merged 18 commits into
mainfrom
re-organize-system-to-handle-microservices
Apr 30, 2026
Merged

Implement micro service system + add and seperate services + development enviroment#5
LEEDASILVA merged 18 commits into
mainfrom
re-organize-system-to-handle-microservices

Conversation

@LEEDASILVA
Copy link
Copy Markdown
Collaborator

@LEEDASILVA LEEDASILVA commented Apr 11, 2026

We would have to talk about the database schema to make sure we are all in sync.

Index:

  • Architecture design for the services

  • DB Schema

  • Auth & Docs routes

But for now this PR focuses on:

Architecture design for the services

  1. Services
  • api
  • migrate
  • front
  • hasura/postgres

api

Service that will handle our REST API and other Jobs like:

  • Scripts for the onboarding (automatic onboarding)
  • Qr (QR code + label PDF generation)
  • cron jobs (SLA monitor, PM scheduler, nightly analytics, reports)
  • Analytics (Pre-aggregation jobs writing to equipment_analytics)
  • Notifications (Email, SMS, push notifications)
  • Authentication (JWT issue, refresh, magic-link)

front

Service responsible for everything related to UI/UX design + most of the client logic will be here

  • React Vite
  • PWA (mobile application)
  • Caddy will be located here as well

hasura & postgresql

Service responsible for the database, hasura will help us with role management + Graphql schema design (which is automatically generated 👍)

migrate

This is a short lived service it will run the migrations to hasura. You might ask "Why not use hasura automatic migration?", well did some research and hasura migration is specific to hasura. So if we were to migrate to another client it would break... So go-migrate allows us to always use the postgresql default format of migration.

So basically the schema is portable and not locked to hasura

For the other platform we use hasura and we have problems when we want to generate schemas and migrating data to other clients, so i just added it like that. If we want we can just remove this service and convert it normally to hasura (its not a big deal, simple refactor if needed)


DB Schema

This PR also contains the full db schema of the tables, that might need some review!


Auth & Docs routes

Implemented the /auth and the /docs routes

  • POST /auth/login, simple login check user credentials and return jwt token with session and cookies
  • POST /auth/refresh, refresh the jwt token
  • POST /auth/logout, logout fully the user, revoking their token
  • GET /auth/me, returns the currently authenticated user's profile
  • GET /auth/magic-link?token=<raw>, validates a magic-link token and issues a full session
  • POST /auth/magic-link, generates a magic-link token and returns it (TODO: but it still does not send it to the email)

RUN

docker compose -f docker-compose.dev.yml up --build --detach -> this would build the dev environment.

For now I would like to merge this one and then the prod we fix as we go. Basicaly we need to test caddy for production.

Comment thread hasura/migrations/default/1000000000001_companies/up.sql
Comment thread hasura/migrations/default/1000000000002_location_types/up.sql
Comment thread hasura/migrations/default/1000000000004_severity_levels/up.sql
Comment thread hasura/migrations/default/1000000000008_equipment/up.sql
Comment thread hasura/migrations/default/1000000000009_issues/up.sql
Comment thread hasura/migrations/default/1000000000010_maintenance_actions/up.sql
Comment thread hasura/migrations/default/1000000000013_preventive_schedules/up.sql
Comment thread hasura/migrations/default/1000000000014_equipment_analytics/up.sql
…mpose + hasura implementation with postgres + api to come
@LEEDASILVA LEEDASILVA force-pushed the re-organize-system-to-handle-microservices branch from 026858c to 53a8e76 Compare April 11, 2026 21:18
Comment thread scripts/init-postgres.sql
@LEEDASILVA LEEDASILVA marked this pull request as ready for review April 13, 2026 22:02
…xed vite config to handle /api directly on the localhost fron route + fix docs route to display api documentation
Comment thread scripts/init-postgres.sql
@HarryVasanth
Copy link
Copy Markdown
Member

Frontend:
image

Hasura console:
image

Logs:

Frontend - operafix_front (of-dev-front:latest):

> operafix@1.0.0 dev

> vite --host 0.0.0.0



  VITE v6.4.2  ready in 355 ms


  ➜  Local:   http://localhost:5173/

  ➜  Network: http://172.20.0.4:5173/

Database - operafix_postgres (postgres:16-alpine)

server started

CREATE DATABASE



/usr/local/bin/docker-entrypoint.sh: running /docker-entrypoint-initdb.d/00-init.sql

psql:/docker-entrypoint-initdb.d/00-init.sql: error: could not read from input file: Is a directory


PostgreSQL Database directory appears to contain a database; Skipping initialization


2026-04-23 08:41:49.926 UTC [1] LOG:  starting PostgreSQL 16.13 on aarch64-unknown-linux-musl, compiled by gcc (Alpine 15.2.0) 15.2.0, 64-bit

2026-04-23 08:41:49.926 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432

2026-04-23 08:41:49.926 UTC [1] LOG:  listening on IPv6 address "::", port 5432

2026-04-23 08:41:49.928 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"

2026-04-23 08:41:49.930 UTC [29] LOG:  database system was interrupted; last known up at 2026-04-23 08:41:49 UTC

2026-04-23 08:41:49.975 UTC [29] LOG:  database system was not properly shut down; automatic recovery in progress

2026-04-23 08:41:49.976 UTC [29] LOG:  redo starts at 0/14F2618

2026-04-23 08:41:49.983 UTC [29] LOG:  invalid record length at 0/191E8F8: expected at least 24, got 0

2026-04-23 08:41:49.983 UTC [29] LOG:  redo done at 0/191E8B0 system usage: CPU: user: 0.00 s, system: 0.00 s, elapsed: 0.00 s

2026-04-23 08:41:49.985 UTC [27] LOG:  checkpoint starting: end-of-recovery immediate wait

2026-04-23 08:41:50.016 UTC [27] LOG:  checkpoint complete: wrote 926 buffers (5.7%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.010 s, sync=0.020 s, total=0.032 s; sync files=301, longest=0.005 s, average=0.001 s; distance=4272 kB, estimate=4272 kB; lsn=0/191E8F8, redo lsn=0/191E8F8

2026-04-23 08:41:50.018 UTC [1] LOG:  database system is ready to accept connections

2026-04-23 08:42:31.100 UTC [92] ERROR:  relation "hdb_catalog.hdb_version" does not exist at character 69

2026-04-23 08:42:31.100 UTC [92] STATEMENT:  

	        SELECT ee_client_id::text, ee_client_secret

	          FROM hdb_catalog.hdb_version

	      

2026-04-23 08:46:49.063 UTC [27] LOG:  checkpoint starting: time

2026-04-23 08:47:09.550 UTC [27] LOG:  checkpoint complete: wrote 199 buffers (1.2%); 0 WAL file(s) added, 0 removed, 0 recycled; write=20.456 s, sync=0.018 s, total=20.488 s; sync files=295, longest=0.003 s, average=0.001 s; distance=1447 kB, estimate=3990 kB; lsn=0/1A88850, redo lsn=0/1A88818

Hasura - operafix_hasura (of-dev-hasura:latest)

{"detail":{"info":{"admin_secret_set":true,"auth_hook":null,"auth_hook_mode":null,"console_assets_dir":null,"console_sentry_dsn":null,"cors_config":{"allowed_origins":"*","disabled":false,"ws_read_cookie":null},"enable_allowlist":false,"enable_console":true,"enable_maintenance_mode":false,"enable_metadata_query_logging":false,"enable_telemetry":true,"enabled_apis":["config","pgdump","metadata","graphql"],"enabled_log_types":["webhook-log","http-log","startup","query-log","websocket-log"],"events_fetch_batch_size":100,"experimental_features":[],"graceful_shutdown_timeout":60,"infer_function_permissions":true,"jwt_secret":[{"audience":null,"claims_format":"json","claims_namespace":"https://hasura.io/jwt/claims","header":null,"issuer":null,"key":"<JWK REDACTED>","type":"<TYPE REDACTED>"}],"live_query_options":{"batch_size":100,"refetch_delay":1},"log_level":"info","port":8080,"remote_schema_permissions":false,"server_host":"HostAny","stringify_numeric_types":false,"transaction_isolation":"ISOLATION LEVEL SERIALIZABLE","unauth_role":null,"use_prepared_statements":true,"v1-boolean-null-collapse":false,"websocket_compression_options":"NoCompression","websocket_connection_init_timeout":"Refined (Seconds {seconds = 3s})","websocket_keep_alive":"KeepAliveDelay {unKeepAliveDelay = Refined (Seconds {seconds = 5s})}"},"kind":"server_configuration"},"level":"info","timestamp":"2026-04-23T08:42:31.101+0000","type":"startup"}

{"detail":"Thread runCronEventsGenerator (re)started","level":"info","timestamp":"2026-04-23T08:42:31.101+0000","type":"unstructured"}

{"detail":"Thread websocketConnectionReaper (re)started","level":"info","timestamp":"2026-04-23T08:42:31.101+0000","type":"unstructured"}

{"detail":{"info":{"database_url":"postgres://operafix:...@postgres:5432/operafix","retries":1},"kind":"postgres_connection"},"level":"info","timestamp":"2026-04-23T08:42:31.101+0000","type":"startup"}

{"detail":{"info":"Successfully initialized the catalog (at version 48).","kind":"catalog_migrate"},"level":"info","timestamp":"2026-04-23T08:42:31.101+0000","type":"startup"}

{"detail":{"info":"Schema sync enabled. Polling at Refined (Milliseconds {milliseconds = 1s})","kind":"schema-sync"},"level":"info","timestamp":"2026-04-23T08:42:31.101+0000","type":"startup"}

{"detail":{"info":{"instance_id":"49d13d59-e1e6-4a65-b90f-8ef2ce96294b","message":"listener thread started","thread_id":"ThreadId 36"},"kind":"schema-sync"},"level":"info","timestamp":"2026-04-23T08:42:31.101+0000","type":"startup"}

{"detail":{"info":"Could not obtain EE client credentials from the metadata database. Hasura EE features are disabled.\nError: {\"arguments\":[],\"error\":{\"description\":null,\"exec_status\":\"FatalError\",\"hint\":null,\"message\":\"relation \\\"hdb_catalog.hdb_version\\\" does not exist\",\"status_code\":\"42P01\"},\"prepared\":false,\"statement\":\"\\n        SELECT ee_client_id::text, ee_client_secret\\n          FROM hdb_catalog.hdb_version\\n      \"}\n","kind":"license_info"},"level":"warn","timestamp":"2026-04-23T08:42:31.101+0000","type":"startup"}

{"detail":{"info":{"message":"source \"default\" has not been initialized yet.","source":"default"},"kind":"source_catalog_migrate"},"level":"info","timestamp":"2026-04-23T08:42:31.101+0000","type":"startup"}

{"detail":"The following environment variables are deprecated and moved to metadata: HASURA_GRAPHQL_PG_CONNECTIONS, HASURA_GRAPHQL_TX_ISOLATION","level":"warn","timestamp":"2026-04-23T08:42:31.101+0000","type":"unstructured"}

{"detail":{"info":{"instance_id":"49d13d59-e1e6-4a65-b90f-8ef2ce96294b","message":"processor thread started","thread_id":"ThreadId 54"},"kind":"schema-sync"},"level":"info","timestamp":"2026-04-23T08:42:31.101+0000","type":"startup"}

{"detail":{"info":"Starting in eventing enabled mode","kind":"server"},"level":"info","timestamp":"2026-04-23T08:42:31.101+0000","type":"startup"}

{"detail":{"info":"Starting workers","kind":"event_triggers"},"level":"info","timestamp":"2026-04-23T08:42:31.101+0000","type":"startup"}

{"detail":{"info":"Help us improve Hasura! The graphql-engine server collects anonymized usage stats which allows us to keep improving Hasura at warp speed. To read more or opt-out, visit [https://hasura.io/docs/latest/graphql/core/guides/telemetry.html","kind":"telemetry"},"level":"info","timestamp":"2026-04-23T08:42:31.101+0000","type":"startup"}](https://hasura.io/docs/latest/graphql/core/guides/telemetry.html%22,%22kind%22:%22telemetry%22%7D,%22level%22:%22info%22,%22timestamp%22:%222026-04-23T08:42:31.101+0000%22,%22type%22:%22startup%22%7D)

API - (of-dev-api:latest)

  __    _   ___  

 / /\  |  | |  |_) 
/_/--\ |_| |_| \_ v1.52.3, built with Go go1.23.12


[08:42:36] mkdir /app/tmp

[08:42:36] watching .

[08:42:36] watching cmd

[08:42:36] watching cmd/server

[08:42:36] watching internal

[08:42:36] watching internal/auth

[08:42:36] watching internal/docs

[08:42:36] watching pkg

[08:42:36] watching pkg/config

[08:42:36] watching pkg/database

[08:42:36] watching .

[08:42:36] !exclude tmp

[08:42:36] building...

[08:42:36] watching cmd

[08:42:36] watching cmd/server

[08:42:36] watching internal

[08:42:36] watching internal/auth

[08:42:36] watching internal/docs

[08:42:36] watching pkg

[08:42:36] watching pkg/config

[08:42:36] watching pkg/database

[08:42:36] !exclude tmp

[08:42:36] watching .

[08:42:36] watching cmd

[08:42:36] watching cmd/server

[08:42:36] watching internal

[08:42:36] watching internal/auth

[08:42:36] watching internal/docs

@HarryVasanth
Copy link
Copy Markdown
Member

We need to update all the images to have the latest versions of them and also not use EE graphql but use the community version.

@LEEDASILVA
Copy link
Copy Markdown
Collaborator Author

We need to update all the images to have the latest versions of them and also not use EE graphql but use the community version.

updated all services to match recent ones at least the stabled ones (postgres will be 17, still don't trust 18 hehe)
the thing about using EE graphql and use the community version, not sure don't know if we will use the backend stuff, but for now we could just use the community version. (to do this i just add the ce? still not sure)

@LEEDASILVA LEEDASILVA force-pushed the re-organize-system-to-handle-microservices branch from e667c56 to 9e5c52a Compare April 28, 2026 20:09
@LEEDASILVA LEEDASILVA changed the title Implement micro service system + add and seperate services Implement micro service system + add and seperate services + development enviroment Apr 28, 2026
Comment thread hasura/config.yaml Outdated
Comment thread api/internal/auth/handler.go
Comment thread hasura/migrations/default/1000000000009_issues/up.sql
Comment thread hasura/migrations/default/1000000000009_issues/up.sql Outdated
Comment thread api/internal/auth/service.go
Comment thread hasura/migrations/default/1000000000008_equipment/up.sql Outdated
Comment thread hasura/migrations/default/1000000000008_equipment/up.sql
Comment thread api/internal/auth/token.go Outdated
Comment thread hasura/migrations/default/1000000000017_auth_tokens/up.sql Outdated
Comment thread hasura/migrations/default/1000000000009_issues/up.sql
Comment thread hasura/migrations/default/1000000000010_maintenance_actions/up.sql
Comment thread docker-compose.prod.yml
Copy link
Copy Markdown
Member

@HarryVasanth HarryVasanth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

Copy link
Copy Markdown
Member

@HarryVasanth HarryVasanth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit:

@LEEDASILVA LEEDASILVA merged commit 6e04820 into main Apr 30, 2026
@LEEDASILVA LEEDASILVA deleted the re-organize-system-to-handle-microservices branch April 30, 2026 22:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants