-
Notifications
You must be signed in to change notification settings - Fork 65
fix(node-dependency-matrix): count gRPC test cases in recording calibration (0 → 2) #101
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
24 commits
Select commit
Hold shift + click to select a range
52f42a8
Add node dependency matrix sample
officialasishkumar 4759af3
Add compose setup for node matrix sample
officialasishkumar 36ceeac
docs: tighten node dependency matrix k8s flow
officialasishkumar 1a7a45a
Stabilize node dependency matrix recording flows
officialasishkumar 8b5cb4f
Document node dependency matrix staging runbook
officialasishkumar 0f32218
Implement async catalog synchronization and deduplication endpoints i…
officialasishkumar fa22767
Update Kubernetes traffic recording instructions and enhance scripts …
officialasishkumar e058e3f
Refactor Kafka and SQS scenario functions to improve error handling a…
officialasishkumar 503c04a
Enhance deploy-kind.sh to support custom node images for Kind cluster…
officialasishkumar faf87b6
fix: update expected-values.json to match actual UI dedup behavior
officialasishkumar 93048b8
fix: update requiredKinds to include PostgresV2 and Redis
officialasishkumar 90ef1d0
fix: address Copilot review feedback for node-dependency-matrix
officialasishkumar c09e5b3
fix: reduce requiredKinds to Http only — mock kinds vary between runs
officialasishkumar 106d008
fix: sync package-lock.json with package.json
officialasishkumar d5c74f0
fix: use ECR public mirrors to avoid Docker Hub rate limits
officialasishkumar a083455
fix: use docker pull + kind load instead of crictl pull
officialasishkumar aaf64db
fix: configure Kind with GCR mirror to avoid all registry rate limits
officialasishkumar 616949a
fix: use Keploy's local registry mirror (192.168.116.165:5000)
officialasishkumar d7fadf6
fix: load Kind images one at a time to avoid all-or-nothing failure
officialasishkumar 4c88fc1
fix(node-dependency-matrix): allow Mongo TLS MITM hostname mismatch
officialasishkumar 894935e
fix(node-dependency-matrix): pin Mongo TLS to direct endpoint
officialasishkumar 5f03052
fix(node-dependency-matrix): poll Mongo server monitor through proxy
officialasishkumar 16ca1de
fix(node-dependency-matrix): recalibrate expectedTestcasesExact 18 ->…
slayerjain 4359959
fix(node-dependency-matrix): count gRPC test cases in recording calib…
Shivanipandey31 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,7 @@ | ||
| node_modules | ||
| dist | ||
| .generated | ||
| README.md | ||
| k8s | ||
| scripts | ||
| keploy |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| dist | ||
| node_modules | ||
| .generated | ||
| keploy |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| FROM node:20-bookworm-slim@sha256:17281e8d1dc4d671976c6b89a12f47a44c2f390b63a989e2e327631041f544fd AS build | ||
|
|
||
| WORKDIR /app | ||
|
|
||
| COPY package*.json ./ | ||
| RUN npm ci | ||
|
|
||
| COPY tsconfig.json ./ | ||
| COPY proto ./proto | ||
| COPY fixtures ./fixtures | ||
| COPY src ./src | ||
| COPY entrypoint.sh ./entrypoint.sh | ||
|
|
||
| RUN npm run build && chmod +x /app/entrypoint.sh | ||
|
|
||
| FROM node:20-bookworm-slim@sha256:17281e8d1dc4d671976c6b89a12f47a44c2f390b63a989e2e327631041f544fd | ||
|
|
||
| WORKDIR /app | ||
|
|
||
| RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates && rm -rf /var/lib/apt/lists/* | ||
|
|
||
| COPY --from=build /app/node_modules ./node_modules | ||
| COPY --from=build /app/dist ./dist | ||
| COPY --from=build /app/proto ./proto | ||
| COPY --from=build /app/fixtures ./fixtures | ||
| COPY --from=build /app/entrypoint.sh ./entrypoint.sh | ||
|
|
||
| ENV NODE_ENV=production | ||
| ENV PORT=8080 | ||
| ENV GRPC_PORT=9090 | ||
|
|
||
| ENTRYPOINT ["./entrypoint.sh"] | ||
| CMD ["node", "dist/bin/app.js"] |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,105 @@ | ||
| # Node Dependency Matrix | ||
|
|
||
| This sample is a TypeScript application built to validate Keploy cloud record/replay against a broad dependency matrix. | ||
|
|
||
| What it exposes: | ||
|
|
||
| - incoming HTTP | ||
| - incoming gRPC | ||
| - outgoing HTTPS | ||
| - outgoing HTTP/2 | ||
| - outgoing gRPC | ||
| - outgoing MySQL | ||
| - outgoing Postgres | ||
| - outgoing Mongo | ||
| - outgoing Redis | ||
| - outgoing Kafka | ||
| - outgoing SQS-over-HTTPS | ||
| - outgoing generic TLS traffic | ||
| - an async background workflow that performs real outbound calls after a `202 Accepted` | ||
| - a noisy endpoint | ||
| - an expected replay failure endpoint | ||
| - duplicate-friendly GET and POST endpoints for static dedup verification | ||
|
|
||
| ## Quick start | ||
|
|
||
| ```bash | ||
| cd samples-typescript/node-dependency-matrix | ||
| npm install | ||
| npm run build | ||
| bash k8s/deploy-kind.sh | ||
| APP_URL=http://localhost:30081 bash scripts/record_traffic.sh | ||
| ``` | ||
|
|
||
| If the default host ports are already occupied, override them: | ||
|
|
||
| ```bash | ||
| HOST_PROXY_PORT=31080 HOST_APP_PORT=31081 HOST_GRPC_PORT=31090 bash k8s/deploy-kind.sh | ||
| APP_URL=http://localhost:31081 bash scripts/record_traffic.sh | ||
| ``` | ||
|
|
||
| Optional incoming gRPC traffic: | ||
|
|
||
| ```bash | ||
| GRPC_TARGET=localhost:30090 bash scripts/send_grpc_traffic.sh | ||
| ``` | ||
|
|
||
| ## Docker Compose | ||
|
|
||
| For a non-Kubernetes local stack: | ||
|
|
||
| ```bash | ||
| cd samples-typescript/node-dependency-matrix | ||
| npm install | ||
| bash scripts/compose_up.sh | ||
| APP_URL=http://localhost:38081 bash scripts/record_traffic.sh | ||
| GRPC_TARGET=localhost:39090 bash scripts/send_grpc_traffic.sh | ||
| ``` | ||
|
|
||
| To stop it: | ||
|
|
||
| ```bash | ||
| bash scripts/compose_down.sh | ||
| ``` | ||
|
|
||
| If `38081` or `39090` are occupied: | ||
|
|
||
| ```bash | ||
| APP_HTTP_PORT=48081 APP_GRPC_PORT=49090 bash scripts/compose_up.sh | ||
| APP_URL=http://localhost:48081 bash scripts/record_traffic.sh | ||
| GRPC_TARGET=localhost:49090 bash scripts/send_grpc_traffic.sh | ||
| ``` | ||
|
|
||
| ## Keploy expectations | ||
|
|
||
| The machine-readable contract is: | ||
|
|
||
| ```bash | ||
| fixtures/expected-values.json | ||
| ``` | ||
|
|
||
| Use it later in manual verification or other automation for: | ||
|
|
||
| - testcase counts | ||
| - required and acceptable mock kinds | ||
| - scenario-to-mock-kind mapping | ||
| - the exact HTTP and gRPC traffic plans | ||
| - the async workflow contract and expected background scenarios | ||
| - hosted UI + Helm expectations for Kind | ||
| - total mock count ranges | ||
| - dedup expectations | ||
| - expected replay pass/noisy/fail scenarios | ||
|
|
||
| ## Kubernetes | ||
|
|
||
| The full Kind and hosted UI flow is in: | ||
|
|
||
| ```bash | ||
| k8s/README.md | ||
| ``` | ||
|
|
||
| The detailed step-by-step staging runbook and troubleshooting guide is in: | ||
|
|
||
| ```bash | ||
| k8s/STAGING_RUNBOOK.md | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,206 @@ | ||
| name: node-dependency-matrix | ||
|
|
||
| x-node-matrix-image: &node_matrix_image | ||
| build: | ||
| context: . | ||
| image: node-dependency-matrix:compose | ||
|
|
||
| services: | ||
| app: | ||
| <<: *node_matrix_image | ||
| depends_on: | ||
| fixture-service: | ||
| condition: service_started | ||
| mysql: | ||
| condition: service_healthy | ||
| postgres: | ||
| condition: service_healthy | ||
| mongo-tls: | ||
| condition: service_started | ||
| redis-tls: | ||
| condition: service_started | ||
| redpanda: | ||
| condition: service_started | ||
| sqs-tls: | ||
| condition: service_started | ||
| environment: | ||
| PORT: "8080" | ||
| GRPC_PORT: "9090" | ||
| FIXTURE_HTTPS_BASE: https://fixture-service:8443 | ||
| FIXTURE_HTTP2_ORIGIN: https://fixture-service:9443 | ||
| FIXTURE_GRPC_TARGET: fixture-service:50051 | ||
| FIXTURE_GENERIC_HOST: fixture-service | ||
| FIXTURE_GENERIC_PORT: "9445" | ||
| MYSQL_URL: mysql://root@mysql:3306/matrix | ||
| POSTGRES_URL: postgresql://postgres@postgres:5432/matrix | ||
| MONGO_URL: mongodb://mongo-tls:27017/matrix?tls=true | ||
| REDIS_URL: rediss://redis-tls:6380 | ||
| KAFKA_BROKERS: redpanda:9092 | ||
| KAFKA_TOPIC: matrix-events | ||
| SQS_ENDPOINT: https://sqs-tls:4567 | ||
| SQS_QUEUE_URL: https://sqs-tls:4567/000000000000/dependency-matrix | ||
| SAMPLE_CA_CERT_PATH: /etc/sample-certs/ca.crt | ||
| COMBINED_CA_CERT_PATH: /tmp/node-dependency-matrix-ca-bundle.crt | ||
| NODE_EXTRA_CA_CERTS: /tmp/node-dependency-matrix-ca-bundle.crt | ||
| SSL_CERT_FILE: /tmp/node-dependency-matrix-ca-bundle.crt | ||
| ports: | ||
| - "${APP_HTTP_PORT:-38081}:8080" | ||
| - "${APP_GRPC_PORT:-39090}:9090" | ||
| volumes: | ||
| - ./.generated/certs:/etc/sample-certs:ro | ||
|
|
||
| fixture-service: | ||
| <<: *node_matrix_image | ||
| command: ["node", "dist/bin/dependencyFixture.js"] | ||
| environment: | ||
| TLS_CERT_PATH: /etc/sample-certs/proxy.crt | ||
| TLS_KEY_PATH: /etc/sample-certs/proxy.key | ||
| volumes: | ||
| - ./.generated/certs:/etc/sample-certs:ro | ||
|
|
||
| mysql: | ||
| image: mysql:8.0 | ||
| command: ["mysqld", "--default-authentication-plugin=mysql_native_password"] | ||
| environment: | ||
| MYSQL_ALLOW_EMPTY_PASSWORD: "yes" | ||
| MYSQL_DATABASE: matrix | ||
| healthcheck: | ||
| test: ["CMD", "mysqladmin", "ping", "-h", "127.0.0.1", "-uroot"] | ||
| interval: 5s | ||
| timeout: 5s | ||
| retries: 20 | ||
|
|
||
| postgres: | ||
| image: postgres:16 | ||
| environment: | ||
| POSTGRES_HOST_AUTH_METHOD: trust | ||
| POSTGRES_DB: matrix | ||
| POSTGRES_USER: postgres | ||
| healthcheck: | ||
| test: ["CMD-SHELL", "pg_isready -U postgres -d matrix"] | ||
| interval: 5s | ||
| timeout: 5s | ||
| retries: 20 | ||
|
|
||
| mongo: | ||
| image: mongo:7 | ||
| command: ["--bind_ip_all"] | ||
|
|
||
| redis: | ||
| image: redis:7-alpine | ||
|
|
||
| redpanda: | ||
| image: docker.redpanda.com/redpandadata/redpanda:v25.1.2 | ||
| command: | ||
| - redpanda | ||
| - start | ||
| - --overprovisioned | ||
| - --smp | ||
| - "1" | ||
| - --memory | ||
| - 512M | ||
| - --reserve-memory | ||
| - 0M | ||
| - --check=false | ||
| - --node-id | ||
| - "0" | ||
| - --kafka-addr | ||
| - PLAINTEXT://0.0.0.0:9092 | ||
| - --advertise-kafka-addr | ||
| - PLAINTEXT://redpanda:9092 | ||
| - --set | ||
| - redpanda.auto_create_topics_enabled=true | ||
|
|
||
| localstack: | ||
| image: localstack/localstack:3.3 | ||
| environment: | ||
| SERVICES: sqs | ||
| AWS_DEFAULT_REGION: us-east-1 | ||
| volumes: | ||
| - ./scripts/localstack-init:/etc/localstack/init/ready.d:ro | ||
|
|
||
| mysql-tls: | ||
| <<: *node_matrix_image | ||
| command: ["node", "dist/bin/tlsProxy.js"] | ||
| depends_on: | ||
| mysql: | ||
| condition: service_healthy | ||
| environment: | ||
| LISTEN_PORT: "3306" | ||
| TARGET_HOST: mysql | ||
| TARGET_PORT: "3306" | ||
| TLS_CERT_PATH: /etc/sample-certs/proxy.crt | ||
| TLS_KEY_PATH: /etc/sample-certs/proxy.key | ||
| volumes: | ||
| - ./.generated/certs:/etc/sample-certs:ro | ||
|
|
||
| postgres-tls: | ||
| <<: *node_matrix_image | ||
| command: ["node", "dist/bin/tlsProxy.js"] | ||
| depends_on: | ||
| postgres: | ||
| condition: service_healthy | ||
| environment: | ||
| LISTEN_PORT: "5432" | ||
| TARGET_HOST: postgres | ||
| TARGET_PORT: "5432" | ||
| TLS_CERT_PATH: /etc/sample-certs/proxy.crt | ||
| TLS_KEY_PATH: /etc/sample-certs/proxy.key | ||
| volumes: | ||
| - ./.generated/certs:/etc/sample-certs:ro | ||
|
|
||
| mongo-tls: | ||
| <<: *node_matrix_image | ||
| command: ["node", "dist/bin/tlsProxy.js"] | ||
| depends_on: | ||
| - mongo | ||
| environment: | ||
| LISTEN_PORT: "27017" | ||
| TARGET_HOST: mongo | ||
| TARGET_PORT: "27017" | ||
| TLS_CERT_PATH: /etc/sample-certs/proxy.crt | ||
| TLS_KEY_PATH: /etc/sample-certs/proxy.key | ||
| volumes: | ||
| - ./.generated/certs:/etc/sample-certs:ro | ||
|
|
||
| redis-tls: | ||
| <<: *node_matrix_image | ||
| command: ["node", "dist/bin/tlsProxy.js"] | ||
| depends_on: | ||
| - redis | ||
| environment: | ||
| LISTEN_PORT: "6380" | ||
| TARGET_HOST: redis | ||
| TARGET_PORT: "6379" | ||
| TLS_CERT_PATH: /etc/sample-certs/proxy.crt | ||
| TLS_KEY_PATH: /etc/sample-certs/proxy.key | ||
| volumes: | ||
| - ./.generated/certs:/etc/sample-certs:ro | ||
|
|
||
| kafka-tls: | ||
| <<: *node_matrix_image | ||
| command: ["node", "dist/bin/tlsProxy.js"] | ||
| depends_on: | ||
| - redpanda | ||
| environment: | ||
| LISTEN_PORT: "9094" | ||
| TARGET_HOST: redpanda | ||
| TARGET_PORT: "9092" | ||
| TLS_CERT_PATH: /etc/sample-certs/proxy.crt | ||
| TLS_KEY_PATH: /etc/sample-certs/proxy.key | ||
| volumes: | ||
| - ./.generated/certs:/etc/sample-certs:ro | ||
|
|
||
| sqs-tls: | ||
| <<: *node_matrix_image | ||
| command: ["node", "dist/bin/tlsProxy.js"] | ||
| depends_on: | ||
| - localstack | ||
| environment: | ||
| LISTEN_PORT: "4567" | ||
| TARGET_HOST: localstack | ||
| TARGET_PORT: "4566" | ||
| TLS_CERT_PATH: /etc/sample-certs/proxy.crt | ||
| TLS_KEY_PATH: /etc/sample-certs/proxy.key | ||
| volumes: | ||
| - ./.generated/certs:/etc/sample-certs:ro |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| #!/bin/sh | ||
| set -eu | ||
|
|
||
| COMBINED_CA_CERT_PATH="${COMBINED_CA_CERT_PATH:-/tmp/node-dependency-matrix-ca-bundle.crt}" | ||
|
|
||
| rm -f "${COMBINED_CA_CERT_PATH}" | ||
|
|
||
| if [ -n "${SAMPLE_CA_CERT_PATH:-}" ] && [ -f "${SAMPLE_CA_CERT_PATH}" ]; then | ||
| cat "${SAMPLE_CA_CERT_PATH}" > "${COMBINED_CA_CERT_PATH}" | ||
| cp "${SAMPLE_CA_CERT_PATH}" /usr/local/share/ca-certificates/node-dependency-matrix.crt | ||
| update-ca-certificates >/dev/null 2>&1 || true | ||
| fi | ||
|
|
||
| if [ -f /tmp/keploy-tls/ca.crt ]; then | ||
| cat /tmp/keploy-tls/ca.crt >> "${COMBINED_CA_CERT_PATH}" | ||
| fi | ||
|
|
||
| if [ ! -f "${COMBINED_CA_CERT_PATH}" ]; then | ||
| : > "${COMBINED_CA_CERT_PATH}" | ||
| >&2 echo "Warning: No CA certificates were found to create the combined CA bundle at '${COMBINED_CA_CERT_PATH}'. To fix this, either set SAMPLE_CA_CERT_PATH to a valid CA bundle file or mount /tmp/keploy-tls/ca.crt before starting this container so fixture endpoints can establish TLS correctly." | ||
| fi | ||
|
|
||
| exec "$@" | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.