Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion CODEBASE_OVERVIEW.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,7 @@ graph LR;
rackup["rackup"];
rake["rake"];
webrick["webrick"];
elasticgraph-json_ingestion["eg-json_ingestion"];
elasticgraph-schema_artifacts["eg-schema_artifacts"];
graphql["graphql"];
elasticgraph --> elasticgraph-support;
Expand All @@ -125,6 +126,7 @@ graph LR;
elasticgraph-local --> webrick;
elasticgraph-schema_definition --> elasticgraph-graphql;
elasticgraph-schema_definition --> elasticgraph-indexer;
elasticgraph-schema_definition --> elasticgraph-json_ingestion;
elasticgraph-schema_definition --> elasticgraph-schema_artifacts;
elasticgraph-schema_definition --> elasticgraph-support;
elasticgraph-schema_definition --> graphql;
Expand All @@ -141,6 +143,7 @@ graph LR;
class rackup externalGemCatStyle;
class rake externalGemCatStyle;
class webrick externalGemCatStyle;
class elasticgraph-json_ingestion otherEgGemStyle;
class elasticgraph-schema_artifacts otherEgGemStyle;
class graphql externalGemCatStyle;
click thor href "https://rubygems.org/gems/thor" "Open on RubyGems.org" _blank;
Expand Down Expand Up @@ -192,12 +195,13 @@ graph LR;
click opensearch-ruby href "https://rubygems.org/gems/opensearch-ruby" "Open on RubyGems.org" _blank;
```

### Extensions (5 gems)
### Extensions (6 gems)

These libraries extend ElasticGraph to provide optional but commonly needed functionality.

* [elasticgraph-apollo](elasticgraph-apollo/README.md): Transforms an ElasticGraph project into an Apollo subgraph.
* [elasticgraph-health_check](elasticgraph-health_check/README.md): Provides a health check for high availability ElasticGraph deployments.
* [elasticgraph-json_ingestion](elasticgraph-json_ingestion/README.md): JSON Schema ingestion support for ElasticGraph.
* [elasticgraph-query_interceptor](elasticgraph-query_interceptor/README.md): Intercepts ElasticGraph datastore queries.
* [elasticgraph-query_registry](elasticgraph-query_registry/README.md): Provides a source-controlled query registry for ElasticGraph applications.
* [elasticgraph-warehouse](elasticgraph-warehouse/README.md): Extends ElasticGraph to support ingestion into a data warehouse.
Expand All @@ -216,6 +220,7 @@ graph LR;
apollo-federation["apollo-federation"];
elasticgraph-health_check["eg-health_check"];
elasticgraph-datastore_core["eg-datastore_core"];
elasticgraph-json_ingestion["eg-json_ingestion"];
elasticgraph-query_interceptor["eg-query_interceptor"];
elasticgraph-schema_artifacts["eg-schema_artifacts"];
elasticgraph-query_registry["eg-query_registry"];
Expand All @@ -228,6 +233,7 @@ graph LR;
elasticgraph-health_check --> elasticgraph-datastore_core;
elasticgraph-health_check --> elasticgraph-graphql;
elasticgraph-health_check --> elasticgraph-support;
elasticgraph-json_ingestion --> elasticgraph-support;
elasticgraph-query_interceptor --> elasticgraph-graphql;
elasticgraph-query_interceptor --> elasticgraph-schema_artifacts;
elasticgraph-query_registry --> elasticgraph-graphql;
Expand All @@ -242,6 +248,7 @@ graph LR;
class apollo-federation externalGemCatStyle;
class elasticgraph-health_check targetGemStyle;
class elasticgraph-datastore_core otherEgGemStyle;
class elasticgraph-json_ingestion targetGemStyle;
class elasticgraph-query_interceptor targetGemStyle;
class elasticgraph-schema_artifacts otherEgGemStyle;
class elasticgraph-query_registry targetGemStyle;
Expand Down
9 changes: 9 additions & 0 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,12 @@ PATH
elasticgraph-support (= 1.2.1.pre)
hashdiff (~> 1.2, >= 1.2.1)

PATH
remote: elasticgraph-json_ingestion
specs:
elasticgraph-json_ingestion (1.2.1.pre)
elasticgraph-support (= 1.2.1.pre)

PATH
remote: elasticgraph-lambda_support
specs:
Expand Down Expand Up @@ -192,6 +198,7 @@ PATH
elasticgraph-schema_definition (1.2.1.pre)
elasticgraph-graphql (= 1.2.1.pre)
elasticgraph-indexer (= 1.2.1.pre)
elasticgraph-json_ingestion (= 1.2.1.pre)
elasticgraph-schema_artifacts (= 1.2.1.pre)
elasticgraph-support (= 1.2.1.pre)
graphql (~> 2.6.2)
Expand Down Expand Up @@ -698,6 +705,7 @@ DEPENDENCIES
elasticgraph-indexer (= 1.2.1.pre)!
elasticgraph-indexer_autoscaler_lambda (= 1.2.1.pre)!
elasticgraph-indexer_lambda (= 1.2.1.pre)!
elasticgraph-json_ingestion (= 1.2.1.pre)!
elasticgraph-lambda_support (= 1.2.1.pre)!
elasticgraph-local (= 1.2.1.pre)!
elasticgraph-opensearch (= 1.2.1.pre)!
Expand Down Expand Up @@ -789,6 +797,7 @@ CHECKSUMS
elasticgraph-indexer (1.2.1.pre)
elasticgraph-indexer_autoscaler_lambda (1.2.1.pre)
elasticgraph-indexer_lambda (1.2.1.pre)
elasticgraph-json_ingestion (1.2.1.pre)
elasticgraph-lambda_support (1.2.1.pre)
elasticgraph-local (1.2.1.pre)
elasticgraph-opensearch (1.2.1.pre)
Expand Down
1 change: 0 additions & 1 deletion Rakefile
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,6 @@ end

configure_local_rake_tasks = ->(tasks) do
tasks.schema_element_name_form = :snake_case
tasks.enforce_json_schema_version = false
tasks.index_document_sizes = true
tasks.env_port_mapping = {test: test_port}
tasks.output = schema_def_output
Expand Down
1 change: 1 addition & 0 deletions config/docker_demo/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ COPY elasticgraph-datastore_core elasticgraph-datastore_core/
COPY elasticgraph-graphiql elasticgraph-graphiql/
COPY elasticgraph-graphql elasticgraph-graphql/
COPY elasticgraph-indexer elasticgraph-indexer/
COPY elasticgraph-json_ingestion elasticgraph-json_ingestion/
COPY elasticgraph-local elasticgraph-local/
COPY elasticgraph-opensearch elasticgraph-opensearch/
COPY elasticgraph-query_registry elasticgraph-query_registry/
Expand Down
1 change: 1 addition & 0 deletions config/schema.rb
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@

ElasticGraph.define_schema do |schema|
schema.json_schema_version 1
schema.enforce_json_schema_version false
end

# Note: anytime you add a file to load here, you'll also have to update the list here:
Expand Down
1 change: 0 additions & 1 deletion config/site/Rakefile
Original file line number Diff line number Diff line change
Expand Up @@ -461,7 +461,6 @@ module ElasticGraph
# to demonstrate any `Rakefile` APIs.
::ElasticGraph::Local::RakeTasks.new(local_config_yaml: settings_file, path_to_schema: schema_file) do |tasks|
tasks.opensearch_versions = []
tasks.enforce_json_schema_version = false
end
end

Expand Down
1 change: 1 addition & 0 deletions config/site/examples/custom_resolver/schema.rb
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@

ElasticGraph.define_schema do |schema|
schema.json_schema_version 1
schema.enforce_json_schema_version false

# :snippet-start: register_graphql_resolver
require(require_path = "roll_dice_resolver")
Expand Down
1 change: 0 additions & 1 deletion config/site/examples/music/Rakefile
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ ElasticGraph::Local::RakeTasks.new(
path_to_schema: File.expand_path("schema.rb", __dir__)
) do |tasks|
tasks.opensearch_versions = []
tasks.enforce_json_schema_version = false
end

# :nocov: -- only used for manual live-server query validation, not during the build
Expand Down
1 change: 1 addition & 0 deletions config/site/examples/music/schema.rb
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@

ElasticGraph.define_schema do |schema|
schema.json_schema_version 1
schema.enforce_json_schema_version false
end

Dir["#{__dir__}/schema/**/*.rb"].each do |schema_def_file|
Expand Down
1 change: 1 addition & 0 deletions config/site/examples/music_simplified/schema.rb
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
ElasticGraph.define_schema do |schema|
schema.json_schema_version 1
schema.enforce_json_schema_version false

schema.object_type "Artist" do |t|
t.field "id", "ID"
Expand Down
1 change: 1 addition & 0 deletions config/site/examples/namespaced_queries/schema.rb
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@

ElasticGraph.define_schema do |schema|
schema.json_schema_version 1
schema.enforce_json_schema_version false
end

# :snippet-start: namespace_type
Expand Down
1 change: 1 addition & 0 deletions config/site/examples/nested_namespaced_queries/schema.rb
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@

ElasticGraph.define_schema do |schema|
schema.json_schema_version 1
schema.enforce_json_schema_version false
end

# :snippet-start: nested_namespace_type
Expand Down
1 change: 1 addition & 0 deletions config/site/examples/schema_customization/schema.rb
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@

ElasticGraph.define_schema do |schema|
schema.json_schema_version 1
schema.enforce_json_schema_version false

# :snippet-start: on_each_generated_schema_element
# Within `ElasticGraph.define_schema { ... }` in your schema definition:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@ ElasticGraph::Local::RakeTasks.new(
path_to_schema: File.expand_path("schema.rb", __dir__)
) do |tasks|
tasks.opensearch_versions = []
tasks.enforce_json_schema_version = false

# :snippet-start: schema_element_name_form
# Within `ElasticGraph::Local::RakeTasks.new { ... }` in your `Rakefile`:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@

ElasticGraph.define_schema do |schema|
schema.json_schema_version 1
schema.enforce_json_schema_version false

schema.object_type "Widget" do |t|
t.field "id", "ID"
Expand Down
1 change: 0 additions & 1 deletion config/site/support/doctest_helper.rb
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,6 @@ module ElasticGraph
artifacts_manager = @api.factory.new_schema_artifact_manager(
schema_definition_results: @api.results,
schema_artifacts_directory: "#{@tmp_dir}/schema_artifacts",
enforce_json_schema_version: true,
output: ::StringIO.new
)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -167,7 +167,6 @@ def fetch_artifact_configuration(schema_artifacts, index_def_name)
factory.new_schema_artifact_manager(
schema_definition_results: schema_def_results,
schema_artifacts_directory: Dir.pwd,
enforce_json_schema_version: true,
output: output_io
).dump_artifacts

Expand Down
16 changes: 10 additions & 6 deletions elasticgraph-apollo/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,15 +64,14 @@ index 2943335..26633c3 100644
require "elastic_graph/local/rake_tasks"
require "elastic_graph/query_registry/rake_tasks"
require "rspec/core/rake_task"
@@ -12,6 +13,8 @@ ElasticGraph::Local::RakeTasks.new(
@@ -12,5 +13,7 @@ ElasticGraph::Local::RakeTasks.new(
local_config_yaml: settings_file,
path_to_schema: "#{project_root}/config/schema.rb"
) do |tasks|
+ tasks.schema_definition_extension_modules << ElasticGraph::Apollo::SchemaDefinition::APIExtension
+
# Set this to true once you're beyond the prototyping stage.
tasks.enforce_json_schema_version = false

# Determines casing of field names. Can be either `:camelCase` or `:snake_case`.
tasks.schema_element_name_form = :camelCase
```

That's it!
Expand All @@ -94,11 +93,16 @@ diff --git a/config/schema.rb b/config/schema.rb
index 015c5fa..362cdcb 100644
--- a/config/schema.rb
+++ b/config/schema.rb
@@ -4,6 +4,8 @@ ElasticGraph.define_schema do |schema|
@@ -4,7 +4,9 @@ ElasticGraph.define_schema do |schema|
# ElasticGraph will tell you when you need to bump this.
schema.json_schema_version 1

-
+
+ schema.target_apollo_federation_version "2.3"
+
# Set this to true once you're beyond the prototyping stage.
schema.enforce_json_schema_version false
-
+
# This registers the elasticgraph-query_registry extension, which can be used to reject queries that
# clients have not registered (and to reject queries that differ from what a client has registered).
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ COPY elasticgraph-elasticsearch /web/elasticgraph-elasticsearch
COPY elasticgraph-graphiql /web/elasticgraph-graphiql
COPY elasticgraph-graphql /web/elasticgraph-graphql
COPY elasticgraph-indexer /web/elasticgraph-indexer
COPY elasticgraph-json_ingestion /web/elasticgraph-json_ingestion
COPY elasticgraph-rack /web/elasticgraph-rack
COPY elasticgraph-schema_artifacts /web/elasticgraph-schema_artifacts
COPY elasticgraph-schema_definition /web/elasticgraph-schema_definition
Expand Down
1 change: 1 addition & 0 deletions elasticgraph-apollo/apollo_tests_implementation/Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ source "https://rubygems.org"
graphiql
graphql
indexer
json_ingestion
rack
schema_artifacts
schema_definition
Expand Down
3 changes: 1 addition & 2 deletions elasticgraph-apollo/apollo_tests_implementation/Rakefile
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,5 @@ ElasticGraph::SchemaDefinition::RakeTasks.new(
index_document_sizes: false,
path_to_schema: project_root / "config/products_schema.rb",
schema_artifacts_directory: project_root / "config/schema/artifacts",
extension_modules: [ElasticGraph::Apollo::SchemaDefinition::APIExtension],
enforce_json_schema_version: false
extension_modules: [ElasticGraph::Apollo::SchemaDefinition::APIExtension]
)
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@ module ApolloTestImpl
# https://github.com/apollographql/apollo-federation-subgraph-compatibility/blob/2.0.0/COMPATIBILITY.md#products-schema-to-be-implemented-by-library-maintainers
ElasticGraph.define_schema do |schema|
schema.json_schema_version 1
schema.enforce_json_schema_version false
schema.target_apollo_federation_version(federation_version) if federation_version

unless federation_version == "2.0"
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -68,7 +68,6 @@ def dump_schema_artifacts(json_schema_version:, team_extras: "")
index_document_sizes: true,
path_to_schema: path_to_schema,
schema_artifacts_directory: "config/schema/artifacts",
enforce_json_schema_version: true,
output: output
)
end
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -404,7 +404,6 @@ def dump_artifacts
index_document_sizes: true,
path_to_schema: path_to_schema,
schema_artifacts_directory: "config/schema/artifacts",
enforce_json_schema_version: true,
output: output
)
end
Expand Down
1 change: 1 addition & 0 deletions elasticgraph-json_ingestion/.rspec
1 change: 1 addition & 0 deletions elasticgraph-json_ingestion/.yardopts
1 change: 1 addition & 0 deletions elasticgraph-json_ingestion/Gemfile
21 changes: 21 additions & 0 deletions elasticgraph-json_ingestion/LICENSE.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
The MIT License (MIT)

Copyright (c) 2024 - 2026 Block, Inc.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.
23 changes: 23 additions & 0 deletions elasticgraph-json_ingestion/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# ElasticGraph::JSONIngestion

JSON Schema ingestion support for ElasticGraph.

This gem contains the JSON Schema helper code used by schema definition to generate indexing
event schemas and merge ElasticGraph metadata into versioned schema artifacts.

## Dependency Diagram

```mermaid
graph LR;
classDef targetGemStyle fill:#FADBD8,stroke:#EC7063,color:#000,stroke-width:2px;
classDef otherEgGemStyle fill:#A9DFBF,stroke:#2ECC71,color:#000;
classDef externalGemStyle fill:#E0EFFF,stroke:#70A1D7,color:#2980B9;
elasticgraph-json_ingestion["elasticgraph-json_ingestion"];
class elasticgraph-json_ingestion targetGemStyle;
elasticgraph-support["elasticgraph-support"];
elasticgraph-json_ingestion --> elasticgraph-support;
class elasticgraph-support otherEgGemStyle;
elasticgraph-schema_definition["elasticgraph-schema_definition"];
elasticgraph-schema_definition --> elasticgraph-json_ingestion;
class elasticgraph-schema_definition otherEgGemStyle;
```
Loading
Loading