Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions docs/develop/.pages
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ nav:
- Develop: index.md
- Accessing Graphs with Java Applications: accessing-graphs-with-java-applications
- Python Plugins: python-plugins
- Marketplace Packages: packages
- cmempy - Python API: cmempy-python-api
- cmemc - Python Scripts: cmemc-scripts
- Build (DataIntegration) APIs: dataintegration-apis
Expand Down
6 changes: 6 additions & 0 deletions docs/develop/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,12 @@ API documentation and programming recipes.

For Python developers, we offer a [Plugin SDK](python-plugins/index.md) as well as an API for accessing and manipulating Corporate Memory Instances ([cmem-cmempy](cmempy-python-api/index.md)).

- :material-package: Packages

---

[Create your own packages](packages/development) to allow easy distribution and reuse of existing content (data, workflows, ...) through [Marketplace Packages](packages/index.md).

- :material-api: OpenAPI specification

---
Expand Down
5 changes: 5 additions & 0 deletions docs/develop/packages/.pages
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
nav:
- Packages: index.md
- Installation and Usage: installation
- Development: development

152 changes: 152 additions & 0 deletions docs/develop/packages/development/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
---
title: "Marketplace Package Development"
icon: material/code-json
tags:
- Marketplace
- Package
---
# Marketplace Package Development

## Introduction

In order to support the development of packages, we published a [package template](https://github.com/eccenca/cmem-package-template).
Please have a look at this project to get started.

This page walks you through a basic example of creating a new package, adding different type of content inside it and finally building it into a package archive ready for distribution.

## Creating your own package

## Initializing

Using [described instructions](https://github.com/eccenca/cmem-package-template/tree/main#usage) have a local folder contain your newly templated project (`my-package-id`), for our example we will use :
```shell
🎤 Type of package
Project Package
🎤 Package ID (e.g., 'eccenca-supply-chain-vocab', 'w3c-org-vocab')
my-package-id
🎤 Human-readable package name (e.g., 'My Awesome Vocabulary', 'My Great Project' ..)
My own package
🎤 Short description of the package (e.g., 'A vocabulary for ...', 'A project ...')
My project and graphs
```

You should now have a folder with two level of files :

- Top level — generic package information such as the changelog, README, CI instructions, and licensing.
- Nested folder (my-package-id) — the actual package content, along with a manifest.

## Package content

The nested folder `my-package-id` represents your working directory for developing the package.

To add content to the package simply drag and drop files you want added into this folder or extract existing content from a live Corporate Memory instance to the working directory.

!!! Example "Extracting Corporate Memory content to add to the package "

```shell
cmemc graph export https://my-company.org/queries/ --output-file my-package-id/queries.ttl

cmemc project export MyProject_78e981443900a761 --output-dir my-package-id
Export project 1/1: MyProject_78e981443900a761 to my-package-id/2026-07-08-unnamed-MyProject_78e981443900a761.project.zip ... done

mv my-package-id/2026-07-08-unnamed-MyProject_78e981443900a761.project.zip my-package-id/project.zip
```

## Declaring the files in the manifest

In order for the package to know about these added files, the `cpa-manifest.json` needs to be edited.

The `"files":[]` section of the manifest references the files the package needs to bundle.
Complete information about the [package manifest can be found here](https://github.com/eccenca/cmem-package-template/tree/main#package-manifest) and more specifically [how to declare new files](https://github.com/eccenca/cmem-package-template/tree/main#adding-files).

For our example we will be adding a query graph and a project file, make sure you use valid `file_path` relative to your package working directory (nested folder).

### Adding graph and project

```json
"files": [
{...},
{
"file_path": "queries.ttl", // inside my-package-id nested folder
"file_type": "graph",
"graph_iri": "https://my-company.org/queries/",
"import_into": [],
"register_as_vocabulary": false
},
{
"file_path": "project.zip", // inside my-package-id nested folder
"file_type": "project",
"project_id": "MyProject_78e981443900a761"
}
]
```

## Testing your package

To ensure the package correctly detects your added files, you can try to import it in a Corporate Memory instance.

The package template comes with a predefined `Taskfile.yaml` allowing you to wrap your development steps in single commands.
```shell
task: Available tasks for this project:
* build: Build package archive
* check: Run whole test suite
* clean: Removes dist, *.cpa, ...
* delete: Delete (uninstall) package from Corporate Memory
* export: Export package content from Corporate Memory
* import: Import (install) package to Corporate Memory
* publish: Publish package archive to the marketplace
```

To tell the package system to take files from the local working directory and to import them inside Corporate Memory we use **task import**.
An import will always try to uninstall previously installed version of the same package, to ensure it is correctly replaced.

```shell
task import
task: [delete] cmemc package uninstall $package_id
Package 'my-package-id' is not installed.
task: [import] cmemc package install --input $package_dir
Installing package 'my-package-id' from 'my-package-id' ... done
```

!!! Warning "Importing duplicated content"

If you extracted from Corporate Memory already existing content and added it to your package with the same identifiers (graph URIs, projects IDs, ...), and try to import it back in the form of a new package, the instance might raise a `MarketplacePackagesImportError` due to conflicting elements : (Repository item 'https://my-company.org/queries/' already exists.) .

In this case, you can simply delete your duplicated content inside Corporate Memory (make sure you do backups in case) before importing them back as a package content.
The difference will be that now Corporate Memory will know this content is part of a managed package, and will handle import/export on that file from now on.

## Updating the package file content

If you make modifications to your package content in Corporate Memory, the files will not automatically sync back with your local working directory.

To extract all the updated content from Corporate Memory into your package working directory in a managed way, simply run **task export**

```shell
task export
```

!!! Warning "Exporting without installing first"

The platform can only export updated versions of package files that were imported at least once before. If you create new information directly in Corporate Memory that the package manifest doesn't yet declare, such as new graphs, you need to manually add them to your working directory and to your manifest and then import them.

The rule of thumb is : if you need to make structural change to your package that requires for you to edit your manifest, then make sure to run `import` right after to let Corporate Memory keep track of new files.

Adding a workflow inside a project is not impacted by this limitation, since it is part of the "project" that is managed and tracked by the package.

## Building your package

To generate a `.cpa` file ready to be distributed and installed in different Corporate Memory instances you can run **task build**.

Make sure your local package folder is a git repository with a clean state (the commit hash is used to generate the cpa).
```shell
task build
```

To check how this output `.cpa` file can be imported in different places refer to the [Installation and Usage section](../installation/index.md).

!!! Success "Next steps"

There are many improvements you can add to your package, such as declaring dependencies to other plugins or packages, to ensure your `.cpa` file can be installed with all it's requirements everywhere, for that you can refer to existing package examples or the template documentation.

The final step is usually publishing a version of the package to a remote Marketplace repository, to avoid having to manually transfert the `.cpa` archive. This requires you to have publishing permissions on an eccenca Marketplace instance (either a public or private instance).
This can be done with task publish, either manually or from a CI runner.
33 changes: 33 additions & 0 deletions docs/develop/packages/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
---
title: "Marketplace Packages: Overview"
icon: material/package
tags:
- Marketplace
- Package
hide:
- toc
---
# Marketplace Packages

A package is a bundle that groups different parts of Corporate Memory configuration — data, workflows and dependencies — letting you share and reuse them across projects, teams, and different Corporate Memory instances.

A package is packed into a `.cpa` file, a zip-based archive you can either hand off directly or distribute through a Marketplace, a central repository that supports pushing and
pulling packages.

The following pages give an overview about this feature:

<div class="grid cards" markdown>

- :material-download-circle-outline: [Installation and Usage](installation/index.md)

---

Intended for Linked Data Experts and Deployment Engineers, this page outlines how to install and use existing marketplace packages.

- :material-code-json: [Development](development/index.md)

---

Intended for Developers, this page explains how to start developing your own packages.

</div>
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
53 changes: 53 additions & 0 deletions docs/develop/packages/installation/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
---
title: "Marketplace Packages: Installation and Usage"
icon: material/download-circle-outline
tags:
- Package
- Marketplace
---
# Installation and Usage of Marketplace Packages

## Installation

If you want to install a marketplace package, you need to use cmemc's package command group.

You can either install from a remote marketplace or from a locally obtained `.cpa` file

```shell-session title="Install a package from marketplace"
$ cmemc package install w3c-xsd-vocab
Installing package 'w3c-xsd-vocab' from marketplace ... done
```

```shell-session title="Install a package from cpa file:"
$ cmemc package install --replace --input my-package-v0.0.0-4b7516f.cpa
Installing package 'my-package' from 'my-package-v0.0.0-4b7516f.cpa'
done
```

## Usage

Depending on the content types inside it, an installed package can appear in different places in Corporate Memory, with each item (graph, workflows, projects, ...) surfacing in its respective component.

<div style="clear: both" markdown>

!!! info inline ""

![Example: Graphs](example-vocabulary.png "Example: Graphs")

**Graphs** such as data graphs but also **Vocabularies** or **Shapes Catelog** are listed in [**Explore > Graphs**](../../../explore-and-author/graph-exploration/#graphs).

</div>

<div style="clear: both" markdown>

!!! info inline ""

![Example: Projects](example-project.png "Example: Projects")

**Projects** are imported inside [**Build**](../../../build/introduction-to-the-user-interface#projects).
When you install your first project package, Corporate Memory also creates a special project to store all installed files.
This folder is automatically managed by the package system, and removed once the last package is uninstalled.

</div>

<div style="clear: both" />
2 changes: 1 addition & 1 deletion docs/develop/python-plugins/installation/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ tags:
---
# Installation and Usage of Python Plugins

Plugins are a released as parts of Python packages.
Plugins are released as parts of Python packages.
They can but do not need to be open-source and published on [pypi.org](https://pypi.org/search/?q=%22cmem-plugin-%22) (a widely used Python Package Index). One package can contain multiple plugins.

## Installation
Expand Down
Loading