IBM Watson Assistant test harness

Install

python -m venv .venv
source .venv/bin/activate
pip install -r requirements.txt
cp config.example.ini config.ini

Configure

Set your workspace ID:

export SPACE_ID="your-workspace-id"

Then edit config.ini.

For IAM auth:

[watson]
auth_type = iam
apikey = your-api-key
url = https://api.us-south.assistant.watson.cloud.ibm.com
version = 2021-11-27

For basic auth:

[watson]
auth_type = basic
username = your-username
password = your-password
url = https://api.us-south.assistant.watson.cloud.ibm.com
version = 2021-11-27

Usage

Fetch example utterances for all intents:

python getallmessages.py

Run baseline tests:

python sendallmessages.py baseline

Run altered-message tests:

python sendallmessages.py lowercaseall
python sendallmessages.py removeallpunctuation

Extract examples by word count:

python extractwordsmatchingcount.py 1
python extractwordsmatchingcount.py 2

Compare two JSON outputs:

python jsoncompare.py report/workspace-a/baseline-intent-a.json report/workspace-a/lowercaseall-intent-a.json