Skip to content

Commit 2bc7c70

Browse files
jaydeepsingh25jaydeepsingh25
authored
Merge pull request #432 from Shared-Reality-Lab/documentation
fix: add documentation
2 parents fea60c6 + 0f040f0 commit 2bc7c70

2 files changed

Lines changed: 217 additions & 36 deletions

File tree

‎FEATURES.md‎

Lines changed: 101 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,101 @@
1+
# IMAGE Browser Extension Features
2+
3+
The IMAGE browser extension provides accessible interpretations of graphics, charts, and maps for users with visual impairments. The goal is to provide people who are blind or have low vision with a new and useful experience of internet graphics that goes beyond automatically generating alt tags.
4+
5+
## Core Features
6+
7+
### Graphics Interpretation
8+
9+
- **Rich Interpretations**: Provides detailed, context-aware interpretations of graphics that go far beyond basic alt text
10+
- **Support for Both Web and Local Graphics**:
11+
- **Web Graphics**: Right-click on any graphic on the internet to access IMAGE processing options via the context menu
12+
- **Local Graphics**: Use the IMAGE Launchpad (Alt+I) and select "Use IMAGE on a graphic on my computer" to interpret graphics saved locally on your machine
13+
- **Multiple Rendering Types**: Receive different types of interpretations tailored to the graphic content:
14+
- Detailed text descriptions
15+
- Audio renderings with spatial cues
16+
- Tactile SVG graphics for haptic devices
17+
- Specialized renderings based on graphic type and content
18+
19+
### Maps Support
20+
21+
- **Interactive Map Processing**: Specialized handling for maps found on web pages
22+
- **Spatial Information**: Convey spatial relationships and geographic information in accessible formats
23+
- **Context-Aware Descriptions**: Descriptions that focus on relevant geographic features and relationships
24+
25+
### Charts Support
26+
27+
- **Data Visualization Interpretation**: Make charts and graphs accessible through multiple modalities
28+
- **Trend Identification**: Highlight important trends, patterns, and outliers in data visualizations
29+
- **Highcharts Integration**: Special support for Highcharts-based visualizations
30+
- **Data Sonification**: Convert chart data to audio patterns where appropriate
31+
32+
### Tactile Graphics
33+
34+
- **Tactile Authoring Tool**: Load graphics into a Tactile Authoring Tool for creating tactile representations
35+
- **Monarch Integration**: Send graphics to Monarch haptic devices for tactile feedback
36+
- **SVG Rendering**: Generate SVG representations of graphics optimized for tactile output
37+
- **Tactile Simplification**: Automatically simplify complex graphics for effective tactile exploration
38+
39+
## Keyboard Shortcuts
40+
41+
The IMAGE browser extension provides several keyboard shortcuts for quick access to its features:
42+
43+
| Shortcut | Function | Description |
44+
|----------|----------|-------------|
45+
| `Alt+I` | Launch IMAGE Launchpad | Opens the IMAGE Launchpad interface with various options for processing graphics |
46+
| `Ctrl+B` | Process Focused Image | When an image is focused (selected), this shortcut processes it with IMAGE |
47+
48+
## Accessibility Features
49+
50+
- **Screen Reader Integration**: Designed to work seamlessly with screen readers
51+
- **Invisible Buttons**: Toggle-able invisible buttons that are only visible to screen readers
52+
- **Keyboard Navigation**: Full keyboard accessibility for all features
53+
- **Tabindex Management**: Automatically adds tabindex to images without it for improved keyboard navigation
54+
- **Progressive Enhancement**: Works alongside existing accessibility features rather than replacing them
55+
56+
## Customization Options
57+
58+
### Interpretation Preferences
59+
60+
- **Audio Renderings**: Toggle audio interpretations on/off
61+
- **Text Renderings**: Toggle text descriptions on/off
62+
- **Rendering Preferences**: Customize which types of renderings are prioritized
63+
64+
### Language Settings
65+
66+
- **Multiple Languages**: Support for English and French
67+
- **Automatic Language Detection**: Option to use the browser's language
68+
69+
### Server Configuration
70+
71+
- **McGill Server**: Use the default McGill University IMAGE server
72+
- **Custom Server**: Specify a custom server URL for processing
73+
74+
### Haptic Device Settings
75+
76+
- **Monarch Configuration**: Settings for Monarch haptic devices:
77+
- Title
78+
- Channel ID
79+
- Secret key
80+
- Encryption key
81+
82+
## Developer Features
83+
84+
- **Developer Mode**: Toggle additional developer options
85+
- **Preprocessing Only**: Request only the preprocessing step
86+
- **Request Only**: Download the raw request JSON
87+
- **Debug Information**: Access to additional debugging information
88+
89+
## User Experience
90+
91+
- **Progress Indication**: Visual feedback during graphic processing
92+
- **Error Handling**: Comprehensive error messages and recovery options
93+
- **First Launch Experience**: Guided introduction for new users
94+
- **Feedback System**: Built-in feedback mechanism for reporting issues or suggestions
95+
96+
## Technical Capabilities
97+
98+
- **Image Compression**: Automatic compression of large graphics (>4MB)
99+
- **Context Extraction**: Analysis of surrounding content to improve interpretation
100+
- **Local File Support**: Process graphics from local files (file:// URLs)
101+
- **Data URL Support**: Process graphics encoded as data URLs

‎README.md‎

Lines changed: 116 additions & 36 deletions
Original file line numberDiff line numberDiff line change
@@ -1,70 +1,150 @@
11
# IMAGE-browser
2-
IMAGE browser extensions & client-side code
3-
4-
To install the latest development version of the extension you'll need to follow [these instructions](https://github.com/Shared-Reality-Lab/IMAGE-browser/wiki/Installing-Development-Extensions), using this [packaged extension file](https://nightly.link/Shared-Reality-Lab/IMAGE-browser/workflows/typescript-check/main/extension.zip).
2+
IMAGE browser extensions & client-side code
53

64
## IMAGE project information
75
Please see https://image.a11y.mcgill.ca for general information about the project.
86

97
If you wish to contribute to the project, the following wiki page is a good starting point, including for those on the IMAGE project team:
108
https://github.com/Shared-Reality-Lab/IMAGE-server/wiki
119

12-
## Set Up for Chrome Extension
10+
## Extension Features
1311

14-
Clone this repository. Note that the schemas are a submodule, so you need to either get them in the initial clone, e.g.,
15-
```
12+
For a comprehensive list of features and capabilities provided by the IMAGE browser extension, please see the [FEATURES.md](FEATURES.md) file. This document details all the functionality available in the extension, including:
13+
14+
- Graphics, charts, and maps interpretation
15+
- Keyboard shortcuts
16+
- Accessibility features
17+
- Customization options
18+
- Developer features
19+
- And more
20+
21+
## Development Setup
22+
23+
### Prerequisites
24+
- [Node.js](https://nodejs.org/) (latest LTS version recommended)
25+
- [npm](https://www.npmjs.com/) (v7 or later)
26+
- Git
27+
28+
### Clone the Repository
29+
Clone this repository with submodules (for schemas):
30+
```bash
1631
git clone --recurse-submodules git@github.com:Shared-Reality-Lab/IMAGE-browser.git
1732
```
1833

19-
or else get them after you've done the initial clone (while in the root of the cloned repo on your local machine):
20-
```
34+
If you've already cloned the repository without submodules, you can initialize and update them:
35+
```bash
2136
git submodule init
2237
git submodule update
2338
```
2439

25-
Install the dependencies using NPM.
26-
Ensure you have npm v7 or later installed, ideally by using [nvm](https://github.com/nvm-sh/nvm)!
40+
### Install Dependencies
41+
Navigate to the project directory and install dependencies:
2742
```bash
28-
npm i
43+
cd IMAGE-browser
44+
npm install
2945
```
30-
This includes the [web-ext](https://github.com/mozilla/web-ext) tool for running the tool in Firefox and Chrome/Chromium and building for firefox.
31-
As the project is written in Typescript, we do use Parcel to resolve packages and convert to JavaScript first.
32-
An automatically updating parcel build can be triggered with
46+
47+
## Building the Extension
48+
49+
### Development/Test Build
50+
To create a development build with debugging enabled:
3351
```bash
34-
npm start
35-
```
36-
from the command line.
37-
To use web-ext to start, from the root of the project run
38-
```
39-
npx web-ext run -s build
52+
npm run pack:test
4053
```
41-
to open it in Firefox.
54+
This will:
55+
- Set the NODE_ENV to "development"
56+
- Use the environment variables from `.env.development`
57+
- Output the build to the `build-chrome` directory
58+
- Add "(test)" suffix to the extension name
4259

43-
A version ready to be bundled for distribution as a browser extension can be created using the
44-
```
45-
npm run pack
46-
```
47-
script.
48-
A version that directly creates an *unsigned* Firefox package can be called with
49-
```
50-
npm run build:ff
60+
### Production Build
61+
To create a production-ready build:
62+
```bash
63+
npm run pack:prod
5164
```
65+
This will:
66+
- Set the NODE_ENV to "production"
67+
- Use the environment variables from `.env.production`
68+
- Output the build to the `build-chrome` directory
69+
- Use the standard extension name without any suffix
5270

53-
A signed package can be made with Chromium by running
54-
```
55-
npm run build:chromium
56-
```
57-
where the environment variable `PATH_TO_CHROME_KEY` points to the extension's key.
71+
#### Submitting Extension Build to Chrome Web Store
72+
When preparing to submit the extension to the Chrome Web Store, follow these steps:
5873

59-
It should also be possible to just load a zipped version of the `build` directory generated by `npm run pack` in any browser supporting WebExtensions, but tools might do other things too.
74+
1. Create a production build using the instructions above.
75+
76+
2. Update the extension version in the generated `build-chrome/manifest.json` file to match the corresponding git tag of the codebase. You can view available tags [here](https://github.com/Shared-Reality-Lab/IMAGE-browser/tags).
77+
78+
3. Create a ZIP file of the `build-chrome` directory.
79+
80+
4. Submit the ZIP file to the Chrome Web Store Developer Dashboard following Google's submission guidelines.
81+
82+
This versioning approach ensures consistency between the published extension and the corresponding source code in the repository.
83+
84+
## Build System
85+
86+
### Webpack Configuration
87+
The IMAGE browser extension uses webpack as its bundling tool. The webpack configuration is defined in `webpack.config.ts` and includes:
88+
89+
- **Entry Points**: Multiple entry points for various parts of the extension (background scripts, content scripts, UI pages, etc.)
90+
- **Output Configuration**: Compiled files are output to the `build-chrome` directory
91+
- **TypeScript Processing**: TypeScript files are processed using ts-loader
92+
- **CSS Processing**: CSS files are processed using style-loader and css-loader
93+
- **File Copying**: HTML, PNG, JSON, and MP3 files are copied from the src directory to the build directory using CopyWebpackPlugin
94+
- **Environment Variables**: Environment variables from `.env.development` or `.env.production` are injected into the build using webpack.DefinePlugin
95+
96+
### Environment Variables
97+
The build process uses different environment variables based on the build type:
98+
99+
- **Development Build**: Uses variables from `.env.development`, which includes `SUFFIX_TEXT = " (test)"` to append "(test)" to the extension name
100+
- **Production Build**: Uses variables from `.env.production`
101+
102+
### Manifest Modification
103+
During the production build process, the `manifest.json` file is modified to set the extension name to `__MSG_extensionName__`, which allows for localization of the extension name.
104+
105+
## Installing and Using the Extension
106+
107+
### Production Build
108+
The official production build of the IMAGE extension is available on the Chrome Web Store:
109+
- **URL**: [https://chromewebstore.google.com/detail/image-extension/iimmeciildnckfmnpbhglofiahllkhop](https://chromewebstore.google.com/detail/image-extension/iimmeciildnckfmnpbhglofiahllkhop)
110+
- **Description**: This is the stable, production-ready version of the extension recommended for most users.
111+
- **Installation**: Click the "Add to Chrome" button on the Chrome Web Store page.
112+
113+
### Test Build
114+
The test build is a pre-release version with newer features that are still being tested:
115+
- **URL**: [https://chromewebstore.google.com/detail/image-extension-test/emmcfbcpilagikejimnbilgoihgjfdln](https://chromewebstore.google.com/detail/image-extension-test/emmcfbcpilagikejimnbilgoihgjfdln)
116+
- **Description**: This version includes features that are being tested before inclusion in the production build.
117+
- **Installation**: Click the "Add to Chrome" button on the Chrome Web Store page.
118+
119+
### Nightly Build
120+
The nightly build contains the latest changes from the main branch:
121+
- **URL**: [https://nightly.link/Shared-Reality-Lab/IMAGE-browser/workflows/typescript-check/main/extension.zip](https://nightly.link/Shared-Reality-Lab/IMAGE-browser/workflows/typescript-check/main/extension.zip)
122+
- **Description**: This build is automatically generated from the latest code in the main branch and may contain experimental features or bugs.
123+
- **Installation**:
124+
1. Download the ZIP file from the URL above
125+
2. Extract the ZIP file to a folder on your computer
126+
3. Follow the "Local Build" installation instructions below
127+
128+
129+
**Warning**: Installing different versions of the extension (production and test) in a browser at the same time can cause unexpected issues, and the extension may not function properly. It is recommended to only have one version of the extension installed at any given time.
130+
131+
### Local Build
132+
To use a locally built version of the extension:
133+
134+
#### Chrome/Chromium
135+
1. Build the extension using either `npm run pack:test` or `npm run pack:prod` as described in the "Building the Extension" section
136+
2. Open Chrome/Chromium
137+
3. Navigate to `chrome://extensions/`
138+
4. Enable "Developer mode" (toggle in the top-right corner)
139+
5. Click "Load unpacked"
140+
6. Select the `build-chrome` directory from your project
60141

61142
## EARLY BETA: Safari Extension (desktop and iOS)
62143
The Chrome extension can be converted into an extension that will run in desktop Safari, or (with a separate conversion) in Safari on iOS.
63144
It is not currently possible to generate a signed extension that is installable on iOS, or deployable to the App Store, due to restrictions on using background workers on iPhones.
64145
However, it is possible to run and test the iOS extension by installing it directly onto an iPhone via Xcode.
65146
Details are available in [Safari.md](Safari.md).
66147

67-
68148
## License
69149

70150
IMAGE project components (e.g., IMAGE browser extension and IMAGE Services), henceforth "Our Software" are licensed under GNU GPL3 (https://www.gnu.org/licenses/gpl-3.0.en.html) or AGPLv3 terms (https://www.gnu.org/licenses/agpl-3.0.txt) or later, as indicated in specific repositories or files on the project github located at https://github.com/Shared-Reality-Lab.

0 commit comments

Comments
 (0)