Aspera CLI (v4.25.1) - Multiple command failures: initdemo, browse, and download

[gsk-work]

Multiple Aspera CLI commands are failing or not working as expected on version 4.25.1 with IBM Aspera Transfer SDK 1.1.6.

Environment

• OS: Linux
• Ruby Version: 3.2.4 (2024-04-23 revision af471c0e01) [x86_64-linux]
• Aspera CLI Version: 4.25.1
• IBM Aspera Transfer SDK: 1.1.6
• ascp Version: 4.4.6.2033 31ae48a
• AES-NI: Supported

Issues Encountered

"ascli server browse /" - command failing with demoserver preset

• Expected Behavior:
  • Should browse the root directory of the demo server.
• Actual Behavior:
  • Command waiting at following line without any result (please find tail result of debug logs)

    DEBG URI=ssh://demo.asperasoft.com:33001, host=demo.asperasoft.com, port=33001, scheme=ssh
   DEBG SSH keys=[]
   DEBG ssh:aspera@demo.asperasoft.com
   DEBG ssh_options (json)=
   {
     "port": 33001,
     "password": "",
     "logger": "#<Logger:0x000071547bad89f0>",
     "verbose": "warn",
     "use_agent": false
   }
   DEBG execute_single:ls:["/"]
   DEBG cmd=ascmd
  

ascli shares files download - 401 Unauthorized Error

ascli shares files download \
  --url=https://<ASPERA_SERVER> \
  --username=<USERNAME> \
  --password=<PASSWORD> \
  /config.json \
  --insecure=yes

• Expected Behavior:
  • File should download successfully with valid credentials.
• Actual Behavior:
  • Authentication fails with 401 Unauthorized error.
• Error Output:
  ERROR: Rest: <ASPERA_SERVER> 401 Unauthorized

Important Note:

I was able to login to Aspera Shares UI with the above credentials and successfully download files through the web interface. This confirms that:

• The credentials are valid and working
• The server is accessible and functioning properly
• The issue appears to be specific to the CLI authentication mechanism

[laurent-martin]

1. HSTS
   I have tested as follows:

$ ascli conf init
Done
Saving config file.
$ ascli conf pre over
╭────────────┬───────────┬─────────────────────────────────╮
│ preset     │ parameter │ value                           │
╞════════════╪═══════════╪═════════════════════════════════╡
│ config     │ version   │ 4.25.1                          │
│ demoserver │ url       │ ssh://demo.asperasoft.com:33001 │
│ demoserver │ username  │ aspera                          │
│ demoserver │ password  │ 🔑                              │
│ default    │ server    │ demoserver                      │
╰────────────┴───────────┴─────────────────────────────────╯
$ ascli server br /
╭────────────┬────────┬────────┬───────┬───────────────────────────┬───────────────────────╮
│ zmode      │ zuid   │ zgid   │ size  │ mtime                     │ name                  │
╞════════════╪════════╪════════╪═══════╪═══════════════════════════╪═══════════════════════╡
│ drwxr-xr-x │ aspera │ aspera │ 57344 │ 2026-01-27 16:57:10 +0000 │ Upload                │
│ dr-xr-xr-x │ aspera │ aspera │ 4096  │ 2022-10-27 15:08:16 +0100 │ aspera-test-dir-large │
│ dr-xr-xr-x │ aspera │ aspera │ 4096  │ 2022-10-27 15:08:17 +0100 │ aspera-test-dir-small │
│ dr-xr-xr-x │ aspera │ aspera │ 4096  │ 2022-10-27 15:08:17 +0100 │ aspera-test-dir-tiny  │
╰────────────┴────────┴────────┴───────┴───────────────────────────┴───────────────────────╯

As you can see this works with version 4.25.1, so there must be something else.
In addition, those commands are parts of tests before release.

If it is waiting, it means that probably the port 33001 of the server is not accessible, try the following:

telnet demo.asperasoft.com 33001

or

nc demo.asperasoft.com 33001

In case of success, both shall display:

SSH-2.0-OpenSSH_8.9p1 Ubuntu-3ubuntu0.13

If not, it's a firewall issue on your side.
Aspera uses TCP and UDP ports 33001. (by default).

---

2. Shares
   For Shares: place single quotes around the username and password, in case there are any special shell characters in those.
   In addition, for Shares: use of API is linked to use of Shares API.
   Shares API must be enabled for users:

So, in order to be able to use the CLI with Shares, you must be granted API access by Shares' admin.

[laurent-martin]

Note

This repository is not a general purpose support channel for IBM Aspera, it is only about ascli.

A typical "single node" Aspera Shares setup is composed with:

• A Linux VM
• with Aspera Shares installed on it
• with Aspera HSTS installed on it
• with local sshd daemon configured to listen on port TCP 33001.

So the typical ports in play are:

• TCP/443 for HTTPS for the web UI of Shares
• TCP/33001 for session contral of Aspera transfers (HSTS)
• UDP/33001 for file transfer (FASP, HSTS)

so, those ports must be reachable from client apps to the server.
This is true for ANY client app, including:

• a web user using browser + web client (Aspera Connect or Aspera for Desktop)
• a CLI user using ascli (which uses the aspera transfer sdk)

Another way to test ssh connectivity:

ssh -v xfer@demo.asperasoft.com -p 33001

In case of successful connection, the output includes:

...
debug1: Connecting to demo.asperasoft.com port 33001.
debug1: Connection established.
...
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.9p1 Ubuntu-3ubuntu0.13
...

You can try from another server that would have full internet access (or laptop, etc...)

If you don't get the banner it means that something in the middle is doing something... (firewall).
Nevertheless, we can see in the output you gave:

Trying 52.116.196.54...
Connected to demo.asperasoft.com.
Escape character is '^]'.

Meaning that a TCP connection is done, but the banner does not come back...

[laurent-martin]

Can you confirm you have solved your issue ?

[gsk-work]

Hi @laurent-martin
Yes, your suggestions resolved my main query.

In the CLI documentation for authentication, it is mentioned to use a username and password. However, I had forgotten that we have already moved to access‑key–based authentication for downloads. After updating the authentication method from the username/password (used for Aspera Shares UI login) to the access ID and access key–based API authentication, everything worked correctly.

I feel this should be documented somewhere for clarity.

[gsk-work]

We are experiencing a permissions enforcement issue where the Aspera CLI (ascli) does not honor the permissions that are configured through the Aspera Shares API. Specifically, users are able to perform unauthorized operations (mkdir, upload) via the CLI, even though these permissions are explicitly disabled in their share configuration.

My application portal creates Aspera accounts and shares via Aspera APIs, and permissions are granted exclusively via the Aspera Shares API with the following restrictions:
✅ Browse - Enabled
✅ Download - Enabled
❌ Upload - Disabled
❌ Create/Mkdir - Disabled
❌ Manage - Disabled

• Expected :
  • Aspera CLI should inherit and enforce the permissions defined through Aspera Shares API
  • Users should only be able to execute browse and download operations via CLI
• Actual:
  • Aspera Shares web UI correctly enforces permissions (users cannot create directories or upload files)
    However, Aspera CLI allows execution of restricted commands:
    • ascli shares repo mkdir - Successfully creates directories
    • ascli shares repo upload - Successfully uploads files
    • Potentially other unauthorized operations from the full feature set: delete, mkfile, mklink, rename, sync, etc.

Verification Performed

• Confirmed user permissions in admin portal show only browse and download enabled
• Tested with test account and verified mkdir/upload operations succeed via CLI
• Confirmed same operations are correctly blocked in Aspera Shares web UI

Questions

• Does ascli inherit permissions from Aspera Shares API at all? If not, what is the recommended approach to enforce share-level permissions for CLI access?
• How can we restrict CLI operations to match our share permissions? We need to limit users to browse and download only.

Any guidance on properly configuring permission enforcement between Aspera Shares and the CLI would really help us.

[laurent-martin]

ascli uses Aspera products APIs just like any other API user would do.
So, enforcement must be done at application level (not ascli level).

If some enforcements are not done correctly at application level, then a support ticket shall be entered to IBM for that application.

In other terms, even if anything was done in ascli, that would not prevent users to use API directly...

[gsk-work]

Understood @laurent-martin

The following api has been invoking internally from CLI for upload action.
DEBG URI=https:/<my-aspera-domain>com/node_api/files/upload_setup

so I will ask IBM aspera support team to look into this.

[laurent-martin]

Note the following, here a demo Aspera Shares (v1.10.0.184434 Patch level 9 ):

curl -si https://shares.aspera-emea.com/node_api/ping | grep X-Powered-By

Output is:

X-Powered-By: Phusion Passenger(R) 6.0.12

Check on your Shares server, it does not anwser the same, meaning there is probably a reverse proxy in front.

The end points under /node_api/ shall go to Shares, and NOT the HSTS node api.

[laurent-martin]

If you are front-ending both HSTS Node API and Shares with a common reverse proxy, for example using nginx, then the endpoint /node_api shall NOT be directed to the HSTS node API (port 9092), it is reserved for Shares, and shall be directed to the Shares server on /node_api.

The confusion probably comes from this:

• the /node_api end points of Shares behave like the actual node API of HSTS, but it is Shares.

[laurent-martin]

here an example of a nginx configuration I use on a server where Aspera services are ALL served on port 443:

# Aspera configuration - reverse proxy for components
# nginx 1.25
user nginx;
worker_processes auto;
error_log /var/log/nginx/error.log;
pid /run/nginx.pid;

events {
  worker_connections 1024;
}

http {
  # Set to 1 if request is from browser
  map $http_user_agent $is_browser {
    default         0;
    "~*(edge|mozilla|chrome|firefox|gecko|opera|safari|applewebkit|lynx|libwww)"  1;
  }
  # Set to 1 if request contains query "context" (faspex)
  map $arg_context $has_context {
    default         0;
    "~^."           1;
  }
  server {
    # ports of backends
    set                         $node_port 9092;
    set                         $faspex_port 10443;
    set                         $fallback_port 9097;
    set                         $wss_port 9093;
    set                         $httpgw_port 9443;
    set                         $faspio_port 9098;
    #listen                    443 quic reuseport default_server;
    listen                      443 ssl;
    listen                      [::]:443 ssl;
    #http2                      on;
    #http3                      on;
    #quic_retry                 on;
    #ssl_early_data             on;
    server_name                 _;
    root                        /opt/localroot;
    ssl_certificate             /etc/letsencrypt/live/myserver.example.com/fullchain.pem;
    ssl_certificate_key         /etc/letsencrypt/live/myserver.example.com/privkey.pem;
    ssl_session_cache           builtin:1000 shared:SSL:10m;
    ssl_protocols               TLSv1.2 TLSv1.3;
    ssl_ciphers ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:RSA+AESGCM:RSA+AES:!aNULL:!MD5:!DSS;
    ssl_prefer_server_ciphers   on;
    access_log                  /var/log/nginx/global.access.log;
    server_tokens               off;
    proxy_set_header            Host              $host;
    #proxy_set_header            Host              $hostname:$server_port;
    proxy_set_header            X-Real-IP         $remote_addr;
    proxy_set_header            X-Forwarded-For   $proxy_add_x_forwarded_for;
    proxy_set_header            X-Forwarded-Proto $scheme;
    proxy_read_timeout          90;
    proxy_buffering             off;
    proxy_request_buffering     off;

    # Static files
    location ~ ^/(robots\.txt|index\.html|wallpaper\.jpg|favicon\.ico)$ {
      try_files   $uri =404;
      #add_header  Alt-Svc 'h3=":$server_port"; ma=86400';
      #add_header  x-quic 'h3';
      autoindex   on;
    }
    location /download/ {
      try_files $uri $uri/ =404;
      autoindex on;
    }
    # Faspex, local main page, HSTS Node API
    location = / {
        if ($has_context) {
          proxy_pass https://127.0.0.1:$faspex_port;
          break;
        }
        if ($is_browser) {
          rewrite ^/$ /index.html last;
        }
      proxy_pass                https://127.0.0.1:$node_port;
    }
    # HSTS: node API
    location / {
      proxy_pass                https://127.0.0.1:$node_port;
      access_log                /var/log/nginx/node.access.log;
    }
    # HSTS: HTTP Fallback (Legacy)
    location ~ ^/aspera/https?/ {
      proxy_pass                https://127.0.0.1:$fallback_port;
      access_log                /var/log/nginx/fallback.access.log;
    }
    # HSTS: Web Socket Session (No easy way to use: ATT-1565): HTTP same as node
    location /v1/transfer/start {
      proxy_pass                https://127.0.0.1:$wss_port;
      access_log                /var/log/nginx/wss.access.log;
      proxy_http_version        1.1;
      proxy_set_header          Upgrade $http_upgrade;
      proxy_set_header          Connection "Upgrade";
      proxy_set_header          Host $host;
    }
    # HTTP Gateway
    location /aspera/http-gwy {
      proxy_pass                https://127.0.0.1:$httpgw_port;
      access_log                /var/log/nginx/httpgw.access.log;
      proxy_http_version        1.1;
      proxy_set_header          Upgrade $http_upgrade;
      proxy_set_header          Connection "Upgrade";
      proxy_set_header          Host $host;
    }
    # faspio
    location /faspio/ {
      rewrite ^/faspio/(.*) /$1 break;
      proxy_pass                http://127.0.0.1:$faspio_port;
      proxy_pass_header         Server;
      access_log                /var/log/nginx/faspio.log;
    }
    # Faspex 5
    location ~ ^/(aspera/faspex|aspera/connect|public|config.js|hls.js|assets|favicons|httpGateway|images|javascripts|static|stylesheets|swfs|system) {
      proxy_pass                https://127.0.0.1:$faspex_port;
      access_log                /var/log/nginx/faspex.access.log;
      add_header                Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;
      add_header                Front-End-Https on;
    }
  }
}

Using this configuration, the following products are served:

• Faspex 5
• HSTS node API
• HTTP Gateway
• HTTP Fallback
• faspio
• WSS for ascp
• local web pages

Note: I do not tell to use the above, but this is to illustrate that the various products can be more or less routed by path.

In the specific case of Shares and HSTS Node API:

• if node api is hosted on the same server: just use any path that is NOT used by Shares, i.e. NOT /node_api, for example use /nodeapi or hstsnode, etc...
• one can check the paths used by shares in: /opt/aspera/shares/u/shares/config/routes.rb

Caution

All above does not concern ascli, ascli is only using the various products APIs...

[gsk-work]

Hi @laurent-martin

Thank you for detailed explanation, My CI/CD team is going through your input on nginx, Mean while I have something to share here to get your validation.

I just reviewed the Aspera Shares API documentation again. The Shares API mainly focuses on administrative tasks such as:

• User management (creating new users)
• Share management (creating new shares and setting permissions)

In our production code, we are using the following endpoints for these tasks:

• Create local user
  • https://my-aspera-domain.com/api/v1/data/local_users
• Create share
  • https://my-aspera-domain.com/api/v1/data/shares
• Authorize share
  • https://my-aspera-domain.com/api/v1/data/shares//user_permissions

For the Aspera Node API, the documentation which available at https://developer.ibm.com/apis/catalog/aspera--aspera-node-api/Introduction

In the introduction, it states:

The Node API includes the following features and functionality: An HTTPS (by default port 9092) and HTTP (by default port 9091) interface.

This gave me the impression that the Node API belongs to the service running on port 9092.

That is exactly how we have set it up. Both Aspera Shares and the Node API (on port 9092) are hosted on the same machine, which is bound to the DNS name . To differentiate between the routes for port 9092 and the actual Shares routes, we introduced a custom prefix.

We selected node_api/ as the differentiator. Therefore, all the APIs documented in the Aspera Node API (such as those responsible for creating access keys and browsing files) are prefixed with node_api in our backend code.

[laurent-martin]

Unfortunately, the IBM documentation is incomplete when it comes to the Shares API.

While administrative APIs are indeed exposed under /api/v1, transfer-related APIs are located under /node_api (both served by the Shares app). These APIs are essentially equivalent to the HSTS node API.

The underlying concept is that the HSTS node API provides administrative-level access to storage. Using the node API, it is possible to access all files under the associated docroot, whether via an access key or a node user.
Shares acts as a broker on top of this model. Internally, Shares relies on the HSTS node API (from one or multiple HSTS instances). Shares API users are exposed to the same API surface under /node_api, but with two key differences:

• access rights are enforced according to the Shares user’s permissions, and
• requests are transparently redirected to the appropriate HSTS instance (as HSTS is not necessarily installed on the same server as Shares).

[gsk-work]

Thank you @laurent-martin

Thank you for the detailed explanation regarding Shares and HSTS — I now have a clear understanding of the core concepts.

My team and I are still unable to locate documentation specifically for the Aspera Shares node_api/ routes. Could you please point us to the correct documentation?

We are particularly interested in the following APIs exposed through Shares:

• Browse API (/node_api/files/browse)
• Transfer specification generation API
  These are the endpoints we rely on most heavily in our workflows, so having the right reference documentation would be very helpful.

[laurent-martin]

Basically, API under /node_api in Shares is a subset of the HSTS node API:
Documentation: https://developer.ibm.com/apis/catalog/aspera--aspera-node-api/api/API--aspera--ibm-aspera-node-api#get-664199063