How to verify tika is indexing files

[spotlesscoder]

I added some files (PDFs, ZIP files and many others) to my opencloud instance via the windows sync client (mainly open source modular synthesizer schematics and documentation)

I wanted to search for a text "TL072" to find a PDF that mentions that chip

No results were returned.

Then I saw that tika was not set up

So I edited .env and added :search/tika.yml and ran docker compose up -d again
Tika server started successfully.

I waited for about 10 minutes but I could neither see a substantial CPU load (Intel G5420 CPU - should be quite busy when indexing, right) nor find the document

[michaelstingl]

The reason Tika doesn't index existing files is that opencloud search index --all-spaces skips files that are already in the index with an unchanged modification time (see the mtime check in service.go). Since the files haven't changed, the Tika extractor is never called for them.

Workaround: Delete the existing search index before re-indexing:

docker compose down
# Delete the Bleve search index (default path inside the container)
docker compose run --rm opencloud rm -rf /var/lib/opencloud/search
docker compose up -d
docker compose exec opencloud opencloud search index --all-spaces

After this, you should see CPU activity during indexing, and full-text search (e.g., "TL072") should return results.

I opened a docs PR to document this behavior: opencloud-eu/opencloud#2285

[spotlesscoder]

Thanks for the quick reply and the advice :)

I thing the second command needs to be `docker compose run --rm --entrypoint sh opencloud -c "rm -rf /var/lib/opencloud/search"

[spotlesscoder]

So actually I did this after adding opensearch now - I got the documents in the "opensearch-resource" index and when I run

GET _search
{
  "query": {
    "multi_match": {
      "query": "TL072",
      "fields": ["*"]
    }
  }
}

Using the opensearch dashboard devtools, get 141 hits but when I enter "TL072" in the opencloud search bar in my space, I get 0 results

[spotlesscoder]

However when I search for "TL431" which is mentioned in another PDF (not in the filename), it finds it

[spotlesscoder]

The max_score for TL072 is 1.4709653 - is that too low?