[install-help]: The initial indexing is still running.

[cdstg]

NexcCloud AIO 33.0.5
context_chat_backend 5.3.0
HARP

Describe the issue
The Docker container is there and is communicating to the appAPI but saying
The initial indexing is still running
Documents in vector database 122 (0 sent)

The stat command says

docker exec --user www-data -it nextcloud-aio-nextcloud php occ context_chat:stats
ContextChat statistics:
The indexing is not complete yet.
Total eligible files: 218
Files in indexing queue: 219
New files in indexing queue (without updates): 219
Queued documents (without files):array (
)
Files successfully sent to backend: 0
Indexed documents: array (
'files__default' => 122,
)
Actions in queue: 0
File system events in queue: 0

When I run
docker exec -u www-data nextcloud-aio-nextcloud php occ background-job:worker -t 60 'OC\TaskProcessing\SynchronousBackgroundJob'

It runs but nothing seems to happen

I did run docker exec --user www-data -it nextcloud-aio-nextcloud php occ context_chat:scan admin
and that is how that first 122 show up

But how to get it too complete or maybe its not doing the embedbing

Any ideas ?

Thanks

Screenshots
If applicable, add screenshots to help explain your problem.

Server logs (if applicable)

Details

``` ```

Context Chat Backend logs (if applicable, from the docker container)
init: embeddings required but some input tokens were not marked as outputs -> overriding
decode: cannot decode batches with this context (calling encode() instead)
init: embeddings required but some input tokens were not marked as outputs -> overriding
decode: cannot decode batches with this context (calling encode() instead)
init: embeddings required but some input tokens were not marked as outputs -> overriding
llama_perf_context_print: load time = 113.30 ms
llama_perf_context_print: prompt eval time = 482.88 ms / 96 tokens ( 5.03 ms per token, 198.81 tokens per second)
llama_perf_context_print: eval time = 0.00 ms / 1 runs ( 0.00 ms per token, inf tokens per second)
llama_perf_context_print: total time = 491.20 ms / 97 tokens

```

Setup Details (please complete the following information):

• Nextcloud Version: [eg. 28.0.1, master]
• AppAPI Version: [eg. 2.0.1]
• Context Chat PHP Version [e.g. 1.0.0]
• Context Chat Backend Version [e.g. 1.0.0]
• Nextcloud deployment method: [eg. Apache bare-metal]
• Context Chat Backend deployment method: [eg. manual, one-click]

[kyteinsky]

hello,
was there a re-install/disable+re-enable of the context_chat php app at some point?
it seems files were added again at that point.

but regarding the main issue of files not being indexed, is cron running on the system?
the background job logs related to the indexing can be found at "<data_directory>/context_chat.log" or from the Admin settings -> Assistant -> Context Chat -> Download the PHP App logs.

the logs can contain sensitive info so only post the sanitised error logs here.

that said, we have a new version in the beta that moves away from the background job system and makes the ex-app pull the files from the nextcloud instance at its own pace. It would be better if you could wait a while until the stable release v5.4.0 lands.

[cdstg]

Ok I will wait till the 5.4 is out thanks

[kriegerse]

@kyteinsky is there an ETA available for the 5.4 release?

I'm waiting for a fix included in main (#276) ending up in a similar issue described above. Indexing got stuck at a document too large to process. The upstream AI Provider behind bifrost ai proxy rejects due to too large payload in submitted "input": [ ].

[kyteinsky]

@kriegerse no concrete ETA but it should land this or the next week.