Feat/configurable summary system prompts

[astkwcz]

Configurable system prompts for summary tasks

This PR adds a functionality for Summary Provider to define custom system prompt and distinguish them based on two cases:

• simple summary in Nextcloud Assistant UI (summary of docs, articles etc)
• meeting specific summary for Talk recordings (purpose of the meeting, participants, arrangements etc)

Those summaries have different context, so different model's instructions are needed

How to provide system prompts?

I created two additional functions to read a value from from config keys. In this case: system_prompt_summary and system_prompt_talk_summary. To set those values you have to use occ command:

php occ config:app:set integration_openai system_prompt_summary --value="Configurable summary system prompt"

and

`php occ config:app:set integration_openai system_prompt_talk_summary --value="Configurable talk summary system prompt"`

If the value of the configurable system prompt is empty string "" or there is no key system_prompt_summary/system_prompt_talk_summary, a system prompt for summary will have default value.
Default value is a value of the summarySystemPrompt before my changes: You are a helpful assistant that summarizes text in the same language as the text. You should only return the summary without any additional information.

Distinguish if a summary is for a Talk meeting

The idea is to pass a flag 'talk_meeting' that equals to 1 in an array passed in Task object. Default value in this PR sets it to 0: 'talk_meeting' => 0. I am going to make a PR to nextcloud spreed repo as well.

Naming convention

Names of the keys that are used to set prompts are similar to naming convention in Summary Provider that was ealier introduced- summarySystemPrompt. I start with a prefix system_prompt because I think it is good idea to provide configurable system prompts for other tasks as well, for example translation. It will be easier to read config:app if system_prompts will be written one above/belowe another (keys are sorted).

Tests

• testSummaryProvider - tests default functionality (no system prompt set using occ) of the changes. It ensures that the default summary system prompt is the same as it was before my changes
• testSummaryProviderConfigurableSystemPrompt - tests if configurable system prompt works. I set key system_prompt_summary value and delete it even if the test fails
• testSummaryProviderConfigurableTalkSystemPrompt - the same as the above + I pass 'talk_meeting' flag in the request

Question for devs

I am currently running nc34 and integration_openai version 4.5.2. Where should I also make PR to have configurable system prompts in my case?

Additional PR (spreed)

After review/changes I will make PR to spreed repo to enable passing a flag 'talk_meeting'.

Thoughts

I am aware of additional instructions for LLM that are set by assigning a value to format and complexity. It affects model output but it is still pretty much hard-coded. The main issue is that there is no possibility to have specific instruction for a specific context of summary task. In this case - call meeting on Talk (summary from a transcript). Sample output format:

1. Meeting purpose (sentence)
The purpose of a meeting ...
2. Time and place (sentence)
 ...
3. Participants (bullet points)
- Name Surname - works in IT department
- ...
4. Arrangements (bullet points)
- person X has to do Y
- ...

Configurable system prompt will provide a possibility to customize the output freely.

I am also aware that devs have a vision for Summary Provider and its output by using specific instructions for:

• format - auto, sentence, paragraph, bullet_points
• complexity - simple, medium, complex
  The output format is controlled in some way. Changes in this PR give full control to the admin of nextcloud instance.

I see the potential conflict between changes that I introduced in this PR and existing intended way to influence the output in specific, controlled way (format, complexity). I am open to changing and adjusting the implementation and hope we could work something out.

[ADNAWB]

talk_meeting suggests sth related to meeting content, it should be is_talk_meeting or is_from_talk_meeting for boolean

[ADNAWB]

We could even make it more general with sth like: 'source': '<enum>'

[ADNAWB]

Flag indicating whether ... sounds more natural and concise

[lukasdotcom]

Hi @astkwcz,

Sorry I missed this pr somehow. I don't think this is the right way of implementing this for talk as all of these would need to be done for the other providers such as llm2. This is also more of a hack anyway and should be handled in talk and not the providers for summary. IMO the better option for completely custom summary instructions specific to talk is to use the text2text task type in talk instead and pass the custom instructions and transcript there. The talk_meeting flag would also look really weird in the assistant's UI and could confuse users.

[astkwcz]

@lukasdotcom

IMO the better option for completely custom summary instructions specific to talk is to use the text2text task type in talk instead and pass the custom instructions and transcript there.

I understand, just to confirm - in this case we are losing chunking mechanism for Talk summary and the whole transcript should fit in max_tokens limit, otherwise it will fail to summarize. Chunking mechanism does not let that happen. I will make PR to spreed repo and see if they accept this change. This change will solve issue for Talk summary system prompt/instruction.

The talk_meeting flag would also look really weird in the assistant's UI and could confuse users.

I was not aware that it also appears in UI. I am looking at it right now in Assistant's UI and you are right, it confuses me.

Following custom instructions for Nextcloud Assistant:

1. What do you think about adding system_promptfor each of Task types (in this case - Summarize) that can be configurable in Admin Settings (OpenAI and LocalAI Integration section) as well as in Nextcloud Assistant's UI? It's default value will be equal to current default value.

2. The other way of implementing custom instructions is to pass system_prompt (as I am now passing talk_meeting flag) in request. TaskTypes does not support system_prompt in Input shape and we will have to extend it for text2text:summary, but as you said it will have to be done for other providers as well, for example llm2. This solution requires more changes, but will work for Nextcloud Assistant, Talk etc, just pass instructions in system_prompt field.

I think option number 1 is better. I would be happy to implement configurable system prompts for Summarize, Translate and other Tasks.

[lukasdotcom]

Hi @astkwcz,

I had an internal talk with my team and we decided the best approach would be implementing this in talk with a text2text task type. If you need help contacting someone from the talk team for that pr I can get you in contact with someone.

[github-actions]

Hello there,
Thank you so much for taking the time and effort to create a pull request to our Nextcloud project.

We hope that the review process is going smooth and is helpful for you. We want to ensure your pull request is reviewed to your satisfaction. If you have a moment, our community management team would very much appreciate your feedback on your experience with this PR review process.

Your feedback is valuable to us as we continuously strive to improve our community developer experience. Please take a moment to complete our short survey by clicking on the following link: https://cloud.nextcloud.com/apps/forms/s/i9Ago4EQRZ7TWxjfmeEpPkf6

Thank you for contributing to Nextcloud and we hope to hear from you soon!

(If you believe you should not receive this message, you can add yourself to the blocklist.)

[astkwcz]

Hi @lukasdotcom

Thanks for insights, I just created PR to spreed repo: nextcloud/spreed#19111

I used text2text task type as you and your team suggested.

I would really appreciate if you could get me in contact with someone from Talk team regarding this PR.