More configurable timeout values in config.toml - #18
Conversation
…fig values in order to support very long-running requests
…ions struct and default value tag in Config struct.
|
Thanks for the PR — making these timeouts configurable is the right call, and the defaults matching the old hard-coded values keeps existing setups safe. A few asks before merging:
Happy to merge after that. Thanks again! |
Following my earlier PR #15 here is another change that removes more hard-coded timeout values from the code and moves them into the config.toml.
I also de-duplicated the duration parsing logic by moving it into a new reflection-driven helper function
parseDurationsthat reads the existingtomland newly addeddefaultstruct tags. All timeouts, intervals and durations have been moved into a composite structProxyConfig.Durationsand the code has been refactored accordingly.This fixes an issue in conjunction with my local llama-cpp instance when running very long LLM sessions where creating an answer by the LLM might take over 10 minutes (600 seconds). llama-cpp would always show "cancel task" in the log files, causing the output to be truncated and agent to re-iterate over and over again (and the advisor-LLM in omp to get "angry" LOL).
Thank you for this project! You are saving a lot of power keeping my space-heater server "Deepthought" powered down most of the time. :D