Samuele Lorefice 4167c75279 Added rop of pending updates on bot start, reset command, AnswerChat method, GPU offload, limit to response lenght, context reduced to 2048, flash attention, 4 parallel decode queues, --keep of the original 810 tokens (which is the starting prompt)

2024-12-26 03:24:56 +01:00

..

.idea

Base commit

2024-12-24 23:08:08 +01:00

Docker

Added rop of pending updates on bot start, reset command, AnswerChat method, GPU offload, limit to response lenght, context reduced to 2048, flash attention, 4 parallel decode queues, --keep of the original 810 tokens (which is the starting prompt)

2024-12-26 03:24:56 +01:00