AI Chat Assistant's Memory State Improvement
Budget: $30 – $250 USD
Have desktop AI chat assistant running local LLMs offline. It's running a Flask web server with a pywebview desktop GUI wrapper, using llama-cpp-python for GGUF model loading and an adaptive AI system that automatically selects models from performance tiers (minimal/low/medium/high) based on hardware detection. The app uses conversation context tracking with topic analysis, reference resolution, and memory monitoring to switch between models when RAM gets tight.
Lloading bigger models (like 24B parameter models in the high tier) is failing or crashing - the RAM checks and fallback logic exist but bigger models won't load reliably, possibly due to memory allocation issues during loading, GPU
layer configuration problems, or the model switching logic interfering with initial loads.
And the small model (tinyllama-1.1b-chat-v1.0.Q4_K_M) is extremely not smart.
Lloading bigger models (like 24B parameter models in the high tier) is failing or crashing - the RAM checks and fallback logic exist but bigger models won't load reliably, possibly due to memory allocation issues during loading, GPU
layer configuration problems, or the model switching logic interfering with initial loads.
And the small model (tinyllama-1.1b-chat-v1.0.Q4_K_M) is extremely not smart.
Related categories:
Python
Machine Learning (ML)
Flask
Chromium
Desktop Application
AI Chatbot Development
AI Model Development