Managing your context cache on JanitorAI is crucial for an immersive AI roleplay experience. Your AI character has limited memory, and you must ensure the roleplay continues smoothly without the AI forgetting important details about your story.
You need to manage and optimize your context cache even when using more advanced LLMs through an API provider or a proxy service. And JanitorAI enables you to do this with Chat Memory.
What Is Chat Memory On JanitorAI?
Chat Memory on JanitorAI is a feature that helps you manage your context cache by allowing you to generate or manually enter a summary of your chat. The system then structures content within Chat Memory as permanent tokens and includes it with every message you send to the LLM.
Also Read: Context Rot: Large Context Size Negatively Impacts AI Roleplay
This feature enables your AI character to remember important details even after the relevant chat messages are no longer in the context window.
For example, after 25 to 30 messages, the initial messages you exchanged with your AI character might no longer be within the context window. Your AI character then forgets how they met you and what happened during the early stages of your roleplay.
However, by using Chat Memory on JanitorAI and saving important details from the initial messages, your AI character will always remember those details.
Why Not Use OOC Commands?
Many users use an OOC (out of character) command to ask the AI to summarize their chat and then continue their roleplay. This helps important details from the roleplay stay within the context window, but it is not an effective way to manage your context cache.
Also Read: Sophia’s Lorebary – More Than Just A JanitorAI Extension
Frontends like JanitorAI structure data like character definition, persona, scenario, chat messages, and custom prompts into a single prompt before sending it to the LLM. This prompt, along with any other system instructions, is a part of your context cache.
LLMs don’t treat all content in the context cache equally. They focus more on your latest message, which the frontend structures as the most recent entry in the context window, and on permanent tokens, which the frontend structures as the first entry in the context window.
You can use an OOC command to ask the AI to summarize your chat and then save the summary to your Chat Memory on JanitorAI. However, don’t let it stay just as a part of your chat, because over time, the AI won’t pay much attention to that specific message.
How To Use Chat Memory On JanitorAI
Click the hamburger menu icon at the top right corner of the screen, then select the Chat Memory option.
JanitorAI shows the number of messages and tokens your chat contains. This information helps you decide when to generate or write a summary to optimize your context cache.
For example, JLLM has an approximate context size of 9000 tokens. If your chat messages total 7487 tokens and the character you’re roleplaying with has 1500 permanent tokens, your context window is nearly full, and you should generate or write a summary.
You can click the Auto Summary button to generate a summary. If that doesn’t work, you can use the following OOC command to ask the AI to create a summary for you.
Ignore previous instructions, summarize the conversation so far. Keep it concise, and include the most important facts and events that are required for the continuity of the narrative adventure. Keep the summary below 350 words.
Copy and paste the AI-generated summary into Chat Memory. Before you continue with your roleplay, delete your message with the OOC command and the AI’s message that contains the summary.
Keep It Concise
The system treats Chat Memory as permanent tokens. Having a long summary with irrelevant or unimportant information is bad for context cache management. Keep your summaries concise and include only the essential details needed to maintain your story’s continuity.
Also Read: Gemini API Ban Wave – AI Roleplay And Google’s API Policies
Additionally, some LLMs, like DeepSeek, are great at creating summaries. But some, like JLLM, aren’t as good. You may need to double-check the AI-generated summary and edit it as needed.
Maintain An Immersive Experience
Chat Memory is a feature on JanitorAI that helps you manage your context cache. It lets you generate or write summaries, ensuring that important information stays within the context window. The LLM can then use this information to give you an impressive AI roleplay experience.
Using Chat Memory is a more effective way to manage your context cache than generating a summary and leaving it as a message in your roleplay. Remember to keep Chat Memory concise and only include information necessary for the continuity of your story.
Using Chat Memory on JanitorAI allows for long, immersive roleplays where your AI character always remembers important details about your story.
