Compare commits
No commits in common. "main" and "v2.1" have entirely different histories.
13 changed files with 133 additions and 289 deletions
30
README.md
30
README.md
|
|
@ -1,6 +1,6 @@
|
|||
# Eucalyptus Chat
|
||||
|
||||
A frontend for large language models like [🐨 Koala](https://bair.berkeley.edu/blog/2023/04/03/koala/) or [🦙 Vicuna](https://lmsys.org/blog/2023-03-30-vicuna/) running on CPU with [llama.cpp](https://github.com/ggerganov/llama.cpp), using the API server library provided by [llama-cpp-python](https://github.com/abetlen/llama-cpp-python).
|
||||
A frontend for [Koala](https://bair.berkeley.edu/blog/2023/04/03/koala/) running on CPU with [llama.cpp](https://github.com/ggerganov/llama.cpp), using the API server library provided by [llama-cpp-python](https://github.com/abetlen/llama-cpp-python).
|
||||
|
||||
|
||||
|
||||
|
|
@ -8,33 +8,22 @@ A frontend for large language models like [🐨 Koala](https://bair.berkeley.edu
|
|||
|
||||
- Python 3.10
|
||||
- The pip packages listed in `requirements.txt`
|
||||
- An AI model in the ggml format (should be quantized)
|
||||
- A Koala model in the ggml format (should be quantized)
|
||||
|
||||
For memory and disk requirements for the different models, see [llama.cpp - Memory/Disk Requirements](https://github.com/ggerganov/llama.cpp#memorydisk-requirements)
|
||||
|
||||
## Supported Models
|
||||
|
||||
- [🐨 Koala](https://bair.berkeley.edu/blog/2023/04/03/koala/)
|
||||
- [🦙 Vicuna v.0](https://lmsys.org/blog/2023-03-30-vicuna/)
|
||||
- [🦙 Vicuna v.1.1](https://lmsys.org/blog/2023-03-30-vicuna/)
|
||||
- [🦁 Manticore Chat](https://huggingface.co/openaccess-ai-collective/manticore-13b-chat-pyg)
|
||||
|
||||
(see `./profiles/`)
|
||||
The 7B-Model, `q4_0`-quantized, requires approx. 5 GB of RAM.
|
||||
|
||||
## Usage
|
||||
|
||||
To use Eucalyptus locally, start both the API-Server (`api-server.py`) and the Frontend-Server (`frontend-server.py`).
|
||||
The default URL of the Frontend-Server is http://localhost:8080.
|
||||
|
||||
You have to choose the correct profile for the model you use. See [Supported Models](#supported-models) and [Frontend Server CLI Argument](#frontend-server-cli-arguments).
|
||||
|
||||
### API Server CLI Arguments
|
||||
|
||||
The following command-line arguments are available:
|
||||
|
||||
* `-m` or `--model`: Specifies the path to the model file. This is required and must be provided.
|
||||
* `--host`: Specifies the address to listen on. By default, it listens on `localhost`.
|
||||
* `--port`: Specifies the port number to listen on. The default value is `7331`.
|
||||
* `--host`: Specifies the address to listen on. By default, it listens on localhost.
|
||||
* `--port`: Specifies the port number to listen on. The default value is 7331.
|
||||
|
||||
```bash
|
||||
python3 api-server.py [-h] -m MODEL [--host HOST] [--port PORT]
|
||||
|
|
@ -44,13 +33,12 @@ python3 api-server.py [-h] -m MODEL [--host HOST] [--port PORT]
|
|||
|
||||
The following command-line options are available:
|
||||
|
||||
* `--profile`: Path to the profile file for the model.
|
||||
* `--host`: Specifies the IP address or hostname to listen on. Defaults to `localhost`.
|
||||
* `--port`: Specifies the port number to listen on. Defaults to `8080`.
|
||||
* `--api`: Specifies the URL of the API server. Defaults to `http://localhost:7331`.
|
||||
* `--host`: Specifies the IP address or hostname to listen on. Defaults to "localhost".
|
||||
* `--port`: Specifies the port number to listen on. Defaults to 8080.
|
||||
* `--api`: Specifies the URL of the API server. Defaults to http://localhost:7331.
|
||||
|
||||
```bash
|
||||
python3 frontend-server.py [-h] [--profile PROFILE] [--host HOST] [--port PORT] [--api API]
|
||||
python3 frontend-server.py [-h] [--host HOST] [--port PORT] [--api API]
|
||||
```
|
||||
|
||||
## Third-Party Licenses
|
||||
|
|
|
|||
|
|
@ -4,8 +4,6 @@
|
|||
from argparse import ArgumentParser
|
||||
from os import environ
|
||||
|
||||
from llama_cpp.server.app import create_app
|
||||
|
||||
import uvicorn
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
|
@ -15,7 +13,10 @@ if __name__ == "__main__":
|
|||
ap.add_argument("--host", help="Address to listen on (default: localhost)", type=str, default="localhost")
|
||||
ap.add_argument("--port", help="Port to listen on (default: 7331)", type=int, default=7331)
|
||||
args = ap.parse_args()
|
||||
# Set environment variable before importing api server
|
||||
environ["MODEL"] = args.model
|
||||
# Import api server
|
||||
from llama_cpp.server.app import create_app
|
||||
# Run
|
||||
app = create_app()
|
||||
uvicorn.run(app, host=args.host, port=args.port)
|
||||
|
|
|
|||
|
|
@ -2,8 +2,6 @@
|
|||
# Copyright (c) 2023 Julian Müller (ChaoticByte)
|
||||
|
||||
from argparse import ArgumentParser
|
||||
from json import load
|
||||
from pathlib import Path
|
||||
|
||||
import uvicorn
|
||||
from frontend.app import app
|
||||
|
|
@ -11,31 +9,11 @@ from frontend.app import app
|
|||
if __name__ == "__main__":
|
||||
# CLI
|
||||
ap = ArgumentParser()
|
||||
ap.add_argument("--profile", help="Path to a profile file that includes settings for a specific model", type=Path, required=True)
|
||||
ap.add_argument("--host", help="Address to listen on (default: localhost)", type=str, default="localhost")
|
||||
ap.add_argument("--port", help="Port to listen on (default: 8080)", type=int, default=8080)
|
||||
ap.add_argument("--api", help="URL of the API Server (default: 'http://localhost:7331')", type=str, default="http://localhost:7331")
|
||||
args = ap.parse_args()
|
||||
# Read profile
|
||||
with args.profile.open("r") as pf:
|
||||
profile = load(pf)
|
||||
# Check profile
|
||||
assert "name" in profile
|
||||
assert "conversation_prefix" in profile
|
||||
assert "user_keyword" in profile
|
||||
assert "assistant_keyword" in profile
|
||||
assert "stop_sequences" in profile
|
||||
# Pass frontend config to the app
|
||||
app.config.frontend_config = {
|
||||
"api_url": args.api.rstrip("/"),
|
||||
"profile": {
|
||||
"name": profile["name"],
|
||||
"conversation_prefix": profile["conversation_prefix"],
|
||||
"user_keyword": profile["user_keyword"],
|
||||
"assistant_keyword": profile["assistant_keyword"],
|
||||
"separator": profile["separator"],
|
||||
"stop_sequences": profile["stop_sequences"]
|
||||
}
|
||||
}
|
||||
app.config.frontend_config = {"api_url": args.api.rstrip("/")}
|
||||
# Run
|
||||
uvicorn.run(app, host=args.host, port=args.port)
|
||||
|
|
|
|||
|
|
@ -18,57 +18,45 @@
|
|||
</button>
|
||||
</div>
|
||||
</div>
|
||||
<div class="flex sidebar-container sidebar-hidden" id="sidebar-container">
|
||||
<button id="sidebar-toggle-open" class="icon-button">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" height="48" viewBox="0 96 960 960" width="48"><path d="M433 712V440L297 576l136 136ZM180 936q-24.75 0-42.375-17.625T120 876V276q0-24.75 17.625-42.375T180 216h600q24.75 0 42.375 17.625T840 276v600q0 24.75-17.625 42.375T780 936H180Zm393-60V276H180v600h393Z"/></svg>
|
||||
</button>
|
||||
<button id="sidebar-toggle-close" class="icon-button hidden">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" height="48" viewBox="0 96 960 960" width="48"><path d="M297 440v272l136-136-136-136ZM180 936q-24.75 0-42.375-17.625T120 876V276q0-24.75 17.625-42.375T180 216h600q24.75 0 42.375 17.625T840 276v600q0 24.75-17.625 42.375T780 936H180Zm393-60V276H180v600h393Z"/></svg>
|
||||
</button>
|
||||
<div class="sidebar flex flex-column">
|
||||
<div class="max-width">Settings</div>
|
||||
<div class="settings flex flex-column">
|
||||
<div class="setting flex">
|
||||
<div>Assistant</div>
|
||||
<div id="settings-label-assistant"></div>
|
||||
</div>
|
||||
<div class="setting flex">
|
||||
<div>max_tokens</div>
|
||||
<div><input type="number" id="settings-max-tokens" min="16" value="100"></div>
|
||||
</div>
|
||||
<div class="setting flex">
|
||||
<div>temperature</div>
|
||||
<div><input type="number" id="settings-temperature" min="0.0" max="2.0" step="0.01" value="0.8"></div>
|
||||
</div>
|
||||
<div class="setting flex">
|
||||
<div>top_p</div>
|
||||
<div><input type="number" id="settings-top-p" min="0.0" max="1.0" step="0.01" value="0.95"></div>
|
||||
</div>
|
||||
<div class="setting flex">
|
||||
<div>top_k</div>
|
||||
<div><input type="number" id="settings-top-k" min="0" step="1" value="40"></div>
|
||||
</div>
|
||||
<div class="setting flex">
|
||||
<div>repeat_penalty</div>
|
||||
<div><input type="number" id="settings-repeat-penalty" min="0.0" step="0.01" value="1.1"></div>
|
||||
</div>
|
||||
<div class="setting flex">
|
||||
<div>presence_penalty</div>
|
||||
<div><input type="number" id="settings-presence-penalty" min="-2.0" max="2.0" step="0.01" value="0"></div>
|
||||
</div>
|
||||
<div class="setting flex">
|
||||
<div>frequency_penalty</div>
|
||||
<div><input type="number" id="settings-frequency-penalty" min="-2.0" max="2.0" step="0.01" value="0"></div>
|
||||
</div>
|
||||
<div class="sidepanel flex flex-column">
|
||||
<div class="max-width">Settings</div>
|
||||
<div class="settings flex flex-column">
|
||||
<div class="setting flex">
|
||||
<div>max_tokens</div>
|
||||
<div><input type="number" id="settings-max-tokens" min="16" value="100"></div>
|
||||
</div>
|
||||
<div class="flex">
|
||||
<button id="reset-settings-btn" class="icon-button">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" height="48" viewBox="0 96 960 960" width="48"><path d="M480 896q-133 0-226.5-93.5T160 576q0-133 93.5-226.5T480 256q85 0 149 34.5T740 385V256h60v254H546v-60h168q-38-60-97-97t-137-37q-109 0-184.5 75.5T220 576q0 109 75.5 184.5T480 836q83 0 152-47.5T728 663h62q-29 105-115 169t-195 64Z"/></svg>
|
||||
</button>
|
||||
<button id="reset-history-btn" class="icon-button">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" height="48" viewBox="0 96 960 960" width="48"><path d="m361 757 119-121 120 121 47-48-119-121 119-121-47-48-120 121-119-121-48 48 120 121-120 121 48 48ZM261 936q-24 0-42-18t-18-42V306h-41v-60h188v-30h264v30h188v60h-41v570q0 24-18 42t-42 18H261Z"/></svg>
|
||||
</button>
|
||||
<div class="setting flex">
|
||||
<div>temperature</div>
|
||||
<div><input type="number" id="settings-temperature" min="0.0" max="2.0" step="0.01" value="0.8"></div>
|
||||
</div>
|
||||
<div class="setting flex">
|
||||
<div>top_p</div>
|
||||
<div><input type="number" id="settings-top-p" min="0.0" max="1.0" step="0.01" value="0.95"></div>
|
||||
</div>
|
||||
<div class="setting flex">
|
||||
<div>top_k</div>
|
||||
<div><input type="number" id="settings-top-k" min="0" step="1" value="40"></div>
|
||||
</div>
|
||||
<div class="setting flex">
|
||||
<div>repeat_penalty</div>
|
||||
<div><input type="number" id="settings-repeat-penalty" min="0.0" step="0.01" value="1.1"></div>
|
||||
</div>
|
||||
<div class="setting flex">
|
||||
<div>presence_penalty</div>
|
||||
<div><input type="number" id="settings-presence-penalty" min="-2.0" max="2.0" step="0.01" value="0"></div>
|
||||
</div>
|
||||
<div class="setting flex">
|
||||
<div>frequency_penalty</div>
|
||||
<div><input type="number" id="settings-frequency-penalty" min="-2.0" max="2.0" step="0.01" value="0"></div>
|
||||
</div>
|
||||
</div>
|
||||
<div class="flex">
|
||||
<button id="reset-settings-btn" class="icon-button">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" height="48" viewBox="0 96 960 960" width="48"><path d="M480 896q-133 0-226.5-93.5T160 576q0-133 93.5-226.5T480 256q85 0 149 34.5T740 385V256h60v254H546v-60h168q-38-60-97-97t-137-37q-109 0-184.5 75.5T220 576q0 109 75.5 184.5T480 836q83 0 152-47.5T728 663h62q-29 105-115 169t-195 64Z"/></svg>
|
||||
</button>
|
||||
<button id="reset-history-btn" class="icon-button">
|
||||
<svg xmlns="http://www.w3.org/2000/svg" height="48" viewBox="0 96 960 960" width="48"><path d="m361 757 119-121 120 121 47-48-119-121 119-121-47-48-120 121-119-121-48 48 120 121-120 121 48 48ZM261 936q-24 0-42-18t-18-42V306h-41v-60h188v-30h264v30h188v60h-41v570q0 24-18 42t-42 18H261Z"/></svg>
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
<script src="/ui/main.js"></script>
|
||||
|
|
|
|||
|
|
@ -1,22 +1,27 @@
|
|||
// Copyright (c) 2023 Julian Müller (ChaoticByte)
|
||||
|
||||
const isMobile = /Android|BlackBerry|iPhone|iPod|Opera Mini/i.test(navigator.userAgent);
|
||||
(() => {
|
||||
|
||||
// Fetch configuration and initialize Eucalyptus Chat Frontend
|
||||
// Koala specific keywords
|
||||
const conversationBeginning = "BEGINNING OF CONVERSATION:";
|
||||
const userKeyword = " USER: ";
|
||||
const assistantKeyword = " GPT:";
|
||||
const koalaStopSequence = "</s>";
|
||||
|
||||
fetch("/config")
|
||||
.then(r => {
|
||||
return r.json();
|
||||
}).then(frontend_config => {
|
||||
// Get frontend config
|
||||
let frontend_config = null;
|
||||
fetch("/config")
|
||||
.then(r => {
|
||||
return r.json();
|
||||
})
|
||||
.then(j => {
|
||||
frontend_config = j;
|
||||
});
|
||||
|
||||
// Message Context
|
||||
let conversation = [frontend_config.profile.conversation_prefix];
|
||||
let conversation = [conversationBeginning];
|
||||
|
||||
// Elements - Sidebar
|
||||
const sidebarOpenButton = document.getElementById("sidebar-toggle-open");
|
||||
const sidebarCloseButton = document.getElementById("sidebar-toggle-close");
|
||||
const sidebarContainer = document.getElementById("sidebar-container");
|
||||
const settingsLabelAssistantNameElement = document.getElementById("settings-label-assistant");
|
||||
const settingsMaxTokensElement = document.getElementById("settings-max-tokens");
|
||||
const settingsTemperatureElement = document.getElementById("settings-temperature");
|
||||
const settingsTopPElement = document.getElementById("settings-top-p");
|
||||
|
|
@ -24,22 +29,20 @@ fetch("/config")
|
|||
const settingsRepeatPenaltyElement = document.getElementById("settings-repeat-penalty");
|
||||
const settingsPresencePenaltyElement = document.getElementById("settings-presence-penalty");
|
||||
const settingsFrequencyPenaltyElement = document.getElementById("settings-frequency-penalty");
|
||||
const resetSettingsButton = document.getElementById("reset-settings-btn");
|
||||
const resetHistoryButton = document.getElementById("reset-history-btn");
|
||||
|
||||
settingsLabelAssistantNameElement.innerText = frontend_config.profile.name;
|
||||
const resetSettingsButtonElement = document.getElementById("reset-settings-btn");
|
||||
const resetHistoryButtonElement = document.getElementById("reset-history-btn");
|
||||
|
||||
// Elements - Main
|
||||
const messageHistoryContainer = document.getElementById("messages");
|
||||
const textInputElement = document.getElementById("text-input");
|
||||
const sendButton = document.getElementById("send-btn");
|
||||
const sendButtonElement = document.getElementById("send-btn");
|
||||
|
||||
// API requests
|
||||
|
||||
async function apiCompletion(prompt, settings) {
|
||||
const bodyData = JSON.stringify({
|
||||
"prompt": prompt,
|
||||
"stop": frontend_config.profile.stop_sequences,
|
||||
"stop": [koalaStopSequence],
|
||||
"max_tokens": settings.max_tokens,
|
||||
"temperature": settings.temperature,
|
||||
"top_p": settings.top_p,
|
||||
|
|
@ -96,36 +99,33 @@ fetch("/config")
|
|||
|
||||
// Chat
|
||||
|
||||
// Message Roles
|
||||
const Roles = {
|
||||
const MessageType = {
|
||||
USER: {
|
||||
name: "User",
|
||||
class: "message-bg-user"
|
||||
},
|
||||
ASSISTANT: {
|
||||
name: frontend_config.profile.name,
|
||||
name: "Koala",
|
||||
class: "message-bg-assistant"
|
||||
}
|
||||
}
|
||||
|
||||
function addMessage(message, role) {
|
||||
if (role == Roles.USER) {
|
||||
conversation.push(
|
||||
frontend_config.profile.user_keyword + " "
|
||||
+ message + frontend_config.profile.separator + frontend_config.profile.assistant_keyword);
|
||||
function addMessage(message, type) {
|
||||
if (type == MessageType.USER) {
|
||||
conversation.push(userKeyword + message + assistantKeyword);
|
||||
}
|
||||
else { conversation.push(message + frontend_config.profile.separator); }
|
||||
else { conversation.push(message); }
|
||||
// UI
|
||||
let messageRoleElem = document.createElement("div");
|
||||
messageRoleElem.classList.add("message-type");
|
||||
messageRoleElem.innerText = role.name;
|
||||
let messageTypeElem = document.createElement("div");
|
||||
messageTypeElem.classList.add("message-type");
|
||||
messageTypeElem.innerText = type.name;
|
||||
let messageTextElem = document.createElement("div");
|
||||
messageTextElem.classList.add("message-text");
|
||||
messageTextElem.innerText = message.trim();
|
||||
messageTextElem.innerText = message;
|
||||
let messageElem = document.createElement("div");
|
||||
messageElem.classList.add("message");
|
||||
messageElem.classList.add(role.class);
|
||||
messageElem.appendChild(messageRoleElem);
|
||||
messageElem.classList.add(type.class);
|
||||
messageElem.appendChild(messageTypeElem);
|
||||
messageElem.appendChild(messageTextElem);
|
||||
messageHistoryContainer.appendChild(messageElem);
|
||||
messageHistoryContainer.scrollTo(0, messageHistoryContainer.scrollHeight);
|
||||
|
|
@ -146,9 +146,9 @@ fetch("/config")
|
|||
settingsRepeatPenaltyElement.disabled = true;
|
||||
settingsPresencePenaltyElement.disabled = true;
|
||||
settingsFrequencyPenaltyElement.disabled = true;
|
||||
resetSettingsButton.disabled = true;
|
||||
resetHistoryButton.disabled = true;
|
||||
sendButton.disabled = true;
|
||||
resetSettingsButtonElement.disabled = true;
|
||||
resetHistoryButtonElement.disabled = true;
|
||||
sendButtonElement.disabled = true;
|
||||
textInputElement.disabled = true;
|
||||
}
|
||||
|
||||
|
|
@ -160,61 +160,48 @@ fetch("/config")
|
|||
settingsRepeatPenaltyElement.disabled = false;
|
||||
settingsPresencePenaltyElement.disabled = false;
|
||||
settingsFrequencyPenaltyElement.disabled = false;
|
||||
resetSettingsButton.disabled = false;
|
||||
resetHistoryButton.disabled = false;
|
||||
sendButton.disabled = false;
|
||||
resetSettingsButtonElement.disabled = false;
|
||||
resetHistoryButtonElement.disabled = false;
|
||||
sendButtonElement.disabled = false;
|
||||
textInputElement.disabled = false;
|
||||
// focus text input
|
||||
if (!isMobile) textInputElement.focus();
|
||||
textInputElement.focus();
|
||||
}
|
||||
|
||||
async function chat() {
|
||||
disableInput();
|
||||
let input = textInputElement.value.trim();
|
||||
if (input == "") {
|
||||
enableInput();
|
||||
if (frontend_config == null) {
|
||||
console.log("Couldn't fetch frontend configuration.");
|
||||
}
|
||||
else {
|
||||
textInputElement.value = "";
|
||||
resizeInputElement();
|
||||
addMessage(input, Roles.USER);
|
||||
let prompt = conversation.join("");
|
||||
let settings = getSettings();
|
||||
apiCompletion(prompt, settings).then(r => {
|
||||
addMessage(r.trim(), Roles.ASSISTANT);
|
||||
disableInput();
|
||||
let input = textInputElement.value.trim();
|
||||
if (input == "") {
|
||||
enableInput();
|
||||
});
|
||||
}
|
||||
else {
|
||||
textInputElement.value = "";
|
||||
resizeInputElement();
|
||||
addMessage(input, MessageType.USER);
|
||||
let prompt = conversation.join("");
|
||||
let settings = getSettings();
|
||||
apiCompletion(prompt, settings).then(r => {
|
||||
addMessage(r, MessageType.ASSISTANT);
|
||||
enableInput();
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
function resetHistory() {
|
||||
conversation = [frontend_config.profile.conversation_prefix];
|
||||
conversation = [conversationBeginning];
|
||||
messageHistoryContainer.innerText = "";
|
||||
}
|
||||
|
||||
// Sidebar
|
||||
|
||||
function toggleSidebar() {
|
||||
if (sidebarContainer.classList.contains("sidebar-hidden")) {
|
||||
sidebarCloseButton.classList.remove("hidden");
|
||||
sidebarOpenButton.classList.add("hidden");
|
||||
sidebarContainer.classList.remove("sidebar-hidden");
|
||||
}
|
||||
else {
|
||||
sidebarOpenButton.classList.remove("hidden");
|
||||
sidebarCloseButton.classList.add("hidden");
|
||||
sidebarContainer.classList.add("sidebar-hidden");
|
||||
}
|
||||
}
|
||||
|
||||
// Event Listeners
|
||||
|
||||
sidebarOpenButton.addEventListener("click", toggleSidebar);
|
||||
sidebarCloseButton.addEventListener("click", toggleSidebar);
|
||||
|
||||
resetSettingsButton.addEventListener("click", resetSettings);
|
||||
resetHistoryButton.addEventListener("click", resetHistory);
|
||||
sendButton.addEventListener("click", chat);
|
||||
resetSettingsButtonElement.addEventListener("click", resetSettings);
|
||||
resetHistoryButtonElement.addEventListener("click", resetHistory);
|
||||
sendButtonElement.addEventListener("click", chat);
|
||||
|
||||
textInputElement.addEventListener("keypress", e => {
|
||||
// Send via Ctrl+Enter
|
||||
|
|
@ -226,4 +213,4 @@ fetch("/config")
|
|||
textInputElement.addEventListener("input", resizeInputElement);
|
||||
resizeInputElement();
|
||||
|
||||
});
|
||||
})();
|
||||
|
|
@ -5,13 +5,11 @@
|
|||
--background2: #303030;
|
||||
--background3: #161616;
|
||||
--background4: #131313;
|
||||
--background5: #1a1a1a;
|
||||
--button-bg: #3b3b3b;
|
||||
--button-bg2: #4f4f4f;
|
||||
--icon-button-fill: #ffffff;
|
||||
--send-icon-button-fill: #29c76d;
|
||||
--color: #fafafa;
|
||||
--color2: #bbbbbb;
|
||||
--border-radius: .5rem;
|
||||
}
|
||||
|
||||
|
|
@ -29,23 +27,11 @@ input[type="number"] {
|
|||
width: 4rem;
|
||||
}
|
||||
|
||||
.sidebar-container {
|
||||
padding: .5rem;
|
||||
height: 100%;
|
||||
background-color: var(--background5);
|
||||
box-sizing: border-box;
|
||||
}
|
||||
|
||||
.sidebar-container.sidebar-hidden > .sidebar {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.sidebar {
|
||||
margin-left: .5rem;
|
||||
margin-right: .5rem;
|
||||
margin-top: .4rem;
|
||||
.sidepanel {
|
||||
gap: .5rem;
|
||||
align-items: flex-end;
|
||||
padding: 1rem;
|
||||
padding-left: 0;
|
||||
min-width: fit-content;
|
||||
}
|
||||
|
||||
|
|
@ -70,38 +56,19 @@ input[type="number"] {
|
|||
}
|
||||
|
||||
.messages {
|
||||
gap: 1rem;
|
||||
gap: 1.1rem;
|
||||
margin-bottom: 1rem;
|
||||
overflow-y: scroll;
|
||||
max-height: 89vh;
|
||||
align-items: center;
|
||||
flex-grow: 2;
|
||||
}
|
||||
|
||||
.message {
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
flex-wrap: wrap;
|
||||
gap: 1rem;
|
||||
}
|
||||
|
||||
.message-type {
|
||||
color: var(--color2);
|
||||
text-align: center;
|
||||
}
|
||||
|
||||
.message-text {
|
||||
white-space: pre-wrap;
|
||||
padding: .5rem .8rem;
|
||||
flex-direction: row;
|
||||
gap: .5rem;
|
||||
padding: .5rem;
|
||||
border-radius: var(--border-radius);
|
||||
}
|
||||
|
||||
.message-bg-assistant > .message-text {
|
||||
background: var(--background2);
|
||||
}
|
||||
|
||||
.message-bg-user > .message-text {
|
||||
background: var(--background3);
|
||||
max-width: fit-content;
|
||||
}
|
||||
|
||||
button {
|
||||
|
|
@ -134,6 +101,19 @@ button:hover {
|
|||
width: 100%;
|
||||
}
|
||||
|
||||
.message-bg-assistant {
|
||||
background: var(--background2);
|
||||
}
|
||||
|
||||
.message-bg-user {
|
||||
background: var(--background3);
|
||||
}
|
||||
|
||||
.message-type {
|
||||
min-width: 3.5rem;
|
||||
padding-left: .1rem;
|
||||
}
|
||||
|
||||
.input-container {
|
||||
margin-top: auto;
|
||||
flex-direction: row;
|
||||
|
|
@ -155,8 +135,6 @@ button:hover {
|
|||
}
|
||||
|
||||
.icon-button {
|
||||
width: fit-content;
|
||||
height: fit-content;
|
||||
padding: .2rem;
|
||||
display: flex;
|
||||
align-items: center;
|
||||
|
|
@ -182,39 +160,3 @@ button:hover {
|
|||
height: 2.2rem;
|
||||
fill: var(--send-icon-button-fill);
|
||||
}
|
||||
|
||||
.hidden {
|
||||
display: none;
|
||||
}
|
||||
|
||||
@media only screen and (max-width: 600px) {
|
||||
|
||||
body {
|
||||
height: 100dvh;
|
||||
}
|
||||
|
||||
.sidebar-container {
|
||||
position: absolute;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
z-index: 1;
|
||||
}
|
||||
|
||||
.sidebar-container.sidebar-hidden {
|
||||
position: absolute;
|
||||
width: unset;
|
||||
height: unset;
|
||||
left: auto;
|
||||
right: 0;
|
||||
background: transparent;
|
||||
}
|
||||
|
||||
.sidebar-container > #sidebar-toggle-close {
|
||||
margin-right: auto;
|
||||
}
|
||||
|
||||
.sidebar {
|
||||
margin-right: auto;
|
||||
padding-right: 2.5rem;
|
||||
}
|
||||
}
|
||||
|
|
|
|||
Binary file not shown.
|
Before Width: | Height: | Size: 129 KiB After Width: | Height: | Size: 94 KiB |
|
|
@ -1,8 +0,0 @@
|
|||
{
|
||||
"name": "None",
|
||||
"conversation_prefix": "",
|
||||
"user_keyword": "",
|
||||
"assistant_keyword": "",
|
||||
"separator": "",
|
||||
"stop_sequences": []
|
||||
}
|
||||
|
|
@ -1,8 +0,0 @@
|
|||
{
|
||||
"name": "Koala",
|
||||
"conversation_prefix": "BEGINNING OF CONVERSATION: ",
|
||||
"user_keyword": "USER:",
|
||||
"assistant_keyword": "GPT:",
|
||||
"separator": " ",
|
||||
"stop_sequences": ["</s>"]
|
||||
}
|
||||
|
|
@ -1,8 +0,0 @@
|
|||
{
|
||||
"name": "Manticore",
|
||||
"conversation_prefix": "",
|
||||
"user_keyword": "USER:",
|
||||
"assistant_keyword": "ASSISTANT:",
|
||||
"separator": "\n",
|
||||
"stop_sequences": ["</s>", "<unk>", "### USER:", "USER:"]
|
||||
}
|
||||
|
|
@ -1,8 +0,0 @@
|
|||
{
|
||||
"name": "Vicuna v0",
|
||||
"conversation_prefix": "A chat between a curious human and a helpful AI assistant.\n\n",
|
||||
"user_keyword": "### Human:",
|
||||
"assistant_keyword": "### Assistant:",
|
||||
"separator": "\n",
|
||||
"stop_sequences": ["### Human:"]
|
||||
}
|
||||
|
|
@ -1,8 +0,0 @@
|
|||
{
|
||||
"name": "Vicuna v1.1",
|
||||
"conversation_prefix": "A chat between a curious user and a helpful AI assistant.\n\n",
|
||||
"user_keyword": "USER:",
|
||||
"assistant_keyword": "ASSISTANT:",
|
||||
"separator": "\n",
|
||||
"stop_sequences": ["</s>"]
|
||||
}
|
||||
|
|
@ -1,3 +1,3 @@
|
|||
llama-cpp-python[server]==0.1.56
|
||||
llama-cpp-python[server]==0.1.50
|
||||
uvicorn==0.22.0
|
||||
sanic==23.3.0
|
||||
Reference in a new issue