ChaoticByte/Eucalyptus-Chat
Archived
1
0
Fork 0
Code Issues Pull requests Projects Releases 10 Packages Wiki Activity Actions

Use streaming HTTP API calls #8

New issue
Closed
opened 2023-05-24 18:32:56 +00:00 by ChaoticByte · 1 comment
ChaoticByte commented 2023-05-24 18:32:56 +00:00 (Migrated from github.com)
Copy link

This allows to display chunks/tokens as soon as they are generated, instead of waiting for the whole output.

This allows to display chunks/tokens as soon as they are generated, instead of waiting for the whole output.
ChaoticByte commented 2023-05-24 21:43:12 +00:00 (Migrated from github.com)
Copy link

After a semi-working implementation, I noticed that using HTTP Streams results in degraded performance (answers will take longer because an increased load time (llama timings).

After a semi-working implementation, I noticed that using HTTP Streams results in degraded performance (answers will take longer because an increased `load time` (llama timings).
This repository is archived. You cannot comment on issues.
No Branch/Tag specified
Branches Tags
main
v4.3
v4.2
v4.1
v4.0
v3.1
v3.0
v2.1
v2.0
v1.1
v1.0
Labels
Clear labels   
bug

Something isn't working

  
dependencies

Pull requests that update a dependency file

  
documentation

Improvements or additions to documentation

  
duplicate

This issue or pull request already exists

  
enhancement

New feature or request

  
good first issue

Good for newcomers

  
help wanted

Extra attention is needed

  
invalid

This doesn't seem right

  
question

Further information is requested

  
security-related

   
wontfix

This will not be worked on

No labels
bug
dependencies
documentation
duplicate
enhancement
good first issue
help wanted
invalid
question
security-related
wontfix
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: ChaoticByte/Eucalyptus-Chat#8
No description provided.
Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?