starred/anything-llm

Fork 0

mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-03-02 22:57:05 -05:00

[BUG]: OpenAI Compatible Endpoints completion without chunk #1170

New issue

Closed

opened 2026-02-28 05:18:00 -05:00 by deekerman · 4 comments

deekerman commented

2026-02-28 05:18:00 -05:00

Owner

Originally created by @nicho2 on GitHub (Jun 27, 2024).

How are you running AnythingLLM?

Docker (local)

What happened?

I tried your new interestring functionnaly "OpenAI Compatible Endpoints" , the path : /v1/openai/chat/completions

you have write : Send a prompt to the workspace with full use of documents as if sending a chat in AnythingLLM. Only supports some values of OpenAI API.

the system prompt is sending to the llm, but no documents contexts are inserted.

Are there known steps to reproduce?

{
"messages": [
{
"role": "user",
"content": "qu'est ce qu'un système BMS"
}
],
"model": "atelier_bms",
"stream": false,
"temperature": 0
}

Originally created by @nicho2 on GitHub (Jun 27, 2024). ### How are you running AnythingLLM? Docker (local) ### What happened? I tried your new interestring functionnaly "OpenAI Compatible Endpoints" , the path : /v1/openai/chat/completions you have write : Send a prompt to the workspace with full use of documents as if sending a chat in AnythingLLM. Only supports some values of OpenAI API. the system prompt is sending to the llm, but no documents contexts are inserted. ### Are there known steps to reproduce? { "messages": [ { "role": "user", "content": "qu'est ce qu'un système BMS" } ], "model": "atelier_bms", "stream": false, "temperature": 0 }

deekerman

2026-02-28 05:18:00 -05:00

closed this issue
added the
possible bug
label

deekerman commented

2026-02-28 05:18:07 -05:00

Author

Owner

@timothycarambat commented on GitHub (Jun 27, 2024):

In the docker logs you would see if any context was retrieved, but the case here is likely that there just simply are no matches to the prompt given in the vector database.

The search does occur
github.com/Mintplex-Labs/anything-llm@7a78ad3960/server/utils/chats/openaiCompatible.js (L83-L97)

@timothycarambat commented on GitHub (Jun 27, 2024): In the docker logs you would see if any context was retrieved, but the case here is likely that there just simply are no matches to the prompt given in the vector database. The search _does occur_ https://github.com/Mintplex-Labs/anything-llm/blob/7a78ad3960e069dbc783971dc7ff13cf1868c767/server/utils/chats/openaiCompatible.js#L83-L97

deekerman commented

2026-02-28 05:18:07 -05:00

Author

Owner

@nicho2 commented on GitHub (Jun 28, 2024):

I checked by sending the same question from the standard UI and by the API

In both cases, the vector base is well questioned and returns many pieces of documents. On the other hand, since the OPENAI_Like API, these pieces of documents are not added to the context

The docker log does not say much:

[OllamaEmbedder] Embedding 1 chunks of text with nomic-embed-text:latest.
[TELEMETRY SENT] {
event: 'sent_chat',
distinctId: '98297666-e886-4011-9303-5720b1f81b92',
properties: {
LLMSelection: 'lmstudio',
Embedder: 'ollama',
VectorDbSelection: 'chroma',
runtime: 'docker'
}
}
[Event Logged] - api_sent_cha

The distances are almost the same:

{"ids":"a85d269b-5da2-4f4c-9069-a49cfd6316da","adeb8a8c-95b4-4099-8912-fa2e22455eea","c2cfa314-96f0-495c-bc45-82f580d5c18f","cecfbcd9-c9ab-45e3-a595-becc01d506a7","b7dd0ad5-1313-4db7-aea9-15f7a1a47b56","distances":369.6019287109375,371.1099548339844,371.14044189453125,371.15972900390625,371.2540588378906,"metadatas":[[{"....

@nicho2 commented on GitHub (Jun 28, 2024): I checked by sending the same question from the standard UI and by the API In both cases, the vector base is well questioned and returns many pieces of documents. On the other hand, since the OPENAI_Like API, these pieces of documents are not added to the context The docker log does not say much: [OllamaEmbedder] Embedding 1 chunks of text with nomic-embed-text:latest. [TELEMETRY SENT] { event: 'sent_chat', distinctId: '98297666-e886-4011-9303-5720b1f81b92', properties: { LLMSelection: 'lmstudio', Embedder: 'ollama', VectorDbSelection: 'chroma', runtime: 'docker' } } [Event Logged] - api_sent_cha **The distances are almost the same:** {"ids":[["a85d269b-5da2-4f4c-9069-a49cfd6316da","adeb8a8c-95b4-4099-8912-fa2e22455eea","c2cfa314-96f0-495c-bc45-82f580d5c18f","cecfbcd9-c9ab-45e3-a595-becc01d506a7","b7dd0ad5-1313-4db7-aea9-15f7a1a47b56"]],"distances":[[369.6019287109375,371.1099548339844,371.14044189453125,371.15972900390625,371.2540588378906]],"metadatas":[[{"....

deekerman commented

2026-02-28 05:18:07 -05:00

Author

Owner

@nicho2 commented on GitHub (Jun 28, 2024):

if i use only Query, i have the answer ::
There is no relevant information in this workspace to answer your query.

but i see the chromadb answer , i have similarity >=0.25 :

{"ids":"135a37f6-8eb4-4452-9297-2cd8a0069975","790a133c-2967-4820-a44a-c2e0beabc418","4849c503-4538-4741-b152-150f91455e20","e3b5c32f-8d75-4971-95ac-bb6a3cf02808","distances":0.3664718270301819,0.3721543550491333,0.3739575743675232,0.37604087591171265,"metadatas":[[{"chunkSource":"","description":"Unknown","docAuthor":"Unknown","docSource":"a text file uploaded by the user.","id":"9753b7ca-7218-410b-8273-2ccdb94fe162","published":"6/21/2024, 2:21:04 PM","title":"Guide technique AREA MANAGER - partie 1.md","token_count_estimate":16438,"url":"file:///app/collector/hotdir/Guide technique AREA MANAGER - partie 1.md","wordCount":12105},{"chunkSource":"","description":"Unknown","docAuthor":"Unknown","docSource":"a text file uploaded by the user.","id":"9753b7ca-7218-410b-8273-2ccdb94fe162","published":"6/21/2024, 2:21:04 PM","title":"Guide technique AREA MANAGER - partie 1.md","token_count_estimate":16438,"url":"file:///app/collector/hotdir/Guide technique A

@nicho2 commented on GitHub (Jun 28, 2024): if i use only Query, i have the answer :: There is no relevant information in this workspace to answer your query. but i see the chromadb answer , i have similarity >=0.25 : {"ids":[["135a37f6-8eb4-4452-9297-2cd8a0069975","790a133c-2967-4820-a44a-c2e0beabc418","4849c503-4538-4741-b152-150f91455e20","e3b5c32f-8d75-4971-95ac-bb6a3cf02808"]],"distances":[[0.3664718270301819,0.3721543550491333,0.3739575743675232,0.37604087591171265]],"metadatas":[[{"chunkSource":"","description":"Unknown","docAuthor":"Unknown","docSource":"a text file uploaded by the user.","id":"9753b7ca-7218-410b-8273-2ccdb94fe162","published":"6/21/2024, 2:21:04 PM","title":"Guide technique AREA MANAGER - partie 1.md","token_count_estimate":16438,"url":"file:///app/collector/hotdir/Guide technique AREA MANAGER - partie 1.md","wordCount":12105},{"chunkSource":"","description":"Unknown","docAuthor":"Unknown","docSource":"a text file uploaded by the user.","id":"9753b7ca-7218-410b-8273-2ccdb94fe162","published":"6/21/2024, 2:21:04 PM","title":"Guide technique AREA MANAGER - partie 1.md","token_count_estimate":16438,"url":"file:///app/collector/hotdir/Guide technique A

deekerman commented

2026-02-28 05:18:07 -05:00

Author

Owner

@timothycarambat commented on GitHub (Jun 28, 2024):

Odd, i dont see a "documents" key in there. There should be ids,distances,metadatas and documents

@timothycarambat commented on GitHub (Jun 28, 2024): Odd, i dont see a "documents" key in there. There should be `ids`,`distances`,`metadatas` and `documents`

deekerman referenced this issue

2026-02-28 05:41:17 -05:00

main.workspaces does not exist error on Microsoft Surface 7 ARM64 Running WIndows 11 #1800

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/anything-llm#1170

No description provided.

Rows
Columns