starred/anything-llm

Fork 0

mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-03-02 22:57:05 -05:00

Anthropic Claude 2 API Integration #101

New issue

Closed

opened 2026-02-28 04:28:02 -05:00 by deekerman · 6 comments

deekerman commented

2026-02-28 04:28:02 -05:00

Owner

Originally created by @jignnsd on GitHub (Jul 22, 2023).

Hey Tim, thank you for this work, it works very good so far on my tests.
I have one question, is it possible to have support (using the same DB and Embeddings API) of the just introduced AnthropIc Claude 2 to use it instead of Openai chat API on an easy way, or it will implicate a lot of changes to be done?
I’m curios because I just compare the answers between openai (on anything-llm) and claude 2 with the same pdf file, and the answer was much better on claude 2, maybe because of the bigger token limit, so maybe will be a good thing to try if this does not implicate a huge change.
Also, the pricing drop using Claude 2 instead of Openai is important, like half of the price.
Again, many thanks for the good work and hopefully you can add a link to make a donation to compensate this great effort.
Javier

Originally created by @jignnsd on GitHub (Jul 22, 2023). Hey Tim, thank you for this work, it works very good so far on my tests. I have one question, is it possible to have support (using the same DB and Embeddings API) of the just introduced AnthropIc Claude 2 to use it instead of Openai chat API on an easy way, or it will implicate a lot of changes to be done? I’m curios because I just compare the answers between openai (on anything-llm) and claude 2 with the same pdf file, and the answer was much better on claude 2, maybe because of the bigger token limit, so maybe will be a good thing to try if this does not implicate a huge change. Also, the pricing drop using Claude 2 instead of Openai is important, like half of the price. Again, many thanks for the good work and hopefully you can add a link to make a donation to compensate this great effort. Javier

deekerman

2026-02-28 04:28:02 -05:00

closed this issue
added the
enhancement

Integration Request
labels

deekerman commented

2026-02-28 04:28:21 -05:00

Author

Owner

@timothycarambat commented on GitHub (Jul 25, 2023):

Oh, that sounds very cool. The LLM side of things needs to diversity (even beyond Azure which is just OpenAI anyway).

If they have a support JS client it should keep things even more simple. Otherwise rolling our own internal API might be a pain. I do think that embedding will still have to be done with OpenAI though, as I don't think Anthropic has an embedded yet?

Chat should be straighforward

@timothycarambat commented on GitHub (Jul 25, 2023): Oh, that sounds very cool. The LLM side of things needs to diversity (even beyond Azure which is just OpenAI anyway). If they have a support JS client it should keep things even more simple. Otherwise rolling our own internal API might be a pain. I do think that embedding will still have to be done with OpenAI though, as I don't think Anthropic has an embedded yet? Chat should be straighforward

deekerman commented

2026-02-28 04:28:21 -05:00

Author

Owner

@jignnsd commented on GitHub (Jul 25, 2023):

You are right, should keept openai for embeddings and also same vectordb, just the chat part will be changed to have the option of Anthropic.

@jignnsd commented on GitHub (Jul 25, 2023): You are right, should keept openai for embeddings and also same vectordb, just the chat part will be changed to have the option of Anthropic.

deekerman commented

2026-02-28 04:28:21 -05:00

Author

Owner

@timothycarambat commented on GitHub (Jul 25, 2023):

For Implementation:
Node client: https://www.npmjs.com/package/@anthropic-ai/sdk
API: https://docs.anthropic.com/claude/reference/getting-started-with-the-api

Will require some new ENV keys in settings to support this config.

@timothycarambat commented on GitHub (Jul 25, 2023): For Implementation: Node client: https://www.npmjs.com/package/@anthropic-ai/sdk API: https://docs.anthropic.com/claude/reference/getting-started-with-the-api Will require some new ENV keys in settings to support this config.

deekerman commented

2026-02-28 04:28:21 -05:00

Author

Owner

@madruga8 commented on GitHub (Jul 30, 2023):

the big context of claude2-100k really makes a huge difference...

@madruga8 commented on GitHub (Jul 30, 2023): the big context of claude2-100k really makes a huge difference...

deekerman commented

2026-02-28 04:28:21 -05:00

Author

Owner

@timothycarambat commented on GitHub (Aug 3, 2023):

Mintplex Labs has applied to the Anthropic Claude 2 API access so we can integrate it fully. This issue is blocked until we or someone else can provide an API key for development (do not share key in issue please)

@timothycarambat commented on GitHub (Aug 3, 2023): Mintplex Labs has applied to the Anthropic Claude 2 API access so we can integrate it fully. This issue is blocked until we or someone else can provide an API key for development (do not share key in issue please)

deekerman commented

2026-02-28 04:28:21 -05:00

Author

Owner

@ishaan-jaff commented on GitHub (Sep 21, 2023):

Hi @jignnsd @timothycarambat I believe I can help with this issue. I’m the maintainer of LiteLLM https://github.com/BerriAI/litellm - we allow you to use any LLM as a drop in replacement for gpt-3.5-turbo.

You can use LiteLLM in the following ways:

With your own API KEY:

This calls the provider API directly

from litellm import completion
import os
## set ENV variables 
os.environ["OPENAI_API_KEY"] = "your-key" # 
os.environ["COHERE_API_KEY"] = "your-key" # 

messages = [{ "content": "Hello, how are you?","role": "user"}]

# openai call
response = completion(model="gpt-3.5-turbo", messages=messages)

# cohere call
response = completion(model="command-nightly", messages=messages)

Using the LiteLLM Proxy with a LiteLLM Key

this is great if you don’t have access to claude but want to use the open source LiteLLM proxy to access claude

from litellm import completion
import os

## set ENV variables 
os.environ["OPENAI_API_KEY"] = "sk-litellm-5b46387675a944d2" # [OPTIONAL] replace with your openai key
os.environ["COHERE_API_KEY"] = "sk-litellm-5b46387675a944d2" # [OPTIONAL] replace with your cohere key

messages = [{ "content": "Hello, how are you?","role": "user"}]

# openai call
response = completion(model="gpt-3.5-turbo", messages=messages)

# cohere call
response = completion(model="command-nightly", messages=messages)

@ishaan-jaff commented on GitHub (Sep 21, 2023): Hi @jignnsd @timothycarambat I believe I can help with this issue. I’m the maintainer of LiteLLM https://github.com/BerriAI/litellm - we allow you to use any LLM as a drop in replacement for `gpt-3.5-turbo`. You can use LiteLLM in the following ways: ## With your own API KEY: This calls the provider API directly ```python from litellm import completion import os ## set ENV variables os.environ["OPENAI_API_KEY"] = "your-key" # os.environ["COHERE_API_KEY"] = "your-key" # messages = [{ "content": "Hello, how are you?","role": "user"}] # openai call response = completion(model="gpt-3.5-turbo", messages=messages) # cohere call response = completion(model="command-nightly", messages=messages) ``` ## Using the LiteLLM Proxy with a LiteLLM Key this is great if you don’t have access to claude but want to use the open source LiteLLM proxy to access claude ```python from litellm import completion import os ## set ENV variables os.environ["OPENAI_API_KEY"] = "sk-litellm-5b46387675a944d2" # [OPTIONAL] replace with your openai key os.environ["COHERE_API_KEY"] = "sk-litellm-5b46387675a944d2" # [OPTIONAL] replace with your cohere key messages = [{ "content": "Hello, how are you?","role": "user"}] # openai call response = completion(model="gpt-3.5-turbo", messages=messages) # cohere call response = completion(model="command-nightly", messages=messages) ```

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/anything-llm#101

No description provided.

Rows
Columns