starred/anything-llm

Fork 0

mirror of https://github.com/Mintplex-Labs/anything-llm.git synced 2026-03-02 22:57:05 -05:00

[BUG/FEAT]: AWS Bedrock reasoning models with `@agent` #2312

New issue

Open

opened 2026-02-28 06:03:29 -05:00 by deekerman · 6 comments

deekerman commented

2026-02-28 06:03:29 -05:00

Owner

Originally created by @timothycarambat on GitHub (Mar 27, 2025).

Originally assigned to: @timothycarambat on GitHub.

How are you running AnythingLLM?

All versions

What happened?

With the refactor of AWS Bedrock to move away from Langchain in https://github.com/Mintplex-Labs/anything-llm/pull/3537

This needs to be expanded to the agent execution provider, as the use of Langchain to use reasoning models for agent execution is not possible with the current implementation.

Current workaround
Do not use reasoning models for AWS Bedrock agent execution as the content response is an array as opposed to a string - like all other models.

Are there known steps to reproduce?

Use a reasoning model for AWS bedrock and send a single agent chat. This error will manifest as a jsonString?.startsWith error - which is a red herring as the real error is the output formats from the response being mishandled by Langchain.

Originally created by @timothycarambat on GitHub (Mar 27, 2025). Originally assigned to: @timothycarambat on GitHub. ### How are you running AnythingLLM? All versions ### What happened? With the refactor of AWS Bedrock to move away from Langchain in https://github.com/Mintplex-Labs/anything-llm/pull/3537 This needs to be expanded to the `agent` execution provider, as the use of Langchain to use reasoning models for agent execution is not possible with the current implementation. **Current workaround** Do not use reasoning models for AWS Bedrock agent execution as the `content` response is an array as opposed to a string - like all other models. ### Are there known steps to reproduce? Use a reasoning model for AWS bedrock and send a single agent chat. This error will manifest as a `jsonString?.startsWith` error - which is a red herring as the real error is the output formats from the response being mishandled by Langchain.

deekerman added the

enhancement

bug

Integration Request

labels

2026-02-28 06:03:29 -05:00

deekerman commented

2026-02-28 06:03:35 -05:00

Author

Owner

@tristan-stahnke-GPS commented on GitHub (Apr 24, 2025):

@timothycarambat I think my PR #3714 resolved this, can you check?

@tristan-stahnke-GPS commented on GitHub (Apr 24, 2025): @timothycarambat I think my PR #3714 resolved this, can you check?

deekerman commented

2026-02-28 06:03:36 -05:00

Author

Owner

@tristan-stahnke-GPS commented on GitHub (Apr 25, 2025):

I was able to create agents using bedrock with my PR applied for the bedrock provider, parsed the content of a google search and sent it back to the LLM; not sure if there's another part of the @agent functionality that's needed, but would be cool to test it out, definitely want to get agents working fully with bedrock 💯

@tristan-stahnke-GPS commented on GitHub (Apr 25, 2025): I was able to create agents using bedrock with my PR applied for the bedrock provider, parsed the content of a google search and sent it back to the LLM; not sure if there's another part of the @agent functionality that's needed, but would be cool to test it out, definitely want to get agents working fully with bedrock 💯

deekerman commented

2026-02-28 06:03:36 -05:00

Author

Owner

@0xbadshah commented on GitHub (Jun 25, 2025):

@tristan-stahnke Amazon Bedrock has multiple models and not every model works correctly despite right IAM permissions to access. I tried agent with Deepseek and it errors out. Which Bedrock model did you this against?

@0xbadshah commented on GitHub (Jun 25, 2025): @tristan-stahnke Amazon Bedrock has multiple models and not every model works correctly despite right IAM permissions to access. I tried agent with Deepseek and it errors out. Which Bedrock model did you this against?

deekerman commented

2026-02-28 06:03:36 -05:00

Author

Owner

@tristan-stahnke-GPS commented on GitHub (Jun 25, 2025):

@Chan9390 I primarily use Claude Sonnet / Opus models (as well as the amazon Nova etc. Models). I haven't had a chance to look at Deepseek; that would definitely be something to chase down! And making the provider more model agnostic would be ideal as well, so we can leave room for additional functionality down the road (maybe allowing capabilities to thought process tokens?) I'll take a look!

@tristan-stahnke-GPS commented on GitHub (Jun 25, 2025): @Chan9390 I primarily use Claude Sonnet / Opus models (as well as the amazon Nova etc. Models). I haven't had a chance to look at Deepseek; that would definitely be something to chase down! And making the provider more model agnostic would be ideal as well, so we can leave room for additional functionality down the road (maybe allowing capabilities to thought process tokens?) I'll take a look!

deekerman commented

2026-02-28 06:03:36 -05:00

Author

Owner

@bhasmang-tri commented on GitHub (Oct 7, 2025):

I also get Invalid message content: empty string. 'ai' must contain non-empty content. when i invoke the @agent with Bedrock (Claude model) using IAM role (i hosted it on AWS via docker)

@bhasmang-tri commented on GitHub (Oct 7, 2025): I also get Invalid message content: empty string. 'ai' must contain non-empty content. when i invoke the @agent with Bedrock (Claude model) using IAM role (i hosted it on AWS via docker)

deekerman commented

2026-02-28 06:03:36 -05:00

Author

Owner

@SquadUpSquid commented on GitHub (Nov 3, 2025):

Hello, Is there any update to this issue. I'm having similar issues when trying to call @agent (MCP Server) using Bedrock models.
Hosted on an AWS EC2 using Docker.

Input: @agent blah blah blah

Output: (it's not always the same)

nothing, just a blank response
AWSBedrock::streamGetChatCompletion failed during setup. Bedrock is unable to process your request.
"ai" must contain non-empty content
Bedrock is unable to process your request.

I have tried this with Bedrock Claude 3.5, Claude 3.7, Nova Pro, Llama 3 70B. I'm using IAM Roles and gave it "bedrock:*" just to rule out any permissions problems with bedrock.

I have used the free Grok models, as a "control", and have gotten a proper response from it.

@SquadUpSquid commented on GitHub (Nov 3, 2025): Hello, Is there any update to this issue. I'm having similar issues when trying to call @agent (MCP Server) using Bedrock models. Hosted on an AWS EC2 using Docker. Input: @agent blah blah blah Output: (it's not always the same) 1. *nothing*, just a blank response 2. AWSBedrock::streamGetChatCompletion failed during setup. Bedrock is unable to process your request. 3. "ai" must contain non-empty content 4. Bedrock is unable to process your request. I have tried this with Bedrock Claude 3.5, Claude 3.7, Nova Pro, Llama 3 70B. I'm using IAM Roles and gave it "bedrock:*" just to rule out any permissions problems with bedrock. I have used the free Grok models, as a "control", and have gotten a proper response from it.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

starred/anything-llm#2312

No description provided.

Rows
Columns

[BUG/FEAT]: AWS Bedrock reasoning models with @agent #2312

How are you running AnythingLLM?

What happened?

Are there known steps to reproduce?

[BUG/FEAT]: AWS Bedrock reasoning models with `@agent` #2312