mirror of
https://github.com/advplyr/audiobookshelf.git
synced 2026-03-02 22:46:56 -05:00
[Bug]: A ; seems to be being added to the custom metadata search #2769
Labels
No labels
authentication
awaiting release
backlog
bug
chapter editor
config-issue
ebooks
encoding/embedding
enhancement
help wanted
listening sessions & progress
planned
possible plugin
progress sync
sorting/filtering/searching
unable to reproduce
upload
users & permissions
waiting
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/audiobookshelf#2769
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @alexander-paterson on GitHub (Feb 19, 2025).
What happened?
I'm using abs-tract to add in extra metadata providers. The search being sent to abs-tract appears to have a
;before the word query.I created a but report in abs-tract's GitHub and the dev said it appears to be an ABS issue and to create a bug here.
What did you expect to happen?
After adding a metadata provider through abs-tract I should be able to get search results.
Steps to reproduce the issue
;before the word queryAudiobookshelf version
2.19.3
How are you running audiobookshelf?
Docker
What OS is your Audiobookshelf server hosted from?
Linux
If the issue is being seen in the UI, what browsers are you seeing the problem on?
None
Logs
Additional Notes
No response
@advplyr commented on GitHub (Feb 19, 2025):
I'm not able to reproduce this. Do you get this result when you use other search text?
@alexander-paterson commented on GitHub (Feb 20, 2025):
Okay, this is weird.
I have Goodreads, Kindle US and Kindle UK set up in abs-tract
Looking at the logs, they all have the same
;in the URL sent to abs-tractThe two Kindle ones don't return any results in ABS, but Goodreads does.
If I take all three URL's, copy them and remove the
;then they all return results.@ahobsonsayers commented on GitHub (Feb 20, 2025):
Hey! Yeah I am also having problems replicating this with ABS sadly 😞
The reason I think this is a (possible) ABS bug rather than to do with abs-tract, is that the lines printed are simply logs of exactly the requests made to the abs-stract server - which in theory means if that is what is being logged, the request is incorrectly being made.
Worth checking (although I believe it should have no impact), you mention you are running ABS using docker, and I assume you are running abs-tract using docker, but in the logs I see an IP address being used rather than a container name to connect them. Are you using a host network?
@alexander-paterson commented on GitHub (Feb 21, 2025):
The logs I posted were from abs-tract, this is what I see in ABS's logs
@nichwall commented on GitHub (Feb 22, 2025):
Is it possible that something between ABS and your abs-tract instance is changing the request to include a semicolon? In doing some research, it looks like older versions of reverse proxies and some server implementations may use a semicolon as the delimiter instead of an ampersand, so could be something rewriting the path?
https://www.w3.org/TR/html4/appendix/notes.html#h-B.2.2
How did you test this? Was this through a browser or using something like curl?
curl -v "http://192.168.8.219:5555/kindle/uk/search?mediaType=book&query=Cujo&author=Stephen+King"Either way, we should add logging of the actual query that ABS is generating to match the other provider debug printouts. I just opened https://github.com/advplyr/audiobookshelf/pull/4034 to add this logging.
@alexander-paterson commented on GitHub (Feb 22, 2025):
I just copied the URL, pasted it into the browser, removed the
;and pressed enter. I then got a json response in my browser.I'm not aware of anything that would mess with the request. I'm just confused as to why the Goodreads one works but the two Kindle ones don't.
They were working at one point back when I initially set up abs-tract a few months ago.
@Vito0912 commented on GitHub (Feb 23, 2025):
I just tested it with both Goodreads and Kindle using
https://provider.vito0912.de/kindle/ukandhttps://localhost:5555/kindle/uk, neither of which sent a ";".I also got the error:
I assume this is because my provider returns a 500 error most of the time for Kindle (because Amazon is Amazon). Goodreads works. However, when I enabled logging in my proxy, I saw that it sent the correct data, and since other custom providers work with parsing, I assume this will work too.
How did you add the provider in ABS? Perhaps there is a character that ABS does not expect which causes it to panic.
@alexander-paterson commented on GitHub (Feb 23, 2025):
This is what I have in ABS
@Vito0912 commented on GitHub (Feb 23, 2025):
If you want, you could try using
https://provider.vito0912.de/kindle/ukorhttps://provider.vito0912.de/goodreadswithabsas the Auth value? I will enable logging/access log and check if;is sent. Then we at least could sort out if this is something with your setup of abs-tract or/and abs.Kindle will most likley not return anything
@alexander-paterson commented on GitHub (Feb 23, 2025):
Just added and tried. Kindle didn't work (as you said) and Goodreads did.
@Vito0912 commented on GitHub (Feb 23, 2025):
Here are the results:
/uk/search?mediaType=book&query=Cujo&author=Stephen+King/search?mediaType=book&query=Cujo&author=Stephen+Kingthe provider is stripped from the logs, because of technical reasons
LGTM, so probably some issue with local networking or the abs-tract image you are running?
@ahobsonsayers commented on GitHub (Feb 24, 2025):
Given the interesting discussion and investigation in this issue, it might be worth highlighting my earlier question that I'm not sure got answered:
I wonder if this could hold some key as to what is happening.
On side note, @Vito0912, I'm very interested in the fact you are not getting any results from kindle in abs-tract! It works really well for me without issue 🤔 Can you open an issue over in abs-stract for me with some logs? I'd be interested in looking into this. Many thanks 🙏
@nichwall commented on GitHub (Feb 24, 2025):
The added logging is added in 2.19.5 if you want to try comparing the request from ABS to what abs-tract is getting @alexander-paterson (as well as @ahobsonsayers question)
@Vito0912 commented on GitHub (Feb 25, 2025):
@ahobsonsayers I think this has nothing to do with abs-tract. You can try yourself and will find out the error.
The servers IP I use has not a good enough reputation to make a connection. It's being blocked, because I use an VPN to route outgoing traffic for these providers
@alexander-paterson commented on GitHub (Mar 1, 2025):
I've been away for a few days and this is the first time I've been able to check this after the logging has been added. This is what I'm seeing when I perform a search.
This is from ABS
This is from abs-tract
@nichwall commented on GitHub (Mar 1, 2025):
Huh, that is really weird. Could you share the existing
metadata.jsonfor the book you are searching with?@alexander-paterson commented on GitHub (Mar 1, 2025):
metadata.json
@Vito0912 commented on GitHub (Mar 1, 2025):
And you are using the docker version from GitHub/Docker (So not build from source or whatever)?
Maybe I am blind, but the query is generated in:
and the
;is inserted between the paramaters.queryis not modifiable, so the;is at a place where it possibly shouldn't be able to be.Even forcing a
;:produces an escaped version:
mediaType=book%26%3B&query=title@alexander-paterson commented on GitHub (Mar 1, 2025):
This is the image I'm using
ghcr.io/advplyr/audiobookshelf:latest@alexander-paterson commented on GitHub (Mar 1, 2025):
It might be something I'm doing, here is my compose file for ABS, abs-tract and abs-autoconverter
abs-autoconverter is currently stopped due to getting a memory error during conversions
The
.envfile just has the paths forDOCKER_DATA_PATHetc@yetti7 commented on GitHub (Mar 2, 2025):
I've been having the same issue using the windows deployment latest version. Kindle was working and then suddenly stopped being able to pull data on any books. I've got both versions of abs-tract running just to make sure.
Goodreads pulls with no issues.
@Vito0912 commented on GitHub (Mar 2, 2025):
@yetti7
Are you sure it is the same error or do you think it is the same error because Kindle does not work anymore?
For testing I also set up the same IP and port etc. as OP, but I cannot reproduce it
@yetti7 commented on GitHub (Mar 2, 2025):
I can curl from kindle with no issues. It only happens when trying to parse it within abs. So I'm not sure honestly.
@nichwall commented on GitHub (Mar 2, 2025):
Using this metadata file, I am not able to reproduce the semicolon being inserted in the ABS logs.
Using 2.19.5 in Docker on Synology.
@Vito0912 commented on GitHub (Mar 2, 2025):
Could you check the logs to be sure. But sounds like the same error
@Vito0912 commented on GitHub (Mar 2, 2025):
@nichwall Do you think we would be able to reproduce if we had the db file of the library?
@nichwall commented on GitHub (Mar 2, 2025):
Yeah, probably. If you're willing to share the backup file or manually zip up the database found in
/configthat would be great @alexander-paterson . A DM or email with the download link would work best so you can share specifically with us without making it available publicly.@alexander-paterson commented on GitHub (Mar 2, 2025):
Sent it in an email to you @nichwall
@nichwall commented on GitHub (Mar 2, 2025):
Awesome, thanks. Using your backup on 2.19.5 (Linux from source), I was unable to replicate the semicolon being inserted.
@yetti7 commented on GitHub (Mar 4, 2025):
So i dont believe goodreads is the issue. Unless im mistaken here. There is something about the Kindle search that is breaking. It curls with no issue but when parsing on ABS it returns with no data. Where as good reads returns with no issue.
@Vito0912 commented on GitHub (Mar 4, 2025):
@yetti7
As mentioned in this comment: https://github.com/advplyr/audiobookshelf/issues/4004#issuecomment-2692756767
It would be helpful if you could check the logs to see if they show the same error or a different one. If it is the same error, could you share details about your setup? This way, we can identify similarities.
@yetti7 commented on GitHub (Mar 4, 2025):
My Apologies was a hectic weekend.
Im running ABS 2.19.5 through windows and have docker desktop set up to run ABS-tract 0.1.1 and 0.2.0.
0.1.1 Returns limited goodreads data so thats why im also running 0.2.0
Kindle returns nothing
metadata.json
Heres the metadata file im using for searching kindle both US UK 0.1.1 and 0.2.0
Let me know what other log files I can pull from the windows version.
@nichwall commented on GitHub (Mar 4, 2025):
Can you enable Debug logs and look for something like below?
This issue is discussing ABS inserting a semicolon in the search query. If nothing is returned from the metadata provider, that is a different issue.
@yetti7 commented on GitHub (Mar 4, 2025):
@nichwall
Looks like you are correct. I'm not seeing the ; I think I ended up following the trail when @alexander-paterson was having the same issue of GoodReads returning and then Kindle suddenly breaking out of nowhere when it was working previously.
@nichwall commented on GitHub (Mar 4, 2025):
That would be something specific to abs-tract, then. Possibly related to the whole Kindle changing things so you can no longer copy files over USB or something. I saw talk of other things changing, but have not been following it myself so not sure if the API was affected
@yetti7 commented on GitHub (Mar 4, 2025):
If that's the case wouldn't curl break as well?
I can absolutely take it over to ABS-Tract. From the issues i saw raised over there it seemed hit and miss being the same issue.
Usually i tend to just look for other peoples solutions before bringing up my own but maybe in this case my hatred of Calibre has led me to be solo in this particular circumstance? My love of ABS and all the work you guys have put into it made me want to keep everything central to that.
@Vito0912 commented on GitHub (Mar 4, 2025):
The error you encounter is an error if (I think it was description) field is not filled and the function therefore crashes. abs-tract probably does not provide that data. Could you share a test response that you get via curl?
@nichwall I can't look that much into it yet, but I think this actually might be a bug from abs if the description field is not provided (I currently do not have the code infornt of me. Just a thought that came to me)
@yetti7 commented on GitHub (Mar 4, 2025):
I tested using
and
The error i get in ABS when i parse is
im not sure how its handled but if description is needed then i can see why it would fail
I would image it falls more on abs-tract than ABS if that is the case but im not sure.
@Vito0912 commented on GitHub (Mar 4, 2025):
Sorry, I meant the result you get using curl, not abs. I didn't make this very clear.
So just the json output you get with your first or second request
@yetti7 commented on GitHub (Mar 4, 2025):
My apologies, Here are both blocks
@Vito0912 commented on GitHub (Mar 4, 2025):
This supports my suspicion. If that's the case and I can reproduce it, I'll make a PR to fix it.
@nichwall commented on GitHub (Mar 4, 2025):
Yeah, I agree that it is probably a missing description. Can we move that discussion to a new issue so this remains focused on the semicolon being added?
@advplyr commented on GitHub (Apr 4, 2025):
Was this ever figured out?
@nichwall commented on GitHub (Apr 4, 2025):
I believe it was narrowed down to being something in the routing setup, but nothing definitive was found.
The logs from ABS do not show the semicolon being added, but the semicolon is present when the request is received by abs-tract.
@Vito0912 commented on GitHub (Apr 13, 2025):
Because I just read this again.
That's not the case. If it were, then this would be 100% due to a misconfigured proxy.
See: https://github.com/advplyr/audiobookshelf/issues/4004#issuecomment-2692161382
@ahobsonsayers commented on GitHub (Jul 19, 2025):
Apologies for reviving this - I've read the replies but I'm still not clear if ABS has resolved this issue.
As the author of abs-tract, is there anything I can do to help? If we can't figure out where the ; is coming from, maybe I could just do some handling my side. This is obviously not the preferred solution, but if it's required to resolve this, then I guess I can see what I can do.
If I helps at all, I have had another issue opened about this, and the issue seems to imply it only occurs on quick match. Maybe this could help replication, as I know people were having trouble recreating it?
See https://github.com/ahobsonsayers/abs-tract/issues/23
@advplyr commented on GitHub (Jul 19, 2025):
The linked issue looks to be unrelated to this. There is no semicolon in the query param.
The only thing I can think of with this issue is the URLSearchParams.toString function has a bug with encoding. Since no one has been able to reproduce that semicolon my guess is it is specific to the users setup. Something with character encoding.
We can see the issue is not abs-tract and not proxy/routing because the URL has the semicolon before making the request.
For the issue you linked, that is definitely a bug with Abs because it is an unhandled rejection. We should open a separate issue for that. If you can get the full payload that is coming back from that request that would help