starred/audiobookshelf-advplyr
1
0
Fork 0
mirror of https://github.com/advplyr/audiobookshelf.git synced 2026-03-02 22:46:55 -05:00
Code Issues 853 Projects Packages Wiki Activity

[Enhancement]: Adding Transcription/Subtitle Viewing Support #1942

New issue
Open
opened 2026-02-20 10:14:14 -05:00 by deekerman · 1 comment
deekerman commented 2026-02-20 10:14:14 -05:00
Owner
Copy link

Originally created by @mfcar on GitHub (May 4, 2024).

Describe the feature/enhancement

Transcription/Subtitle support

Summary

Add initial support for transcriptions. Apple now supports transcriptions in podcasts.

Some audiobook files have transcriptions, and currently, we can use tools based on Whisper to transcribe audio to text.

In fact, most software based on Whisper support transcribing audio to text and exporting it as an SRT or VTT file. VTT is a native format for the web, and SRT is a common format for subtitles.

I'm creating this issue to discuss the best way to implement transcription support on the web player. I'm trying to implement some features on the pull request #

Podcast transcription is supported by:

Possible tasks:

  • Implement support for parsing and displaying VTT files. WebVTT #2918
  • Implement support for parsing and displaying SRT files (convert to vtt or try to parse directly?).
  • Support <podcast:transcript> tag in RSS feed. Apple Docs / Podcast Namespace
  • Handle CORS issues when fetching transcription files from external sources. Podcasting 2.0 CORS
  • Implement logic to read transcription from a file with the same name as the audio file. #2918
  • Define priority if both VTT and SRT files exist with the same name?
  • When a podcast has a transcription tag, automatically download the transcription and store it in the file system (offline mode).
  • Add a visual indication in the UI that a podcast/audiobook has a transcription available?
  • Implement a feature to search within the transcription (better if using a lateral panel)?
  • Implement a feature to highlight the current line of the transcription as the podcast/audiobook plays. #2918
  • Implement a feature to navigate to a specific part of the podcast/audiobook by clicking on the transcription text. Seek to the corresponding time in the audio. #2918
  • Implement a feature to toggle the display of the transcription on/off on the web player. #2918
  • Implement a button to download the transcription file? Can be useful for editing or sharing. #2918
  • Allows to upload a VTT file for transcription?

Note: I think we need define a standard for multi-language transcriptions. For example use some prefix in the file name like en- for English and es- for Spanish.

UI Ideas on the Web Player:

What's the best way to display the transcription on the web player?

  • Above/Below audio player controls. #2918
    • Good: because the audio player is omnipresent on all pages and the transcription can be displayed in a fixed position.
    • Bad: Not enough space to display multiple lines of text.
Screenshot 2024-05-04 at 20 47 33
  • Modal.
    • Good: More space to display multiple lines of text. Can float over the UI.
    • Bad: The modal can be intrusive.
  • Lateral Panel. (like the iTunes/Apple Music).
    • Good: More space to display multiple lines of text. Can have a search feature. Better for implement the seek feature (click on the line and seek to the corresponding time).
    • Bad: Take up space on the screen. Not good for small screens.
Screenshot 2024-05-02 at 09 09 47

Related

Originally created by @mfcar on GitHub (May 4, 2024). ### Describe the feature/enhancement Transcription/Subtitle support === ### Summary Add initial support for transcriptions. Apple now supports transcriptions in podcasts. Some audiobook files have transcriptions, and currently, we can use tools based on [Whisper](https://github.com/openai/whisper) to transcribe audio to text. In fact, most software based on Whisper support transcribing audio to text and exporting it as an [SRT](https://en.wikipedia.org/wiki/SubRip) or [VTT](https://en.wikipedia.org/wiki/WebVTT) file. VTT is a native format for the web, and SRT is a common format for subtitles. I'm creating this issue to discuss the best way to implement transcription support on the web player. I'm trying to implement some features on the pull request # ### Podcast transcription is supported by: - [ ] https://podcasting2.org/podcast-namespace/tags/transcript ### Possible tasks: - [ ] Implement support for parsing and displaying VTT files. [WebVTT](https://developer.mozilla.org/en-US/docs/Web/API/WebVTT_API) #2918 - [ ] Implement support for parsing and displaying SRT files (convert to vtt or try to parse directly?). - [ ] Support `<podcast:transcript>` tag in RSS feed. [Apple Docs](https://help.apple.com/itc/podcasts_connect/#/itcb54353390) / [Podcast Namespace](https://github.com/Podcastindex-org/podcast-namespace) - [ ] Handle CORS issues when fetching transcription files from external sources. [Podcasting 2.0 CORS](https://github.com/Podcastindex-org/podcast-namespace/blob/main/podcasting2.0.md#cors) - [ ] Implement logic to read transcription from a file with the same name as the audio file. #2918 - [ ] Define priority if both VTT and SRT files exist with the same name? - [ ] When a podcast has a transcription tag, automatically download the transcription and store it in the file system (offline mode). - [ ] Add a visual indication in the UI that a podcast/audiobook has a transcription available? - [ ] Implement a feature to search within the transcription (better if using a lateral panel)? - [ ] Implement a feature to highlight the current line of the transcription as the podcast/audiobook plays. #2918 - [ ] Implement a feature to navigate to a specific part of the podcast/audiobook by clicking on the transcription text. Seek to the corresponding time in the audio. #2918 - [ ] Implement a feature to toggle the display of the transcription on/off on the web player. #2918 - [ ] Implement a button to download the transcription file? Can be useful for editing or sharing. #2918 - [ ] Allows to upload a VTT file for transcription? > Note: I think we need define a standard for multi-language transcriptions. For example use some prefix in the file name like `en-` for English and `es-` for Spanish. ### UI Ideas on the Web Player: What's the best way to display the transcription on the web player? - [ ] Above/Below audio player controls. #2918 - Good: because the audio player is omnipresent on all pages and the transcription can be displayed in a fixed position. - Bad: Not enough space to display multiple lines of text. <img width="989" alt="Screenshot 2024-05-04 at 20 47 33" src="https://github.com/advplyr/audiobookshelf/assets/814828/4ba864bb-c2cf-4ef3-8bc6-5affadc3d52d"> --- - [ ] Modal. - Good: More space to display multiple lines of text. Can float over the UI. - Bad: The modal can be intrusive. - [ ] Lateral Panel. (like the iTunes/Apple Music). - Good: More space to display multiple lines of text. Can have a search feature. Better for implement the seek feature (click on the line and seek to the corresponding time). - Bad: Take up space on the screen. Not good for small screens. <img width="1219" alt="Screenshot 2024-05-02 at 09 09 47" src="https://github.com/advplyr/audiobookshelf/assets/814828/f1e8320d-ae6c-4ce8-bb39-1c37c05aa0ab"> ### Related - #1723
deekerman commented 2026-02-20 10:14:14 -05:00
Author
Owner
Copy link

@zkvsky commented on GitHub (Jun 24, 2024):

IMHO, when it comes to the UI you should combine both, the big panel for browsing and the panel below controls perhaps just with the current line but bigger. You have to take into account accessibility, some folks will want it to be resizable.

@zkvsky commented on GitHub (Jun 24, 2024): IMHO, when it comes to the UI you should combine both, the big panel for browsing and the panel below controls perhaps just with the current line but bigger. You have to take into account accessibility, some folks will want it to be resizable.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
book_tags_genres_dedupe
episode_download_fallback
Issue-4540-SortBy-StartedDate-and-FinishedDate
episode_meta_tagging
fix_authorize_race_condition
redirect_transcode_requests
progress_updated_sort
fix_ereader_socket_event
fix_change_empty_root_password
fix_podcast_session_track_index
fix_set_token
session_modal_user
localize_durations
fix_oidc_create_user
jwt_auth_refactor
fix_scanner_deleting_single_file_books
fix_mediaprogress_updatedat_2
experimental_next_client
podcast_episode_duration
episode-timestamps-clickable
book_author_secondary_sort_title
podcast_useragents
pathexists_user_access
fix_pathexists_join
book_author_secondary_sort
clean_duplicate_mediaprogress
sanitize_html_description
trix_prevent_attachments
check_path_api_fix
fix_mediaprogress_updatedat
increase_express_json_limit
fix_dockerfile_nunicode
search_episodes
audiobook_tools_update
episode_secondary_sorts
hls_stream_url_update
new_session_track_endpoint
audiobook_tools_enhancements
watcher_rescans_update
player_track_tooltip
fix_exclude_prefixes_crash
socket_item_events
fix_podcast_episode_scanner_promise
new_stats_controller
count_cache_for_userpermissions
parsing-opf-v3
validate_migration_files
fix-quick-match-all-crash
fix-chapter-end-sleep-timer
stringify_sequelize_query
remove-col-ambiguity
fix_next_prev_edit_description
details_trim_whitespace
fix_content_url_basepath
fix_logger_fatal
progress_bar_visibility
batch-edit-populate-map-details
feed_generator_updates
bookmark-modal-updates
migrate-library-item-in-scanner
migrate-new-library-items
migrate-podcasts-new-library-item-2
migrate-podcasts-new-library-item
fix-remove-episode-from-playlist
playback-session-use-new-library-item
refactor-library-item
fix-heatmap-caption
feed-episodes-upsert
share-media-player-media-session-api
remove-old-playlist
remove_old_collection_object
plugin-implementation-demo
feed_migration
refactor-feeds-from-item
fix_remove_authors_no_books
v2.17.3-fk-constraints-migration
migrations-first-upgrade
sqlite_2
feature/nuxt-target-server
waveform
sqlite
playlists
video
v2.32.1
v2.32.0
v2.31.0
v2.30.0
v2.29.0
v2.28.0
v2.27.0
v2.26.3
v2.26.2
v2.26.1
v2.26.0
v2.25.1
v2.25.0
v2.24.0
v2.23.0
v2.22.0
v2.21.0
v2.20.0
v2.19.5
v2.19.4
v2.19.3
v2.19.2
v2.19.1
v2.19.0
v2.18.1
v2.18.0
v2.17.7
v2.17.6
v2.17.5
v2.17.4
v2.17.3
v2.17.2
v2.17.1
v2.17.0
v2.16.2
v2.16.1
v2.16.0
v2.15.1
v2.15.0
v2.14.0
v2.13.4
v2.13.3
v2.13.2
v2.13.1
v2.13.0
v2.12.3
v2.12.2
v2.12.1
v2.12.0
v2.11.0
v2.10.1
v2.10.0
v2.9.0
v2.8.1
v2.8.0
v2.7.2
v2.7.1
v2.7.0
v2.6.0
v2.5.0
v2.4.4
v2.4.3
v2.4.2
v2.4.1
v2.4.0
v2.3.5
v2.3.4
v2.3.3
v2.3.2
v2.3.1
v2.3.0
v2.2.23
v2.2.22
v2.2.21
v2.2.20
v2.2.19
v2.2.18
v2.2.17
v2.2.16
v2.2.15
v2.2.14
v2.2.13
v2.2.12
v2.2.11
v2.2.10
v2.2.9
v2.2.8
v2.2.7
v2.2.6
v2.2.5
v2.2.4
v2.2.3
v2.2.2
v2.2.1
v2.2.0
v2.1.5
v2.1.4
v2.1.3
v2.1.2
v2.1.1
v2.1.0
v2.0.24
v2.0.23
v2.0.22
v2.0.21
v2.0.20
v2.0.19
v2.0.18
v2.0.17
v2.0.16
v2.0.15
v2.0.14
v2.0.13
v2.0.12
v2.0.11
v2.0.10
v2.0.9
v2.0.8
v2.0.7
v2.0.6
v2.0.5
v2.0.4
v2.0.3
v2.0.2
v2.0.1
v1.7.2
v1.7.1
v1.7.0
v1.6.0
v1.5.5
v1.5.0
v1.4.11
v1.4.9
v1.4.7
v1.4.6
v1.4.4
v1.4.2
v1.4.1
v1.4.0
v1.3.4
v1.3.3
v1.3.1
v1.2.8
v1.2.6
v1.2.5
v1.2.4
v1.2.1
v1.1.15
v1.1.14
v1.1.13
v1.1.12
v1.1.11
v1.1.10
v1.1.9
v1.1.8
v1.0.0
0.9.61-beta.0
0.9.61-beta
Labels
Clear labels   
authentication

   
awaiting release

   
backlog

   
bug

   
chapter editor

   
config-issue

   
ebooks

   
encoding/embedding

   
enhancement

   
help wanted

   
listening sessions & progress

   
planned

   
possible plugin

   
progress sync

   
sorting/filtering/searching

   
unable to reproduce

   
upload

   
users & permissions

   
waiting
No labels
authentication
awaiting release
backlog
bug
chapter editor
config-issue
ebooks
encoding/embedding
enhancement
help wanted
listening sessions & progress
planned
possible plugin
progress sync
sorting/filtering/searching
unable to reproduce
upload
users & permissions
waiting
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/audiobookshelf-advplyr#1942
No description provided.