AI: Integrate Florence 2 Vision AI for Auto Caption #2129

New issue

Open

opened 2026-02-20 01:06:47 -05:00 by deekerman · 2 comments

deekerman commented

2026-02-20 01:06:47 -05:00

Owner

Originally created by @david-ng-hk on GitHub (Jun 26, 2024).

Microsoft release a new AI model recently, MIT license.
It will use Vision AI to auto generate a brief or detail caption for a photo or video.

I think this AI can generate the caption for the search in photo prism.
Please integrate, if not all GPU, then NVidia card go first.
Thank you.

model download
https://huggingface.co/microsoft/Florence-2-large-ft

demo
https://huggingface.co/spaces/SixOpen/Florence-2-large-ft

Originally created by @david-ng-hk on GitHub (Jun 26, 2024). Microsoft release a new AI model recently, MIT license. It will use Vision AI to auto generate a brief or detail caption for a photo or video. I think this AI can generate the caption for the search in photo prism. Please integrate, if not all GPU, then NVidia card go first. Thank you. model download https://huggingface.co/microsoft/Florence-2-large-ft demo https://huggingface.co/spaces/SixOpen/Florence-2-large-ft

deekerman added the

idea

help wanted

labels

2026-02-20 01:06:47 -05:00

deekerman commented

2026-02-20 01:06:48 -05:00

Author

Owner

@GlassedSilver commented on GitHub (Jun 27, 2024):

Played around with the demo for a bit and what can I say, seems pretty darn good!

MIT license isn't too shabby either.

@GlassedSilver commented on GitHub (Jun 27, 2024): Played around with the demo for a bit and what can I say, seems pretty darn good! MIT license isn't too shabby either.

deekerman commented

2026-02-20 01:06:48 -05:00

Author

Owner

@graciousgrey commented on GitHub (Dec 2, 2025):

Although there is no support for Florence 2 Vision, you can now use our Ollama integration to generate labels or captions:

@graciousgrey commented on GitHub (Dec 2, 2025): Although there is no support for Florence 2 Vision, you can now use our Ollama integration to generate labels or captions: - https://docs.photoprism.app/user-guide/ai/using-ollama/ - https://docs.photoprism.app/user-guide/ai/