People: Sort autocomplete list of people based on metadata (when assigning to face) #1273

Owner

Originally created by @joachimtingvold on GitHub (Dec 31, 2021).

When searching for people to assign a face, the list could maybe be sorted/prioritized based on different metadata. Right now, the list is long if you have many persons, and it's tedious to write the whole name, or scroll through the list with mouse/keyboard.

people from photos in the same album(s)
people from photos taken at the same place and/or same time
relevant people already tagged in the same photo (based on how many times they have been in the same photo before)
other people from same moment (if photo is part of a moment)
relation to people based on some or all of the other results above (relevant to #144)
prioritize given name before surname (or the other way, depending on preference), as opposed to search from the beginning of the name (as sometimes you need to filter by surname)
prioritize beginning of given name and surname, before "middle of" (i.e. search for "foo" should prioritize "Zulu Foobar" before "Allan Barfoo")
number of photos they are already tagged in (i.e. a person that is in 1k photos is more likely to be in another photo, than one that is only in 10 photos)

Not sure if all of these would be sensible in terms of performance. Also not sure of the prioritization of these.

Originally created by @joachimtingvold on GitHub (Dec 31, 2021). When searching for people to assign a face, the list could maybe be sorted/prioritized based on different metadata. Right now, the list is long if you have many persons, and it's tedious to write the whole name, or scroll through the list with mouse/keyboard. * people from photos in the same album(s) * people from photos taken at the same place and/or same time * relevant people already tagged in the same photo (based on how many times they have been in the same photo before) * other people from same moment (if photo is part of a moment) * relation to people based on some or all of the other results above (relevant to #144) * prioritize given name before surname (or the other way, depending on preference), as opposed to search from the beginning of the name (as sometimes you need to filter by surname) * prioritize beginning of given name and surname, before "middle of" (i.e. search for "foo" should prioritize "Zulu Foobar" before "Allan Barfoo") * number of photos they are already tagged in (i.e. a person that is in 1k photos is more likely to be in another photo, than one that is only in 10 photos) Not sure if all of these would be sensible in terms of performance. Also not sure of the prioritization of these.

deekerman added the

idea

faces

needs-analysis

labels

deekerman commented

Author

Owner

@lastzero commented on GitHub (Dec 31, 2021):

Tried that, but turned out to be confusing.

@lastzero commented on GitHub (Dec 31, 2021): Tried that, but turned out to be confusing.

deekerman commented

Author

Owner

@joachimtingvold commented on GitHub (Dec 31, 2021):

Maybe make it configureable/user preference, then? I guess it's somewhat personal taste to some degree. And I guess some of them makes more sense than others.

You usually refer to people by given name or surname, not something "in the middle". Alas, searching from the start of given name and/or surname would make more sense than the current search.

As an example, search for "foo" should sort like this (where first part is given name, and last part is surname):

Foobar Baz
Zulu Foobar
Allan Barfoo

... which is currently not the case (regardless of all the other metadata). Not sure if that's just my opinion, though.

@joachimtingvold commented on GitHub (Dec 31, 2021): Maybe make it configureable/user preference, then? I guess it's somewhat personal taste to some degree. And I guess some of them makes more sense than others. You usually refer to people by given name or surname, not something "in the middle". Alas, searching from the start of given name and/or surname would make more sense than the current search. As an example, search for "foo" should sort like this (where first part is given name, and last part is surname): 1. Foobar Baz 2. Zulu Foobar 3. Allan Barfoo ... which is currently not the case (regardless of all the other metadata). Not sure if that's just my opinion, though.

deekerman commented

Author

Owner

@graciousgrey commented on GitHub (Jan 1, 2022):

I agree sorting can be improved but we need to do more research to find a solution that is not confusing.

We are currently focusing on multi-user support.

Let's keep this issue open to hear more opinions :)

@graciousgrey commented on GitHub (Jan 1, 2022): I agree sorting can be improved but we need to do more research to find a solution that is not confusing. We are currently focusing on multi-user support. Let's keep this issue open to hear more opinions :)

deekerman commented

Author

Owner

@srett commented on GitHub (Jan 1, 2022):

DigiKam bumps the last 10 (or so) people you assigned to a marker to the top of the list and prints them in bold, and likewise, when typing part of a name, those same 10 people, if matching, are displayed at the top, again in bold. It's rather primitive but still effective, and probably less confusing than adding a bunch of magic and heuristics.

@srett commented on GitHub (Jan 1, 2022): DigiKam bumps the last 10 (or so) people you assigned to a marker to the top of the list and prints them in bold, and likewise, when typing part of a name, those same 10 people, if matching, are displayed at the top, again in bold. It's rather primitive but still effective, and probably less confusing than adding a bunch of magic and heuristics.

deekerman commented

Author

Owner

@joachimtingvold commented on GitHub (Jan 1, 2022):

I have not used DigiKam, but I guess it somewhat depends on the context you are assigning them from? (cluster, album, image, etc).

If you are browsing an album, people assigned to other photos in the same album would be most likely to reappear. If you are going through photos already assigned to a person (to verify and/or assign other people in those photos), you can suddenly span albums/events/years/whatnot, where "last used" might not make much sense?

Maybe I'm overthinking this :D

@joachimtingvold commented on GitHub (Jan 1, 2022): I have not used DigiKam, but I guess it somewhat depends on the context you are assigning them from? (cluster, album, image, etc). If you are browsing an album, people assigned to other photos in the same album would be most likely to reappear. If you are going through photos already assigned to a person (to verify and/or assign other people in those photos), you can suddenly span albums/events/years/whatnot, where "last used" might not make much sense? Maybe I'm overthinking this :D

deekerman commented

Author

Owner

@srett commented on GitHub (Jan 1, 2022):

If you are going through photos already assigned to a person (to verify and/or assign other people in those photos), you can suddenly span albums/events/years/whatnot, where "last used" might not make much sense?

I'd optimize for the common case, which is (assuming you finished your initial import) adding new photos every now and then, which will be mostly from one or a few events, with recurring people.
But even in the scenario you describe, you will most likely go through the pictures of that person in chronological order, so if you fix assignments, it's probably the same few people again, e.g. Bob got confused with Bill a couple times at Jane's wedding, then the next time Bob appears in your collection is some trip in the Alps, where he got tagged as John a few times, and so on.
From my experience I can say it works well enough to be convenient and not get in your way.

I don't think it's worth it adding 5 different heuristics for suggestions plus a config dialog, or even trying to pick the right one automatically (and getting it wrong sometimes for maximum confusion).

@srett commented on GitHub (Jan 1, 2022): > If you are going through photos already assigned to a person (to verify and/or assign other people in those photos), you can suddenly span albums/events/years/whatnot, where "last used" might not make much sense? I'd optimize for the common case, which is (assuming you finished your initial import) adding new photos every now and then, which will be mostly from one or a few events, with recurring people. But even in the scenario you describe, you will most likely go through the pictures of that person in chronological order, so if you fix assignments, it's probably the same few people again, e.g. Bob got confused with Bill a couple times at Jane's wedding, then the next time Bob appears in your collection is some trip in the Alps, where he got tagged as John a few times, and so on. From my experience I can say it works well enough to be convenient and not get in your way. I don't think it's worth it adding 5 different heuristics for suggestions plus a config dialog, or even trying to pick the right one automatically (and getting it wrong sometimes for maximum confusion).

deekerman commented

Author

Owner

@joachimtingvold commented on GitHub (Jan 1, 2022):

I'd optimize for the common case, which is (assuming you finished your initial import) adding new photos every now and then, which will be mostly from one or a few events, with recurring people.

I don't disagree, but the "initial import" can be a big pain depending on the size. I'm migrating to PP from a self-made image system, and I have about 70k+ pictures that are sorted into albums, and probably equally many unsorted. I'm a few days into importing them into PP now. and my wrists are not happy with me at all. Mostly due to the bad/non-optimal workflow of assigning persons to faces/photos (but also some other workflows).

I'm not complaining, and I know we're very early in the whole facial recognition implementation, but yeah.

I don't necessarily mean that we should implement all 5 heuristics (and there might be other ones that is more suitable, like the one you suggested), but we should at least improve the current search significantly. I have only scratched the surface of my import, and I already have 300+ people (and only about 30-40k of my 120k+ images imported). Maybe I'm an outlier, but that shouldn't be an excuse to improve on things.

@joachimtingvold commented on GitHub (Jan 1, 2022): > I'd optimize for the common case, which is (assuming you finished your initial import) adding new photos every now and then, which will be mostly from one or a few events, with recurring people. I don't disagree, but the "initial import" can be a big pain depending on the size. I'm migrating to PP from a self-made image system, and I have about 70k+ pictures that are sorted into albums, and probably equally many unsorted. I'm a few days into importing them into PP now. and my wrists are not happy with me at all. Mostly due to the bad/non-optimal workflow of assigning persons to faces/photos (but also some other workflows). I'm not complaining, and I know we're very early in the whole facial recognition implementation, but yeah. I don't necessarily mean that we should implement all 5 heuristics (and there might be other ones that is more suitable, like the one you suggested), but we should at least improve the current search significantly. I have only scratched the surface of my import, and I already have 300+ people (and only about 30-40k of my 120k+ images imported). Maybe I'm an outlier, but that shouldn't be an excuse to improve on things.

deekerman commented

Author

Owner

@IeuanK commented on GitHub (May 7, 2022):

As per my now-closed issue #2305, I think it would be ideal to sort primarily by AI confidence.
Given that this issue has other ideas, it would be neat to make it configurable; have an option to store confidence levels for all people/faces and some options for which factors you want to use when sorting the dropdown?

I really hope this gets added at some point because it would make fixing faces a lot less of a pain.

@IeuanK commented on GitHub (May 7, 2022): As per my now-closed issue #2305, I think it would be ideal to sort primarily by AI confidence. Given that this issue has other ideas, it would be neat to make it configurable; have an option to store confidence levels for all people/faces and some options for which factors you want to use when sorting the dropdown? I really hope this gets added at some point because it would make fixing faces a lot less of a pain.

deekerman commented

Author

Owner

@Sohalt commented on GitHub (Jul 30, 2022):

I very much agree that complicated heuristics can be too confusing, but the suggestion in https://github.com/photoprism/photoprism/issues/1865#issuecomment-1003487639 is very simple and a huge improvement over the current state imho (highest priority for exact match, then prefix match, then prefix of subword (e.g. surname), then any other substring match).
If I try to search for "Maria", it's extremely annoying to have "Alma" show up first when typing "Ma".

@Sohalt commented on GitHub (Jul 30, 2022): I very much agree that complicated heuristics can be too confusing, but the suggestion in https://github.com/photoprism/photoprism/issues/1865#issuecomment-1003487639 is very simple and a huge improvement over the current state imho (highest priority for exact match, then prefix match, then prefix of subword (e.g. surname), then any other substring match). If I try to search for "Maria", it's extremely annoying to have "Alma" show up first when typing "Ma".

deekerman commented