mirror of
https://github.com/photoprism/photoprism.git
synced 2026-03-02 22:57:18 -05:00
Metadata: Prefer SubSecDateTimeOriginal and SubSecCreateDate timestamps #1431
Labels
No labels
ai
android
api
auth
awesome
bug
bug
ci
cli
config
database
declined
deprecated
docker
docs 📚
documents
duplicate
easy
enhancement
enhancement
enhancement
epic
faces
feedback wanted
frontend
hacktoberfest
help wanted
idea
in-progress
incomplete
index
invalid
ios
labels
live
live
low-priority
macos
member-feature
metadata
mobile
nas
needs-analysis
no-coding-required
no-coding-required
observability
performance
places
please-test
plus-feature
priority
pro-feature
question
raspberry-pi
raw
released
released
released
research
resolved
security
sharing
tested
tests
third-party-issue
thumbnails
upgrade
upstream-issue
ux
vector
video
waiting
won't fix
won't fix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/photoprism#1431
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @danielhoherd on GitHub (May 13, 2022).
Originally assigned to: @lastzero on GitHub.
What does not work as described in the documentation?
When viewing collections of images, timezones are not considered when sorting photos. Photos from two separate cameras set to the same objective time but with a different timezone will not correctly sort chronologically. It appears that the location of the photo may be interfering with correct interpretation of the time.
Photo details views show incorrect TZ and time when the TZ of the camera body does not match the photo geolocation.
How can we reproduce it?
Steps to reproduce the behavior:
What behavior do you expect?
I expect photos that include the TZ in their exif data to be shown with correct times and sorted correctly among other files from different TZs.
What could be the cause of your problem?
It looks like there may be multiple things going on. The DB does not appear to store TZ data, although there is a
photos.taken_at_localcolumn. This column appears to be the result of some changes that take into account the location of the photo and the photo exif data. For example, the following photo shows 10:01 PM EDT.The photo was taken in EST, but clearly it is not 10pm EDT since it's daytime. The details view of that photo does indeed show that it is in America/New_York TZ.
However, the camera that took that photo was set to GMT (because who wants to mess with TZ's every time you travel?) We can see in the exif data that it was taken at 10:01 GMT (+00:00):
So my suspicion is that the import of photo metadata into Photoprism is ignoring fields that have more rich datetime data that includes the TZ. These fields are less ubiquitous than the TZ naive fields, but they are quite common in modern cameras, and if the TZ data is included in the exif then it should certainly be used. This is especially important for people who travel because they often forget to change the TZ on their camera, and end up shooting a bunch of photos with the clock set to the wrong timezone. If TZ data is available in the file and is used, then photoprism can gracefully deal with the photographer's mistake. If TZ data from the file is ignored, then we have chronological sort problems and incorrect metadata showing up in photo detail views.
Can you provide us with example files for testing, error logs, or screenshots?
I can provide sample files if asked to, but I don't think I have anything especially rare that you couldn't create by just using exiftool to change the dates on a file or using a DSLR with its clock set to another TZ.
Which software versions do you use?
On what kind of device is PhotoPrism installed?
lscpu)free)lsblk -do NAME,SIZE,MODEL)@lastzero commented on GitHub (May 16, 2022):
Support for SubSec timestamps was added recently. Read priority has been increased so that they are used first. Might be all that's needed! Please test 👍
@danielhoherd commented on GitHub (May 17, 2022):
@lastzero I just tested this on
photoprism/photoprism@sha256:299ea4b4862e9e0385856dcdb75be0b82aff9f3aa4e65139d3342df33246ed2ecreated at2022-05-17T01:56:40.667145444Z, and sadly the new build does not fix the problem.I looked in the database and
taken_atandtaken_at_localappear to be the same as they were before.time_zoneshows the geotagged location, not the TZ from the file.@lastzero commented on GitHub (May 17, 2022):
Your initial example showed time offsets +00:00, not time zones. Now your example shows a time zone, Z. However, reading the fields you mentioned first, leads to the DateCreated field being ignored. We can't read all fields and mix the information in creative ways.
@danielhoherd commented on GitHub (May 18, 2022):
@lastzero sorry, I've been using TZ and offset interchangeably. The original example didn't include the
DateCreatedfield, which is why the Z was not shown. The file I used in the original example (20220511-22-01-32_455c68.dng) does show the TZ forDateCreatedand an offset forCreateDate, just like the file in my prior comment (20211206-03-29-10_493FjF.dng) does. These two photos were taken with the same camera body and same TZ settings.@danielhoherd commented on GitHub (May 18, 2022):
Looking more at the exiftool side of this, exiftool does provide a composite
TimeZonefield via custom config. After adding those contents to my~/.ExifTool_configI now see my DSLR with GMT offset, and my iPhone with my local timezone offset. Here are the two files I used in examples above, and another iPhone file for comparison:So it is pretty simple to coax exiftool into providing the bare timezone from file metadata. With these pieces, it should be possible to add TZ metadata contained within the file to photoprism.
@lastzero commented on GitHub (May 18, 2022):
DateCreated is an XMP Field, DatesTimes in Exif for Images are not UTC but local time by default. Main exception are QuickTime and MP4 videos, which default to UTC. Since when does Nikon create DNG files with XMP Metadata? Writing this to XMP sidecar files instead should work for you. Note that even though Exiftool calls -7 a time zone, it's a time offset really.
@danielhoherd commented on GitHub (May 18, 2022):
AFAIK Nikon does not create DNG files. These were run through Adobe DNG Converter, and manually geotagged. Nikon does support TimeZone as a default property though:
If the feedback I'm leaving is not helpful then I can back off and let the experts do their thing. The original issue still stands though: Photoprism currently does not respect the offset of the clock in the camera that took the photo.
@lastzero commented on GitHub (May 18, 2022):
It's helpful to learn about your needs. However, time ZONE and OFFSET have defined, distinct meanings (independent of how Exiftool labels the value +01:00, it is not a zone): https://www.w3.org/International/core/2005/09/timezone.html
Money Quote:
@lastzero commented on GitHub (May 18, 2022):
If the timestamps have an offset, it is local time. This already is the default for Exif timestamps. If the metadata also contains the GPS coordinates, the ZONE and OFFSET should be correct and properly displayed. If the GPS coordinates are missing, then how does the Camera or other tools you use know the offset or zone?
@lastzero commented on GitHub (May 18, 2022):
Also note that the latest release will sort by local times instead of UTC (if available). We may add settings for this in a future release.
@danielhoherd commented on GitHub (May 18, 2022):
I have created some unmodified test images that show the problem and I can share them if you'd like. I have both JPG and NEF files available.
I set my D850 TZ to UTC +00:00, plugged in a GPS, and took some photos in America/Los_Angeles (-07:00). The camera was configured to set its clock by GPS time, so the clock is correct. This setting did not interfere with the camera's TZ being set to UTC.
Above, we see the the image was taken at 8:17pm +00:00 (utc). However, Photoprism shows it as 8:17pm PDT, which is incorrect.
This was tested in Photoprism Build 220517-b9c68f8f-Linux-AMD64 (docker image
photoprism/photoprism@sha256:299ea4b4862e9e0385856dcdb75be0b82aff9f3aa4e65139d3342df33246ed2e)@lastzero commented on GitHub (May 18, 2022):
You are welcome to send samples to the samples email address specified on our contact page. This is the first time someone uses UTC in a regular camera, maybe our other Exif library requires changes too or we have to prefer GPS time to handle this special case.
However, we need to be careful not to break existing functionality as local time is the default / standard. Exif does not include a time zone (which is the actual problem). These are custom Nikon Maker Notes in combination with logic the Author of Exiftool seemed to have developed for users like you:
@danielhoherd commented on GitHub (May 19, 2022):
I see several TZ and offset related tags the exif 2 tag reference https://exiv2.org/tags.html The two most relevant are:
OffsetTime is included by default Nikon D850 files:
And in iOS files:
Even so, it appears to be not widely supported. I downloaded all the raw files for Sony, Nikon and Canon from https://www.rawsamples.ch and not a single one of them had OffsetTime, or any TZ or Offset related information in EXIF. A lot of the Nikon and Canon files had
TimeZonein theirMakerNotesthough, but that's it. Nothing from Sony.Handling time is difficult, so it's not too surprising that there's no widely adopted standard. It's a shame though.
@lastzero commented on GitHub (May 19, 2022):
As you already noted, these fields are not commonly used. It would also not make much of a difference as by convention the time in Exif is local time. Any explicit offset just confirms that by providing the specific time difference to UTC. If your pictures were taken in LA, then UTC+0 is the wrong offset. It is important to understand that UTC+0 is not the same as UTC (Z):
@lastzero commented on GitHub (May 19, 2022):
The features you are requesting seem to be...
a) override any other timestamp with the GPS time if present, which is always UTC
and/or
b) automatically adjust the Exif local time using the matching offset for the location if the camera clock offset is present but wrong and there are GPS coordinates
Is that correct?
@danielhoherd commented on GitHub (May 19, 2022):
I'm really not sure if the implementation details you gave are exactly what I'm asking for, but if that's what it takes to solve the problem then that works for me.
Really?
@lastzero commented on GitHub (May 19, 2022):
Yes. As explained above +00:00 is an offset, not a zone. Z is the UTC zone. Any offset is as good as another offset and it may well change during the year if you stay at the exact same location:
https://en.m.wikipedia.org/wiki/UTC%C2%B100:00
@lastzero commented on GitHub (May 19, 2022):
Money Quote from the URL you provided:
CURRENTLY is key here. Zulu time or UTC always is the same, not just currently.
Edit: For me UTC+0 was a shortcut for +00:00 as I have a hard time typing all these details on a phone. My point always was to distinguish between a ZONE and an OFFSET.
@lastzero commented on GitHub (May 19, 2022):
I will try to explain it even better, and hope you don't find it too pedantic (back in the office now with a proper keyboard):
See
GPSDateTimeand the XMP create time for example:@lastzero commented on GitHub (Jun 1, 2022):
Since the prioritization of SubSecDateTimeOriginal and SubSecCreateDate has been released, we will close this issue. You can decide if you want to submit a follow-up issue, now that you (hopefully) have a detailed understanding of the situation.
Of course, feature requests should not conflict with what I wrote in the comments. In particular, we should not mix offsets and zones, as this will likely result in incorrect data in other places / for other users.