HEIC: Converted JPG has wrong orientation #806

Author

Owner

@lastzero commented on GitHub (Feb 17, 2021):

If you need JPEGs, just use them straight away so that you don't need extra storage and transcoding. Most software doesn't support HEIC yet or just with tradeoffs like in this case.

@lastzero commented on GitHub (Feb 17, 2021): If you need JPEGs, just use them straight away so that you don't need extra storage and transcoding. Most software doesn't support HEIC yet or just with tradeoffs like in this case.

deekerman commented

Author

Owner

@lastzero commented on GitHub (Feb 20, 2021):

I'd be great to have test images for debugging... you're welcome to mail them to hello@photoprism.app

Please let us know if we have permission to add them to our public test suite to avoid regressions 👍 No problem if not, we'll help you anyway.

@lastzero commented on GitHub (Feb 20, 2021): I'd be great to have test images for debugging... you're welcome to mail them to hello@photoprism.app Please let us know if we have permission to add them to our public test suite to avoid regressions :+1: No problem if not, we'll help you anyway.

deekerman commented

Author

Owner

@lastzero commented on GitHub (Feb 21, 2021):

As it turns out, heif-convert (the reference implementation) has a bug that rotates the image but keeps the orientation in the Exif header - thus all images get rotated twice UNLESS Exif is not supported. Obviously we support Exif.

Apparently HEIF support may be compiled in imagemagick, but their compiled 7.x version doesn't include it. Ubuntu still ships 6.x. Fail.

We then tried to compile the latest version from https://github.com/strukturag/libheif on our own:

$ heif-convert 20210131_IMG_8219.heic 20210131_IMG_8219.heic.jpeg
File contains 1 images
Could not decode image: 0: Unsupported feature: Unsupported codec

The whole point of the tool is to read this file format. Unsupported Codec? Maybe JPEG? It's pretty new and probably needs extra work 👯

@lastzero commented on GitHub (Feb 21, 2021): As it turns out, `heif-convert` (the reference implementation) has a bug that rotates the image but keeps the orientation in the Exif header - thus all images get rotated twice UNLESS Exif is not supported. Obviously we support Exif. Apparently HEIF support may be compiled in imagemagick, but their compiled 7.x version doesn't include it. Ubuntu still ships 6.x. Fail. We then tried to compile the latest version from https://github.com/strukturag/libheif on our own: ``` $ heif-convert 20210131_IMG_8219.heic 20210131_IMG_8219.heic.jpeg File contains 1 images Could not decode image: 0: Unsupported feature: Unsupported codec ``` The whole point of the tool is to read this file format. Unsupported Codec? Maybe JPEG? It's pretty new and probably needs extra work :dancers:

deekerman commented

Author

Owner

@lastzero commented on GitHub (Feb 21, 2021):

Once this dependency is fixed, it should work just fine. It will also work when installed directly on a Mac as we use Apple's sips command there, see https://ss64.com/osx/sips.html.

The refactoring lays the foundation for further improvements like #464.

@lastzero commented on GitHub (Feb 21, 2021): Once this dependency is fixed, it should work just fine. It will also work when installed directly on a Mac as we use Apple's `sips` command there, see https://ss64.com/osx/sips.html. The refactoring lays the foundation for further improvements like #464.

deekerman commented

Author

Owner

@lastzero commented on GitHub (Feb 22, 2021):

Found another NodeJS-based converter, but it was 12 times slower and didn't preserve any Exif data at all.

Ended up implementing a wrapper for heif-convert that sets the orientation of all JPEG output files to 1. Part of today's update.

@lastzero commented on GitHub (Feb 22, 2021): Found another NodeJS-based converter, but it was 12 times slower and didn't preserve any Exif data at all. Ended up implementing a wrapper for `heif-convert` that sets the orientation of all JPEG output files to 1. Part of today's update.

deekerman commented

Author

Owner

@melyux commented on GitHub (Sep 17, 2022):

I'm still having this issue where the .heic files in portrait orientation are always rotated 90 degrees into landscape when converted to JPEG for preview.

@melyux commented on GitHub (Sep 17, 2022): I'm still having this issue where the .heic files in portrait orientation are always rotated 90 degrees into landscape when converted to JPEG for preview.

deekerman commented

Author

Owner

@lastzero commented on GitHub (Sep 17, 2022):

Could you provide us with a test file and specify which version of PhotoPrism you are using? Note that we offer a number of different Docker images, for example based on Ubuntu or Debian. Also note that previously created JPEGs will not be replaced after upgrading to a (hopefully) fixed version, so you will need to recreate them.

@lastzero commented on GitHub (Sep 17, 2022): Could you provide us with a test file and specify which version of PhotoPrism you are using? Note that we offer a number of different Docker images, for example based on Ubuntu or Debian. Also note that previously created JPEGs will not be replaced after upgrading to a (hopefully) fixed version, so you will need to recreate them.

deekerman commented

Author

Owner

@melyux commented on GitHub (Sep 17, 2022):

Sure, is there an email I can send a sample file to? I saw a couple floating around.

I'm using the photoprism/photoprism:latest image, and I first installed it today, so it's a brand new library and installation. The only change I made is turn Dynamic Previews on, but there doesn't seem to be a difference before/after turning that on/off.

@melyux commented on GitHub (Sep 17, 2022): Sure, is there an email I can send a sample file to? I saw a couple floating around. I'm using the photoprism/photoprism:latest image, and I first installed it today, so it's a brand new library and installation. The only change I made is turn Dynamic Previews on, but there doesn't seem to be a difference before/after turning that on/off.

deekerman commented

Author

Owner

@lastzero commented on GitHub (Sep 17, 2022):

Yes, it is described on our website: https://photoprism.app/contact

Sample image and video files can be sent to samples@photoprism.app. Please include the format and GitHub issue number or other helpful reference in the subject line, and let us know if we have permission to upload your files to dl.photoprism.app/samples for future use.

It might also be possible to attach files to comments on GitHub Issues, or you could add a link to a file uploaded to your own server and service like Google Drive.

@lastzero commented on GitHub (Sep 17, 2022): Yes, it is described on our website: https://photoprism.app/contact Sample image and video files can be sent to [samples@photoprism.app](mailto:samples@photoprism.app). Please include the format and [GitHub issue number](https://github.com/photoprism/photoprism/issues) or other helpful reference in the subject line, and let us know if we have permission to upload your files to [dl.photoprism.app/samples](https://dl.photoprism.app/samples/) for future use. It might also be possible to attach files to comments on GitHub Issues, or you could add a link to a file uploaded to your own server and service like Google Drive.

deekerman commented

Author

Owner

@melyux commented on GitHub (Sep 17, 2022):

Sent an email with two original HEIC images, a converted JPEG version of one of them, and screenshots of the orientations shown in the PhotoPrism UI. Hope that helps

@melyux commented on GitHub (Sep 17, 2022): Sent an email with two original HEIC images, a converted JPEG version of one of them, and screenshots of the orientations shown in the PhotoPrism UI. Hope that helps

deekerman commented

Author

Owner

@lastzero commented on GitHub (Sep 17, 2022):

Unfortunately, we could not reproduce this. Note that even if you use an image with the :latest tag, Docker does not automatically download new images for you and the current image can theoretically be out of date.

You can perform a manual upgrade as described in our docs at https://docs.photoprism.app/getting-started/updates/#docker-compose.

Some tools like Portainer also set a custom PATH variable that can change the behavior of our Docker images so that the wrong converter is used (we use a wrapper script that must be called instead of the binary in /usr/bin).

@lastzero commented on GitHub (Sep 17, 2022): Unfortunately, we could not reproduce this. Note that even if you use an image with the `:latest` tag, Docker does not automatically download new images for you and the current image can theoretically be out of date. You can perform a manual upgrade as described in our docs at https://docs.photoprism.app/getting-started/updates/#docker-compose. Some tools like Portainer also set a custom `PATH` variable that can change the behavior of our Docker images so that the wrong converter is used (we use a wrapper script that must be called instead of the binary in `/usr/bin`).

deekerman commented

Author

Owner

@lastzero commented on GitHub (Sep 17, 2022):

@melyux In the meantime, I was able to reproduce the problem. Good catch, thanks for reporting it! 🥳

It was not reproducible in our development environment, so I suspect it only occurs in production if /bin is symlinked to /usr/bin. As far as I remember, this is one of the changes in the new Ubuntu and Debian versions.

My changes in the commit above should fix the issue by changing the order of the search paths so that the wrapper script is found first again.

@lastzero commented on GitHub (Sep 17, 2022): @melyux In the meantime, I was able to reproduce the problem. Good catch, thanks for reporting it! :partying_face: It was not reproducible in our development environment, so I suspect it only occurs in production if `/bin` is symlinked to `/usr/bin`. As far as I remember, this is one of the changes in the new Ubuntu and Debian versions. **My changes in the commit above should fix the issue by changing the order of the search paths so that the wrapper script is found first again.**

deekerman commented

Author

Owner

@lastzero commented on GitHub (Sep 17, 2022):

Our preview image and demo have been updated so you can test if it works for you. I have also included a link to this issue in our release notes: https://docs.photoprism.app/release-notes/#development-preview

@lastzero commented on GitHub (Sep 17, 2022): Our preview image and demo have been updated so you can test if it works for you. I have also included a link to this issue in our release notes: https://docs.photoprism.app/release-notes/#development-preview

deekerman commented

Author

Owner

@lastzero commented on GitHub (Sep 17, 2022):

@melyux While the Ubuntu 22.04 LTS "Jammy" preview image now works as expected, the orientation is (still) not correct with Debian 12 "Bookworm", see https://demo-debian.photoprism.app/. It was already oriented differently there before, which could be your actual issue.

My guess is that they changed the behavior of the heif-convert command so that the problem we fixed with our wrapper script is now fixed there directly, resulting in a duplicate fix that eventually leads to a bug again.

Note the different version numbers of libheif-examples below. It would help us if you could test these with your files so we know if anything has changed and maybe even what...

Debian

Ubuntu

@lastzero commented on GitHub (Sep 17, 2022): @melyux While the Ubuntu 22.04 LTS "Jammy" preview image now works as expected, the orientation is (still) not correct with Debian 12 "Bookworm", see <https://demo-debian.photoprism.app/>. It was already oriented differently there before, which could be your actual issue. My guess is that they changed the behavior of the `heif-convert` command so that the problem we fixed with our wrapper script is now fixed there directly, resulting in a duplicate fix that eventually leads to a bug again. Note the different version numbers of `libheif-examples` below. It would help us if you could test these with your files so we know if anything has changed and maybe even what... ### Debian ![debian-libheif](https://user-images.githubusercontent.com/301686/190874693-b7220f0a-ec86-4ffb-a50c-6a3f67b9d239.png) ### Ubuntu ![ubuntu-libheif](https://user-images.githubusercontent.com/301686/190874695-d0c8b5b2-cf8e-47c3-95c4-6fac321600ae.png)

deekerman commented

Author

Owner

@melyux commented on GitHub (Sep 17, 2022):

Can confirm the rotation in the previews is correct now using the :preview image. (Side note: could you clarify what the Dynamic Previews option does if it still generates 11 static thumbnails per image? What exactly does the option do then?)

I'd like to help. How can I test the Debian and Ubuntu ones? Do you mean test the 0.12 vs 0.13 libheif versions somehow, or test the Debian vs Ubuntu photoprism images?

@melyux commented on GitHub (Sep 17, 2022): Can confirm the rotation in the previews is correct now using the `:preview` image. (Side note: could you clarify what the Dynamic Previews option does if it still generates 11 static thumbnails per image? What exactly does the option do then?) I'd like to help. How can I test the Debian and Ubuntu ones? Do you mean test the 0.12 vs 0.13 libheif versions somehow, or test the Debian vs Ubuntu photoprism images?

deekerman commented

Author

Owner

@lastzero commented on GitHub (Sep 18, 2022):

Let's say you have Docker installed and want to test Debian 12 "Bookworm", you can simply run this command to open a terminal:

docker run --rm -v ${PWD}:/heic -w /heic -ti debian:bookworm bash

This will mount the current working directory as /heic. Of course, you can also specify a full path instead of ${PWD}.

A complete list of available Ubuntu and Debian images can be found on Docker Hub:

Now install the packages you want to test via apt:

apt update
apt install -y libheif-examples exiftool

Finally, run the heif-convert command (-q 92 is optional and determines the JPEG quality/compression):

root@1ad9fb887a4f:/heic# heif-convert -q 92 IMG_8437.HEIC IMG_8437.HEIC.jpg
File contains 1 image
Written to IMG_8437.HEIC.jpg

To view the image metadata, run exiftool -n <filename> (-n displays the raw values without changes) and optionally use grep to filter the output:

root@1ad9fb887a4f:/heic# exiftool -n IMG_8437.HEIC.jpg | grep ation
File Modification Date/Time     : 2022:09:18 08:16:28+00:00
Orientation                     : 6
Exposure Compensation           : 0
Acceleration Vector             : -0.1612657755 -0.1648305952 -0.9718825824
Chromatic Adaptation            : 1.04788 0.02292 -0.0502 0.02959 0.99048 -0.01706 -0.00923 0.01508 0.75168
root@1ad9fb887a4f:/heic# exiftool -n IMG_8437.HEIC | grep ation
File Modification Date/Time     : 2022:09:17 16:57:40+00:00
Orientation                     : 6
Exposure Compensation           : 0
Acceleration Vector             : -0.1612657755 -0.1648305952 -0.9718825824
Chromatic Adaptation            : 1.04788 0.02292 -0.0502 0.02959 0.99048 -0.01706 -0.00923 0.01508 0.75168
HEVC Configuration Version      : 1
Min Spatial Segmentation IDC    : 0
Rotation                        : 270

Rotation and Orientation are the important values you should pay attention to and compare. The rotation is in degrees. The 8 Exif orientation values are numbered from 1 to 8:

= 0 degrees: the correct orientation, no adjustment is required.
= 0 degrees, mirrored: image has been flipped back-to-front.
= 180 degrees: image is upside down.
= 180 degrees, mirrored: image has been flipped back-to-front and is upside down.
= 90 degrees: image has been flipped back-to-front and is on its side.
= 90 degrees, mirrored: image is on its side.
= 270 degrees: image has been flipped back-to-front and is on its far side.
= 270 degrees, mirrored: image is on its far side.

@lastzero commented on GitHub (Sep 18, 2022): Let's say you have Docker installed and want to test Debian 12 "Bookworm", you can simply run this command to open a terminal: ```bash docker run --rm -v ${PWD}:/heic -w /heic -ti debian:bookworm bash ``` This will mount the current working directory as `/heic`. Of course, you can also specify a full path instead of `${PWD}`. A complete list of available Ubuntu and Debian images can be found on Docker Hub: - https://hub.docker.com/_/ubuntu - https://hub.docker.com/_/debian --- Now install the packages you want to test via `apt`: ```bash apt update apt install -y libheif-examples exiftool ``` Finally, run the `heif-convert` command (`-q 92` is optional and determines the JPEG quality/compression): ``` root@1ad9fb887a4f:/heic# heif-convert -q 92 IMG_8437.HEIC IMG_8437.HEIC.jpg File contains 1 image Written to IMG_8437.HEIC.jpg ``` To view the image metadata, run `exiftool -n <filename>` (`-n` displays the raw values without changes) and optionally use grep to filter the output: ``` root@1ad9fb887a4f:/heic# exiftool -n IMG_8437.HEIC.jpg | grep ation File Modification Date/Time : 2022:09:18 08:16:28+00:00 Orientation : 6 Exposure Compensation : 0 Acceleration Vector : -0.1612657755 -0.1648305952 -0.9718825824 Chromatic Adaptation : 1.04788 0.02292 -0.0502 0.02959 0.99048 -0.01706 -0.00923 0.01508 0.75168 root@1ad9fb887a4f:/heic# exiftool -n IMG_8437.HEIC | grep ation File Modification Date/Time : 2022:09:17 16:57:40+00:00 Orientation : 6 Exposure Compensation : 0 Acceleration Vector : -0.1612657755 -0.1648305952 -0.9718825824 Chromatic Adaptation : 1.04788 0.02292 -0.0502 0.02959 0.99048 -0.01706 -0.00923 0.01508 0.75168 HEVC Configuration Version : 1 Min Spatial Segmentation IDC : 0 Rotation : 270 ``` **Rotation** and **Orientation** are the important values you should pay attention to and compare. The rotation is in degrees. The 8 Exif orientation values are numbered from 1 to 8: 1. = 0 degrees: the correct orientation, no adjustment is required. 2. = 0 degrees, mirrored: image has been flipped back-to-front. 3. = 180 degrees: image is upside down. 4. = 180 degrees, mirrored: image has been flipped back-to-front and is upside down. 5. = 90 degrees: image has been flipped back-to-front and is on its side. 6. = 90 degrees, mirrored: image is on its side. 7. = 270 degrees: image has been flipped back-to-front and is on its far side. 8. = 270 degrees, mirrored: image is on its far side.

deekerman commented

Author

Owner

@lastzero commented on GitHub (Sep 18, 2022):

@melyux We have added information to our Developer Guide on how to install packages into Docker images and run commands there:

Hope this helps!

@lastzero commented on GitHub (Sep 18, 2022): @melyux We have added information to our [Developer Guide](https://docs.photoprism.app/developer-guide/) on how to install packages into Docker images and run commands there: - https://docs.photoprism.app/developer-guide/media/heif/#testing-conversion-and-orientation - https://docs.photoprism.app/developer-guide/metadata/orientation/ Hope this helps!

deekerman commented

Author

Owner

@joachimtingvold commented on GitHub (Sep 19, 2022):

I had this issue as well. Can confirm that updating to latest :preview (for me that was 220919-cc8bab446) fixes the issue. I'm running default image: photoprism/photoprism:preview, which seems to be running Ubuntu Jammy. It's running libheif 1.12.0 at least.

After the upgrade, I did a photoprism convert --force 2022/09 (just to speed it up, as only the last few days had issues), which seems to have produced proper sidecar files (where the orientation attributes in the EXIF data is correct). Still waiting for the thumbs to re-create, but so far so good.

Could we maybe get a way to limit photoprism thumbs to a certain directory? So we could do photoprism thumbs --force 2022/09 to limit it to the 2022/09 path?

@joachimtingvold commented on GitHub (Sep 19, 2022): I had this issue as well. Can confirm that updating to latest `:preview` (for me that was `220919-cc8bab446`) fixes the issue. I'm running default `image: photoprism/photoprism:preview`, which seems to be running Ubuntu Jammy. It's running `libheif 1.12.0` at least. After the upgrade, I did a `photoprism convert --force 2022/09` (just to speed it up, as only the last few days had issues), which seems to have produced proper sidecar files (where the orientation attributes in the EXIF data is correct). Still waiting for the thumbs to re-create, but so far so good. Could we maybe get a way to limit `photoprism thumbs` to a certain directory? So we could do `photoprism thumbs --force 2022/09` to limit it to the `2022/09` path?

deekerman commented

Author

Owner

@lastzero commented on GitHub (Sep 19, 2022):

Generally yes, that might be possible. It would help a lot if you could test heif-convert on Debian 11 and 12. That's because Ubuntu will eventually get the same version as Debian 12, or even newer. Meaning if there is a problem, you will have the same issue soon again.

As thumbs get generated based on hashes, I don't think the force option is needed which should accelerate the update significantly.

@lastzero commented on GitHub (Sep 19, 2022): Generally yes, that might be possible. It would help a lot if you could test `heif-convert` on Debian 11 and 12. That's because Ubuntu will eventually get the same version as Debian 12, or even newer. Meaning if there is a problem, you will have the same issue soon again. As thumbs get generated based on hashes, I don't think the force option is needed which should accelerate the update significantly.

deekerman commented

Author

Owner

@joachimtingvold commented on GitHub (Sep 20, 2022):

Converted sidecar images (from original HEIC) was re-created when using photoprism convert --force 2022/09 (confirmed by downloading the raw sidecar image). Followed up with a photoprism thumbs (without --force flag), and it did not create new thumbnails for the pictures that had their sidecar re-created. Hence the thumbnails (including the preview images) are still wrongly orientated.

@joachimtingvold commented on GitHub (Sep 20, 2022): Converted sidecar images (from original HEIC) was re-created when using `photoprism convert --force 2022/09` (confirmed by downloading the raw sidecar image). Followed up with a `photoprism thumbs` (without `--force` flag), and it did not create new thumbnails for the pictures that had their sidecar re-created. Hence the thumbnails (including the preview images) are still wrongly orientated.

deekerman commented

Author

Owner

@lastzero commented on GitHub (Sep 20, 2022):

Did you update the index in between? Without looking at the code, I can't tell you where the hashes come from but it seems entirely possible that these are retrieved from the index database and thus the new preview hashes won't get recognized without indexing.

@lastzero commented on GitHub (Sep 20, 2022): Did you update the index in between? Without looking at the code, I can't tell you where the hashes come from but it seems entirely possible that these are retrieved from the index database and thus the new preview hashes won't get recognized without indexing.

deekerman commented

Author

Owner

@joachimtingvold commented on GitHub (Oct 5, 2022):

@lastzero, I did a photoprism index --cleanup followed by photoprism thumbs. It did regenerate some thumbnails, but not the ones that are wrongly orientated. Any other suggestions?

@joachimtingvold commented on GitHub (Oct 5, 2022): @lastzero, I did a `photoprism index --cleanup` followed by `photoprism thumbs`. It did regenerate some thumbnails, but not the ones that are wrongly orientated. Any other suggestions?

deekerman commented

Author

Owner

@lastzero commented on GitHub (Oct 7, 2022):

@joachimtingvold The latest heif-convert version should finally fix all these issues:

#2726

It may take a while before a new stable version is available for you, as it also includes many other changes that need to be tested.

@lastzero commented on GitHub (Oct 7, 2022): @joachimtingvold The latest [heif-convert version](https://dl.photoprism.app/dist/libheif/) should finally fix all these issues: - #2726 It may take a while before a new stable version is available for you, as it also includes many other changes that need to be tested.

deekerman commented

Author

Owner

@joachimtingvold commented on GitHub (Nov 4, 2022):

@lastzero , excellent. Can confirm that 221027-6a33ae2e9 followed by photoprism index --cleanup + photoprism convert -f + photoprism thumbs -f seems to have fixed the issues.

@joachimtingvold commented on GitHub (Nov 4, 2022): @lastzero , excellent. Can confirm that `221027-6a33ae2e9` followed by `photoprism index --cleanup` + `photoprism convert -f` + `photoprism thumbs -f` seems to have fixed the issues.

deekerman commented

Author

Owner

@lastzero commented on GitHub (Nov 4, 2022):

@joachimtingvold As that looks like a preview version, you should upgrade to today's stable version after reading the full release notes for Build 221102-905925b4d 👍

@lastzero commented on GitHub (Nov 4, 2022): @joachimtingvold As that looks like a preview version, you should upgrade to today's stable version after reading the full release notes for Build 221102-905925b4d 👍

deekerman commented

Author

Owner

@martin-g-it commented on GitHub (Jan 7, 2023):

Hi, it appears this fix is not applied when running PhotoPrism without Docker (ie. as Proxmox LXC). Is that correct?
Running Build 230106-e58fee0fb

@martin-g-it commented on GitHub (Jan 7, 2023): Hi, it appears this fix is not applied when running PhotoPrism without Docker (ie. as Proxmox LXC). Is that correct? Running Build [230106-e58fee0fb](https://docs.photoprism.app/release-notes/)

deekerman commented

Author

Owner

@graciousgrey commented on GitHub (Jan 8, 2023):

@martin-g-it we do not offer LXC images for PhotoPrism yet and can't help much with third party installations. A potential issue could be that your installation uses a different version of the library that is used for heif conversion.

@graciousgrey commented on GitHub (Jan 8, 2023): @martin-g-it we do not offer LXC images for PhotoPrism yet and can't help much with third party installations. A potential issue could be that your installation uses a different version of the library that is used for heif conversion.

deekerman commented

Author

Owner

@martin-g-it commented on GitHub (Jan 9, 2023):

Hi @graciousgrey, it's not really a third party installation I build up PhotoPrism by;
git clone
https://github.com/photoprism/photoprism.git &>/dev/null
cd photoprism
git checkout release &>/dev/null
make all &>/dev/null
./scripts/build.sh prod /opt/photoprism/bin/photoprism &>/dev/null
cp -r assets/ /opt/photoprism/ &>/dev/null

I expect the conversion of heic > jpg being done by:
https://github.com/photoprism/photoprism/blob/develop/scripts/dist/heif-convert.sh

Or did I miss any other place where I should look?

@martin-g-it commented on GitHub (Jan 9, 2023): Hi @graciousgrey, it's not really a third party installation I build up PhotoPrism by; `git clone` `https://github.com/photoprism/photoprism.git &>/dev/null` `cd photoprism` `git checkout release &>/dev/null` `make all &>/dev/null` `./scripts/build.sh prod /opt/photoprism/bin/photoprism &>/dev/null` `cp -r assets/ /opt/photoprism/ &>/dev/null` I expect the conversion of heic > jpg being done by: https://github.com/photoprism/photoprism/blob/develop/scripts/dist/heif-convert.sh Or did I miss any other place where I should look?

deekerman commented

Author

Owner

@graciousgrey commented on GitHub (Jan 9, 2023):

@martin-g-it

it's not really a third party installation

Building from source results in custom builds.

Make sure you follow all steps in the Dockerfile and install the same build tools and dependencies, for example the exact same version of libheif, as those tiny little differences are what makes your build custom. It's not all in the source we provide.

@graciousgrey commented on GitHub (Jan 9, 2023): @martin-g-it >it's not really a third party installation Building from source results in custom builds. Make sure you follow all steps in the Dockerfile and install the same build tools and dependencies, for example the exact same version of libheif, as those tiny little differences are what makes your build custom. It's not all in the source we provide.

deekerman commented

Author

Owner

@martin-g-it commented on GitHub (Jan 10, 2023):

The libheif is all OK, it's just the orientation that is incorrect when .HEIC files were converted to .jpg.

I expect because the convert_jpeg.go converts using heif-convert without the exif correction (like done in the Docker script). github.com/photoprism/photoprism@727b3eb552/internal/photoprism/convert_jpeg.go line 171:
result = append(result, exec.Command(c.conf.HeifConvertBin(), "-q", c.conf.JpegQuality().String(), f.FileName(), jpegName))
I ended up making a wrapper around heif-convert (copying /usr/bin/heif-convert > /usr/bin/heif-convert-orig and set /usr/bin/heif-convert with below script):

#!/bin/bash
heif-convert-orig "$@"
exiftool -overwrite_original -P -n  -Orientation=1 "${@: -1}"

This resulted in .HEIC files being converted/imported correctly.. :-)

Small disadvantage though; while importing .HEIC files; the sidecar folder size is growing due to the converted .HEIC.jpg files.
Luckily you can use config option PHOTOPRISM_SIDECAR_PATH to place the sidecar files on eg. a mounted NAS location.

Another option (or workaround) is to convert the .HEIC files upfront before importing into PhotoPrism library;
Convert HEIC to jpg:
for i in *.HEIC; do heif-convert "$i" "${i%.*}.heic.jpg"; done
Set orientation correct:
for i in *.heic.jpg; do exiftool -overwrite_original -P -n '-ModifyDate<FileModifyDate' -Orientation=1 $i; done

@martin-g-it commented on GitHub (Jan 10, 2023): The libheif is all OK, it's just the orientation that is incorrect when .HEIC files were converted to .jpg. I expect because the convert_jpeg.go converts using heif-convert without the exif correction (like done in the Docker script). https://github.com/photoprism/photoprism/blob/727b3eb55255e79fdff64943648ced1f8f3fae42/internal/photoprism/convert_jpeg.go line 171: `result = append(result, exec.Command(c.conf.HeifConvertBin(), "-q", c.conf.JpegQuality().String(), f.FileName(), jpegName)) ` I ended up making a wrapper around heif-convert (copying /usr/bin/heif-convert > /usr/bin/heif-convert-orig and set /usr/bin/heif-convert with below script): ``` #!/bin/bash heif-convert-orig "$@" exiftool -overwrite_original -P -n -Orientation=1 "${@: -1}" ``` This resulted in .HEIC files being converted/imported correctly.. :-) Small disadvantage though; while importing .HEIC files; the sidecar folder size is growing due to the converted .HEIC.jpg files. Luckily you can use config option PHOTOPRISM_SIDECAR_PATH to place the sidecar files on eg. a mounted NAS location. Another option (or workaround) is to convert the .HEIC files upfront before importing into PhotoPrism library; Convert HEIC to jpg: `for i in *.HEIC; do heif-convert "$i" "${i%.*}.heic.jpg"; done ` Set orientation correct: `for i in *.heic.jpg; do exiftool -overwrite_original -P -n '-ModifyDate<FileModifyDate' -Orientation=1 $i; done `

deekerman commented

Author

Owner

@lastzero commented on GitHub (Jan 10, 2023):

What do you mean by "libheif is all OK"? Is it the same version we use or the one that e.g. comes with the Linux distribution you use? This is the correct version with bug fixes:

👉 https://dl.photoprism.app/dist/libheif/

@lastzero commented on GitHub (Jan 10, 2023): What do you mean by "libheif is all OK"? Is it the same version we use or the one that e.g. comes with the Linux distribution you use? This is the correct version with bug fixes: 👉 https://dl.photoprism.app/dist/libheif/

deekerman commented

Author

Owner

@martin-g-it commented on GitHub (Jan 11, 2023):

Is it the same version we use or the one that e.g. comes with the Linux distribution you use?

Maybe thats the thing.. I took the package installed via:
apt install -y libheif-examples exiftool

Running Debian GNU/Linux 11 (bullseye)

This one works fine converting .HEIC into .jpg. It's just the orientation that is wrong (which is how I ended up in this thread) and which can be corrected by the exiftool command.

I tried the version you've referred, but ran into libjpeg.so.8 is missing errors..

@martin-g-it commented on GitHub (Jan 11, 2023): > Is it the same version we use or the one that e.g. comes with the Linux distribution you use? Maybe thats the thing.. I took the package installed via: `apt install -y libheif-examples exiftool` Running Debian GNU/Linux 11 (bullseye) This one works fine converting .HEIC into .jpg. It's just the orientation that is wrong (which is how I ended up in this thread) and which can be corrected by the exiftool command. I tried the version you've referred, but ran into libjpeg.so.8 is missing errors..

deekerman commented

Author

Owner

@lastzero commented on GitHub (Jan 11, 2023):

It's the metadata and orientation that we've fixed (pulled from upstream and built, to be exact).

@lastzero commented on GitHub (Jan 11, 2023): It's the metadata and orientation that we've fixed (pulled from upstream and built, to be exact).

deekerman commented

Author

Owner

@lastzero commented on GitHub (Jan 11, 2023):

Also, Bullseye is outdated compared to Ubuntu Jammy plus Debian doesn't ship libjpeg, only libjpeg-turbo, so you'd have to build it yourself. That's the reason we use Docker - it's impossible to get it right otherwise.

@lastzero commented on GitHub (Jan 11, 2023): Also, Bullseye is outdated compared to Ubuntu Jammy plus Debian doesn't ship libjpeg, only libjpeg-turbo, so you'd have to build it yourself. That's the reason we use Docker - it's impossible to get it right otherwise.

deekerman commented

Author

Owner

@martin-g-it commented on GitHub (Jan 12, 2023):

Right, makes sense now... Thanks for the swift reply! Will try to upgrade to Bookworm or run a fresh install on that release to see whether this will resolve.

@martin-g-it commented on GitHub (Jan 12, 2023): Right, makes sense now... Thanks for the swift reply! Will try to upgrade to Bookworm or run a fresh install on that release to see whether this will resolve.

deekerman commented

Author

Owner

@lastzero commented on GitHub (Jan 12, 2023):

The version we are using, and that has proven to be bug free, is v1.13.0. As far as I remember, all versions that shipped with Debian had problems. Also, libjpeg-turbo, which ships with Debian, does not support ICC profiles in combination with libheif, so libheif had to add code to handle this. It is not known if this code works without any bugs and does the same as our solution.

@lastzero commented on GitHub (Jan 12, 2023): The version we are using, and that has proven to be bug free, is v1.13.0. As far as I remember, all versions that shipped with Debian had problems. Also, libjpeg-turbo, which ships with Debian, does not support ICC profiles in combination with libheif, so libheif had to add code to handle this. It is not known if this code works without any bugs and does the same as our solution.

deekerman commented

Author

Owner

@lapo-luchini commented on GitHub (Feb 9, 2023):

Just as an extra data point, running 221118-e58fee0 on FreeBSD (compiled using this port) and using system libheif-1.14.0_1 seems to still have the bug, even though it is marked as solved in libheif's 1.14.0 release log.

@lapo-luchini commented on GitHub (Feb 9, 2023): Just as an extra data point, running `221118-e58fee0` on FreeBSD (compiled [using this port](https://github.com/huo-ju/photoprism-freebsd-port)) and using system `libheif-1.14.0_1` seems to still have the bug, even though it is marked as solved in libheif's [1.14.0 release log](https://github.com/strukturag/libheif/releases/tag/v1.14.0).

deekerman commented

Author

Owner

@lapo-luchini commented on GitHub (Mar 28, 2023):

I meant to go deeper in this (I still have wrong rotation out of HEIF files) and I wonder if this is right:

# original HEIF
File Modification Date/Time     : 2023:03:24 21:12:52+01:00
Orientation                     : 0
Components Configuration        : 1 2 3 0
Exposure Compensation           : 0
HEVC Configuration Version      : 1
Min Spatial Segmentation IDC    : 0
Rotation                        : 270
# generated JPEG
File Modification Date/Time     : 2023:03:26 16:43:02+02:00
Orientation                     : 1
Components Configuration        : 1 2 3 0
Exposure Compensation           : 0

Orientation 0 + rotation 270 ⇒ orientation 1.
I'm not sure about these values, but it looks correct to me…?

Still… (using libheif-1.15.1)

@lapo-luchini commented on GitHub (Mar 28, 2023): I meant to go deeper in this (I still have wrong rotation out of HEIF files) and I wonder if this is right: ``` # original HEIF File Modification Date/Time : 2023:03:24 21:12:52+01:00 Orientation : 0 Components Configuration : 1 2 3 0 Exposure Compensation : 0 HEVC Configuration Version : 1 Min Spatial Segmentation IDC : 0 Rotation : 270 # generated JPEG File Modification Date/Time : 2023:03:26 16:43:02+02:00 Orientation : 1 Components Configuration : 1 2 3 0 Exposure Compensation : 0 ``` Orientation 0 + rotation 270 ⇒ orientation 1. I'm not sure about these values, but it looks correct to me…? Still… (using `libheif-1.15.1`) <img width="219" alt="image" src="https://user-images.githubusercontent.com/420454/228361483-949a4dc7-c2d3-4845-8bc5-8a225315f2c8.png">

deekerman commented

Author

Owner

@lastzero commented on GitHub (Mar 28, 2023):

No, that's the problem. Exif Orientation and QuickTime rotation add up, causing the wrong orientation in the end. You can either update libheif or use a wrapper script to fix this. Both can be found in our code base.

@lastzero commented on GitHub (Mar 28, 2023): No, that's the problem. Exif Orientation and QuickTime rotation add up, causing the wrong orientation in the end. You can either update libheif or use a wrapper script to fix this. Both can be found in our code base.

deekerman commented

Author

Owner

@lapo-luchini commented on GitHub (Mar 28, 2023):

But the photo shows like this, I think "rotation 270" is right; orientation 0 is unknown, assuming 1 as default adding 270 rotation I should get an 8 in output, right?

(for some reason my Xiaomi Mi 11T seems to only change quicktime rotation, not orientation in HEICs — I guess this makes my issue totally different from the rest of this issue)
I'm on FreeBSD, not Linux, so I prefer to stick to binary libheif as found in official packages, but that's not even the problem probably. I will try and work around it it with a wrapper, thanks!

@lapo-luchini commented on GitHub (Mar 28, 2023): But the photo shows like this, I think "rotation 270" is right; orientation 0 is unknown, assuming 1 as default adding 270 rotation I should get an 8 in output, right? <img width="200" alt="image" src="https://user-images.githubusercontent.com/420454/228377699-0f809a54-a9cf-40c3-ac09-e2409a14cd15.png"> (for some reason my Xiaomi Mi 11T seems to only change quicktime rotation, not orientation in HEICs — I guess this makes my issue totally different from the rest of this issue) I'm on FreeBSD, not Linux, so I prefer to stick to binary libheif as found in official packages, but that's not even the problem probably. I will try and work around it it with a wrapper, thanks!

deekerman commented

Author

Owner

@lastzero commented on GitHub (Mar 28, 2023):

Note that browsers show JPEG images and not native QuickTime or HEIC. So the Exif Orientation field should be used. JPEG readers typically don't know much about video metadata.

@lastzero commented on GitHub (Mar 28, 2023): Note that browsers show JPEG images and not native QuickTime or HEIC. So the Exif Orientation field should be used. JPEG readers typically don't know much about video metadata.

deekerman commented

Author

Owner

@lastzero commented on GitHub (Mar 28, 2023):

FreeBSD ports should be more up-to-date than most Linux distributions. The libheif version that finally worked for us was released last year, although newer versions could theoretically have a regression. Unfortunately, we have very little time to focus on this given all the other work.

@lastzero commented on GitHub (Mar 28, 2023): FreeBSD ports should be more up-to-date than most Linux distributions. The libheif version that finally worked for us was released last year, although newer versions could theoretically have a regression. Unfortunately, we have very little time to focus on this given all the other work.

deekerman commented

Author

Owner

@lapo-luchini commented on GitHub (Mar 29, 2023):

Just in case anyone with Xiaomi phone is interested: this is the produced metadata with the 4 possible orientation of the phone:

% exiftool -m -n -p '$filename $imagewidth $imageheight Orientation: $orientation Rotation: $rotation' *.HEIC
IMG_20230329_090439.HEIC 3000 4000 Orientation: 0 Rotation: 270
IMG_20230329_090443.HEIC 4000 3000 Orientation: 1 Rotation:
IMG_20230329_090446.HEIC 3000 4000 Orientation: 0 Rotation: 90
IMG_20230329_090450.HEIC 4000 3000 Orientation: 0 Rotation: 180

Image is physically rotated, the already applied rotation is shown in QuickTime data (only for HEIC, for JPG it is not done), and EXIT orientation is either 1 (normal) or 0 (undefined) for all 3 other rotations.
I think the best approach is to overwrite all the metadata madness like this (as the image is pre-rotated, it shows ok):

exiftool -r -if '$model eq "21081111RG"' -exif:orientation#=1 -quicktime:rotation=0 .

My only regret is that this way the "original rotation" is lost, but I can't keep that unless I de-rotate the actual image too, and I don't want to touch the image itself, only metadata. I'll maybe add it as a custom EXIF meta, just for the sake of it.

@lapo-luchini commented on GitHub (Mar 29, 2023): Just in case anyone with Xiaomi phone is interested: this is the produced metadata with the 4 possible orientation of the phone: ``` % exiftool -m -n -p '$filename $imagewidth $imageheight Orientation: $orientation Rotation: $rotation' *.HEIC IMG_20230329_090439.HEIC 3000 4000 Orientation: 0 Rotation: 270 IMG_20230329_090443.HEIC 4000 3000 Orientation: 1 Rotation: IMG_20230329_090446.HEIC 3000 4000 Orientation: 0 Rotation: 90 IMG_20230329_090450.HEIC 4000 3000 Orientation: 0 Rotation: 180 ``` Image is physically rotated, the already applied rotation is shown in QuickTime data (only for HEIC, for JPG it is not done), and EXIT orientation is either 1 (normal) or 0 (undefined) for all 3 other rotations. I think the best approach is to overwrite all the metadata madness like this (as the image is pre-rotated, it shows ok): ``` exiftool -r -if '$model eq "21081111RG"' -exif:orientation#=1 -quicktime:rotation=0 . ``` My only regret is that this way the "original rotation" is lost, but I can't keep that unless I de-rotate the actual image too, and I don't want to touch the image itself, only metadata. I'll maybe add it as a custom EXIF meta, just for the sake of it.

deekerman commented