machine learning: hardware acceleration on apple silicon (metal)

[traktuner]

Hello,
just wanted to ask if it's planned to support ML hardware acceleration on apple silicon gpu's (metal).
This would result in a good performance for the initial ML scan on macs with "max" and/or "ultra" CPUs.
Thank you!

[radh21301]

This would be awesome! Even people that have macbook air or pro as a daily driver laptop would benefit if they had immich running on low capability server. Say a pi or something.

Maybe convert this into a feature request instead of 'Q&A'

[Qhilm]

just to clarify: a MacBook is already insanely faster than a Pi, even without the hardware acceleration. My MacBook is 30-50 times faster than my Synology NAS, even though there is no hardware acceleration.

I still would love to have this feature though, because then I can rerun machine learning on my entire library with a different model in a matter of hours (or maybe even faster?) instead of taking two days 🙂

[radh21301]

updated my comment. I meant - many users have a macbook air or pro as a daily driver laptop. but use a pi as a server for immich. They would definitely befefit like you say.

[bo0tzz]

To my understanding, docker on mac runs inside of a Linux VM and it's not possible to pass the Apple Metal hwaccel through to that.

[xobust]

Seems to be possible using GPU virtio-gpu.
See this post using podman and llama https://sinrega.org/2024-03-06-enabling-containers-gpu-macos/
I wonder if a similar technique can be used with Immich

[tdonovic]

Is virtio-gpu not an intel only thing?
This would require the underlying models to support the apple gpu bindings too no?
When I check on openclip it doesn't look like native mac support is there yet openai/CLIP#255

Establishing that would be the first step to whether its even worth trying to expose the gpu through docker.
Then I read something like https://chariotsolutions.com/blog/post/apple-silicon-gpus-docker-and-ollama-pick-two/, (only 3 months old) which says its not possible to expose the gpu and suggests running it locally.

I get the impression there is a lot of dev required elsewhere if its even possible before its worth spending time on the immich end.

[mertalev]

You would have to run the machine learning service directly outside of Docker. The instructions to do this are as follows:

1. Clone the immich repo
2. Navigate to the machine-learning folder
3. Add CoreMLExecutionProvider to the start of this list
4. Install Poetry
5. Install dependencies with poetry install --with cpu
6. Set the cache folder with export MACHINE_LEARNING_CACHE_FOLDER=~/.model-cache (you can set a different path here if you like)
7. Create the model cache folder with mkdir $MACHINE_LEARNING_CACHE_FOLDER
8. Run ./start.sh

It should then be listening on 3003.

Keep in mind that none of the models have been tested for CoreML compatibility, so you might need to experiment with different models to find one that works well.

[pippo73]

Thank you @xobust.
It was a problem of environment.
Here is how I fix it (hope It could help someone else)

$ brew update
$ brew upgrade
$ pyenv install 3.11.6
$ pyenv global 3.11.6
$ poetry env use 3.11

< follower the steps on the comment 10633589 >
$ poetry install --with cpu
$ export MACHINE_LEARNING_CACHE_FOLDER=~/.model-cache 
$ mkdir $MACHINE_LEARNING_CACHE_FOLDER
$  ./start.sh
[01/28/25 09:33:07] INFO     Starting gunicorn 23.0.0
[01/28/25 09:33:07] INFO     Listening at: http://[::]:3003 (251)
[01/28/25 09:33:07] INFO     Using worker: app.config.CustomUvicornWorker
[01/28/25 09:33:07] INFO     Booting worker with pid: 278



Fontconfig warning: ignoring UTF-8: not a valid region tag
[01/28/25 09:33:37] INFO     Started server process [278]
[01/28/25 09:33:37] INFO     Waiting for application startup.
[01/28/25 09:33:37] INFO     Created in-memory cache with unloading after 300s of inactivity.
[01/28/25 09:33:37] INFO     Initialized request thread pool with 10 threads.
[01/28/25 09:33:37] INFO     Application startup complete.
[01/28/25 09:33:53] INFO     Handling signal: winch
[01/28/25 09:45:28] INFO     Handling signal: winch

Now it seems working and listening a the port 3003 of my server.
Now I would like to connect my other server to this one.

In the gui of the second server I've added the ip of this server but is seems not talking each other

I've launched the jobs no the server, but it seems not talking to this one :-(
Do I miss something?

[pyronaur]

@pippo73 Is the URL correct - immich-machine-learning:3003? If your immich server itself is running in docker - Docker by default isn't able to "connect to your machine" (unless you've setup a network configuration for immich-machine-learning hostname) because localhost is itself (the docker image). I'm also not sure if it's able to find 192.168.1.60:3003 over the "network", depending on permissions.

I'm going to give something similar a try next week, but essentially, I think you need to point immich to http://host.docker.internal:3003 - that should let the container "break out of the container" and access the port 3003 on your machine.

[vjeeva]

Hi @mertalev thank you so much for providing these instructions! It seems the instructions above have gotten out of date. Do you have new instructions given recent refactoring of the codebase?

EG - start.sh is not in the machine-learning folder, it's under server/ but that shell script also requires Node to run.

[kerhbal]

Thank you @xobust. It was a problem of environment. Here is how I fix it (hope It could help someone else)

$ brew update
$ brew upgrade
$ pyenv install 3.11.6
$ pyenv global 3.11.6
$ poetry env use 3.11

< follower the steps on the comment 10633589 >
$ poetry install --with cpu
$ export MACHINE_LEARNING_CACHE_FOLDER=~/.model-cache 
$ mkdir $MACHINE_LEARNING_CACHE_FOLDER
$  ./start.sh
[01/28/25 09:33:07] INFO     Starting gunicorn 23.0.0
[01/28/25 09:33:07] INFO     Listening at: http://[::]:3003 (251)
[01/28/25 09:33:07] INFO     Using worker: app.config.CustomUvicornWorker
[01/28/25 09:33:07] INFO     Booting worker with pid: 278



Fontconfig warning: ignoring UTF-8: not a valid region tag
[01/28/25 09:33:37] INFO     Started server process [278]
[01/28/25 09:33:37] INFO     Waiting for application startup.
[01/28/25 09:33:37] INFO     Created in-memory cache with unloading after 300s of inactivity.
[01/28/25 09:33:37] INFO     Initialized request thread pool with 10 threads.
[01/28/25 09:33:37] INFO     Application startup complete.
[01/28/25 09:33:53] INFO     Handling signal: winch
[01/28/25 09:45:28] INFO     Handling signal: winch

Now it seems working and listening a the port 3003 of my server. Now I would like to connect my other server to this one.

In the gui of the second server I've added the ip of this server but is seems not talking each other

I've launched the jobs no the server, but it seems not talking to this one :-( Do I miss something?

@pippo73 hi not sure you solved it. in order to force immich to talk to your ML server, you need to change the order or just replace the url to leave only one ML server in list(for testing purpose) https://immich.app/docs/guides/remote-machine-learning#forcing-remote-processing

[pippo73]

@pippo73 hi not sure you solved it. in order to force immich to talk to your ML server, you need to change the order or just replace the url to leave only one ML server in list(for testing purpose) https://immich.app/docs/guides/remote-machine-learning#forcing-remote-processing

Thanks , I've follower the guide of @HeroSizy and everythig works fine. :-) but thank you anyway ;-)

[pippo73]

I all,
I'm planing to buy a mac mini m4, and I was looking about infos if I can squeeze it somehow.
Does anybody of you heard about news of it?
Update :

if it could help to do some tests I can use also a mac mini m4 (not pro) to see how it works :)

[pyronaur]

Same here, just ordered a mac mini m4 to replace my Unraid NAS because immich was consistently bricking my Unraid with 100% CPU usage on an 8-core Ryzen (no graphics card), seems like adding HW acceleration is going to be an uphill battle here too, but it's something I think I'll have to figure out one way or another.

[scrapix]

Hello immich community, I've tried this setup as follows but getting an error. Can anyone help?
Also I've put the various instructions together to start creating a comprehensive step-by-step approach. Possibly we can complete it together and add it to the docs for future users?

# Clone immich repo
git clone --depth 1 https://github.com/immich-app/immich.git

# Add CoreMLExecutionProvider to the start of constants.py
nano "~/immich/machine-learning/app/models/constants.py"

# from app.config import clean_name
# from app.schemas import ModelSource
#
# ...
# SUPPORTED_PROVIDERS = ["CoreMLExecutionProvider",  "CUDAExecutionProvider", "OpenVINOExecutionProvider", "CPUExecutionProvider"]
# ....
# }

# set up or install prerequisites
# homebrew needs to be installed https://brew.sh/
brew update
brew upgrade
brew reinstall gfortran
brew install scipy
brew install pyenv
pyenv install 3.11.6
pyenv global 3.11.6

# install pipx to install poetry
brew install pipx

#add pipx to path
pipx ensurepath

###  open new terminal session ###

# install prerequisites
pipx install gunicorn

# Install Poetry
pipx install poetry

# tell pipx to use specific python version
poetry env use 3.11

# cd into immich/machine-learning
cd ~/immich/machine-learning

# Install dependencies with poetry
poetry install --with cpu

#Set the cache folder
export MACHINE_LEARNING_CACHE_FOLDER=~/.model-cache

#Create the model cache folder 
mkdir $MACHINE_LEARNING_CACHE_FOLDER

Run machine learning locally
./start.sh

Error I receive:

Initializing Immich ML
Error: Unable to configure handler 'console'

[alevys25]

@scrapix I was also running into this error. Make sure you do source $(poetry env info --path)/bin/activate before you run ./start.sh since the system does not know to use the poetry environment with your script, and will default to your general Python environment instead.
Hope this works!

[fredrike]

This enabled me to start ml locally.

brew update
brew upgrade
softwareupdate --install-rosetta --agree-to-license
brew reinstall gcc gfortran scipy pyenv uv
git clone git@github.com:immich-app/immich.git
cd immich/machine-learning
uv sync --extra cpu
source .venv/bin/activate
python -m immich_ml

Next time the run command is:

cd immich/machine-learning
source .venv/bin/activate
python -m immich_ml

[pippo73]

Thank you for the great job

Immich ML Auto-Start Setup

This guide explains how to automatically run the start_immich_ml.sh script for Immich machine learning tasks on macOS, both at login and on a daily schedule, using a Launch Agent.

1. Prerequisites

• macOS system
• Immich project with start_immich_ml.sh in the Immich directory
• Script is executable (chmod +x start_immich_ml.sh)

2. Script Overview

The script:

• Changes to the Immich directory
• Pulls the latest code
• Syncs the Python environment
• Activates the virtual environment
• Starts the Immich ML process
• Uses a lock file to prevent multiple instances

3. The Shell Script

Below is the content of start_immich_ml.sh:

#!/bin/zsh
LOCKFILE="/tmp/immich_ml.lock"
if [ -e "$LOCKFILE" ] && kill -0 "$(cat $LOCKFILE)" 2>/dev/null; then
    echo "immich_ml is already running"
    exit 1
fi
echo $$ > "$LOCKFILE"
trap "rm -f $LOCKFILE" EXIT

cd /path/to/ImmichRepo/machine-learning
git pull
uv sync --extra cpu --python=3.12
source .venv/bin/activate
python -m immich_ml

4. Setting Up the Launch Agent

4.1. Create the Launch Agent plist

Place the following in ~/Library/LaunchAgents/com.user.immichml.daily.plist (update the script path if needed):

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.user.immichml.daily</string>
    <key>ProgramArguments</key>
    <array>
        <string>/path/to/Immich/start_immich_ml.sh</string>
    </array>
    <key>StartCalendarInterval</key>
    <dict>
        <key>Hour</key>
        <integer>3</integer>
        <key>Minute</key>
        <integer>0</integer>
    </dict>
    <key>RunAtLoad</key>
    <true/>
    <key>StandardOutPath</key>
    <string>/tmp/immichml.log</string>
    <key>StandardErrorPath</key>
    <string>/tmp/immichml.err</string>
</dict>
</plist>

• StartCalendarInterval: Runs daily at 3:00 AM
• RunAtLoad: Runs at user login

4.2. Load the Launch Agent

launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.user.immichml.daily.plist

5. Manual Run

You can manually trigger the Launch Agent at any time:

launchctl kickstart -k gui/$(id -u)/com.user.immichml.daily

6. Troubleshooting

• Script not running?

  • Check the script path in the plist matches the actual location.
  • Ensure the script is executable: chmod +x /path/to/Immich/start_immich_ml.sh
  • Check logs: cat /tmp/immichml.err and cat /tmp/immichml.log
  • Validate the plist: plutil ~/Library/LaunchAgents/com.user.immichml.daily.plist

• Multiple instances?

  • The script uses a lock file at /tmp/immich_ml.lock to prevent this.

• Want to trigger manually as Launch Agent?

  • launchctl kickstart -k gui/$(id -u)/com.user.immichml.daily

7. Unloading the Launch Agent

To stop automatic runs:

launchctl bootout gui/$(id -u) ~/Library/LaunchAgents/com.user.immichml.daily.plist

Edit this README if your paths or requirements change.

Thank you for your great job!

[zd-project]

Please correct me if I'm wrong but the uv sync --extra cpu would only direct processing to CPUs right? In this situation what advantage do we get from running it natively in macos as opposed to in docker container? Docker container would be easy to set up with only minimal overhead anyway.

[mertalev]

Please correct me if I'm wrong but the uv sync --extra cpu would only direct processing to CPUs right? In this situation what advantage do we get from running it natively in macos as opposed to in docker container? Docker container would be easy to set up with only minimal overhead anyway.

No, the CPU build of ONNX Runtime supports CoreML as well. But CoreML only works outside of Docker.

[Xiseous]

This enabled me to start ml locally.

brew update
brew upgrade
softwareupdate --install-rosetta --agree-to-license
brew reinstall gcc gfortran scipy pyenv uv
git clone git@github.com:immich-app/immich.git
cd immich/machine-learning
uv sync --extra cpu
source .venv/bin/activate
python -m immich_ml

Next time the run command is:

cd immich/machine-learning
source .venv/bin/activate
python -m immich_ml

Is it normal to have 8 warnings and 14 errors when uv sync --extra cpu the script or should it be warnings only, no errors?

Never mind, I followed,
#10454 (comment)

up to

> #tell pipx to use specific python version
> poetry env use 3.11

and then reinstalled gcc, uv and the rest of this guide, then it worked

[nksoori]

Hi, when I try to do this, I get the error below. I'm on a M1 Macbook Air.

onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running 13422875215945187172_CoreML_13422875215945187172_32 node.
Name:'CoreMLExecutionProvider_13422875215945187172_CoreML_13422875215945187172_32_32' Status Message: Error executing model: Unable to compute the prediction using a neural network model. It can be an invalid
input data or broken/unsupported model (error code: -1).

[kerhbal]

@HeroSizy hi thanks for the script. though a dum question: why do we want/need to run it daily?
isn't it a ML server that listens to immich passively, and immich will decide when to call it (based on jobs)?

[pippo73]

The script runs the update of the server (it redownloads from the git server the source code and recompiles it daily)

[kerhbal]

@pippo73 oh I see thanks

[fivestones]

Say I have immich running on a linux server, and set up this machine learning on my mac. In immich settings, for machine learning, I have the mac url listed first, and then the linux server url. How can I tell that it is working? Is there a log somewhere that will tell me?

Thanks @fredrike and @HeroSizy for helping me get it set up!

[scrapix]

@fivestones - after starting your OSX ML instance (in terminal), you can monitor the logs. When you'll start ML tasks, you'll see if it's working properly:

[Xiseous]

Is there a list of supported models?

I've switched from the default to https://huggingface.co/immich-app/ViT-SO400M-16-SigLIP2-384__webli and now I seem to be getting traceback errors?

Edit: Errors on an M1 Mac due to

CoreML does not support input dim > 16384. Input:text.token_embedding.weight, shape: {256000,1152}

So I'm hitting the CoreMLExecutionProvider dimension cap of 16,384, which is a hardware/software limitation of CoreML,

I've switched this back to CPU and I'm getting fast, good performance so idk if there was any point of settings all this up just to do it via CPU? At least it's out of the container for better performance on resources.

Would be great to have another discussion open on here for the best models supported by CoreML

[Qhilm]

Are you using an M1? You are not getting the above-mentioned error?

My point was not that Apple GPU is not faster than Apple CPU (of course it is), my point was, I don't need additional speed in my specific setup, because the NAS cannot generate thumbnails fast enough to put my CPU at 100%. So leveraging the GPU wouldn't make anything faster, it would be idling, waiting for the NAS to generate thumbnails.

[fredrike]

My point was not that Apple GPU is not faster than Apple CPU (of course it is), my point was, I don't need additional speed in my specific setup, because the NAS cannot generate thumbnails fast enough to put my CPU at 100%. So leveraging the GPU wouldn't make anything faster, it would be idling, waiting for the NAS to generate thumbnails.

But then it comes to energy efficiency, perhaps it's better to let the GPU work and then idle..

[Qhilm]

Energy saving is a valid point, I agree, but I am not importing thousands of pictures, so the difference is minimal.

I got lucky though and could get my hands on a second hand M3 Max, I'll try again and see if it works now.

[Xiseous]

Are you using an M1? You are not getting the above-mentioned error?

No, because I switched to a model that requires less nodes, ViT-SO400M-16-SigLIP2-384__webli was throwing the errors due to being too big to fit on the MLNodes due to a MacOS restriction, so I went down the list from best to worst and ended up on number 3 or 4 at the time, which produced results within .5% of each other, at the cost that it almost perfectly fits on the MLNodes,

[09/22/25 01:13:55] INFO Loading visual model 'ViT-B-16-SigLIP-512__webli' to memory
[09/22/25 01:13:55] INFO Setting execution providers to ['CoreMLExecutionProvider', 'CPUExecutionProvider'], in descending order of preference
2025-09-22 01:13:55.377277 [W:onnxruntime:, coreml_execution_provider.cc:113 GetCapability] CoreMLExecutionProvider::GetCapability, number of partitions supported by CoreML: 68 number of nodes in the graph: 473 number of nodes supported by CoreML: 405

of course, CoreML node requirement depends on the complexity of the photo itself,

My point was not that Apple GPU is not faster than Apple CPU (of course it is), my point was, I don't need additional speed in my specific setup, because the NAS cannot generate thumbnails fast enough to put my CPU at 100%. So leveraging the GPU wouldn't make anything faster, it would be idling, waiting for the NAS to generate thumbnails.

Yeah, I get that, but everyone's use case is different, when I end up running a backup, I can have 1,000+ photos syncing, so keeping overall system usage in check on top of all the other services the M1 Mini runs is useful

[Xiseous]

But then it comes to energy efficiency, perhaps it's better to let the GPU work and then idle..

Not sure about wattage, the Mac Mini is already an efficient machine, but I know that the overall temp does not increase as much compared to the CPU/GPU recognition in my scenario

[GovSat1]

Has anyone benchmarked the performance in this setup vs docker cpu-only?

In some brief testing on a base M1 using this native setup, I'm seeing around 45% CPU usage (not much GPU, although I believe I've read in the past that GPU usage on apple silicon is not accurately reflected).

Using ViT-L-16-SigLIP-256__webli model, it is processing basically 1 asset per second. Definitely better than running on a NAS but I expected a little faster..

Edit: that was with the default concurrency of 2. On the M1 it seems like 6 about maxes out the CPU

I went ahead and gave docker a try as well for comparison. Very unscientific, take these with a grain of salt. Definitely possible I missed a step above to where hw accel is not kicking in.

Results:

• Using ViT-L-16-SigLIP-256__webli model, scanning a new library
• MBA M1 16gb, docker on orbstack
• Concurrency of 6 results in about 90% cpu on both
• Did 5 runs of 1 minute each, per method
• Bare metal: 72 assets/min avg
• Docker: 84 assets/min avg
• Fairly consistent results between runs

Not sure why docker would be faster, but either way I think I will go that route for simplicity sake. Speed should certainly be sufficient for most. MBA is likely throttling due to heat also, M4 mini should do better in that regard

[Xiseous]

I would say my M1 Mini performance is the same for CPU only,

Heavy usage of all available performance cores during ML maybe 2 photos processed per 15-25 seconds as per the updates on the Admin Page (this is with ML running on host as CPU rather than in docker, I imagine the performance would be similar in docker as a container with some overhead with the ViT-SO400M-16-SigLIP2-384__webli model).

I have briefly used the ViT-B-16-SigLIP-512__webli model on CPU and the time cut was about 2 assets per 5-10 seconds.

As for going back to MLCore on Host, currently utilising ViT-B-16-SigLIP-512__webli which has some fall back on the CPU in more complex images and still getting about 2 photos a second processed with this method (As per my testing ViT-B-16-SigLIP-512__webli seems to be the top end sweet spot for accuracy without too many photos relying on the CPU fallback with the model just slightly exceeding the available CoreML nodes)

The overall CPU usage seems to also be reported as lower by Activity Monitor during ML tasks.

[Qhilm]

So I now have an M3 instead of an M1 so I decided to give it a go again. After cloning the repository I did:

uv sync --python=$(which python3.12) --extra cpu
source .venv/bin/activate
python -m immich_ml

I had the error "[::]' does not appear to be an IPv4 or IPv6 address" but editing machine-learning/immich_ml/config.py, removing the brackets on the line immich_host: str = "[::]", solved this issue.

Now it processes pictures, but it uses the CPU, not the GPU.

Did I miss a step somewhere?

[Qhilm]

That model runs fine (albeit slowly) on my NAS with an Intel Celeron J3455 quad-core 1.5GHz.

I seriously doubt an M3 Max cannot handle the model. That error is misleading, something else is wrong.

The fact the "number of nodes supported by CoreML" is varying is also an indicator that something is wrong, how could this number be sometime 129 and sometimes 1010, that does make sense, it should be static if it's tied to the hardware.

[Xiseous]

That model runs fine (albeit slowly) on my NAS with an Intel Celeron J3455 quad-core 1.5GHz.

I seriously doubt an M3 Max cannot handle the model. That error is misleading, something else is wrong.

The fact the "number of nodes supported by CoreML" is varying is also an indicator that something is wrong, how could this number be sometime 129 and sometimes 1010, that does make sense, it should be static if it's tied to the hardware.

Probably because the variation in supported nodes and partitions is due to ONNX Runtime’s dynamic graph processing and decision-making on which nodes to assign to CoreML or CPU execution providers, rather than differences in the “model”.

Assign it all to CPU and let it work without the MLCores

[Qhilm]

What's the point then, if I use CPU? I can just run remote machine learning the "standard" way, as documented by Immich docs, using docker. The whole point here was to leverage the GPU.

[flsabourin]

I run into the same issue
CoreML does not support input dim > 16384. Input:681, shape: {-1,25088}
when running any of the compatible detection model (buffalo l, m, s, antelopev2)

on my M4 mini

[Xiseous]

What's the point then, if I use CPU? I can just run remote machine learning the "standard" way, as documented by Immich docs, using docker. The whole point here was to leverage the GPU.

Efficiency and speed, CPU will do whatever you task it to

[flsabourin]

Hello

The ML app starts correctly but I am getting this error on my M4 when I try to run face recognition task.
Any clue?

Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running 11938055035134306363_CoreML_11938055035134306363_0 node. Name:'CoreMLExecutionProvider_11938055035134306363_CoreML_11938055035134306363_0_0' Status Message: Error executing model: Unable to compute the prediction using a neural network model. It can be an invalid input data or broken/unsupported model (error code: -1). 2025-10-18 23:16:00.643210 [E:onnxruntime:, sequential_executor.cc:572 ExecuteKernel] Non-zero status code returned while running 11938055035134306363_CoreML_11938055035134306363_0 node. Name:'CoreMLExecutionProvider_11938055035134306363_CoreML_11938055035134306363_0_0' Status Message: Error executing model: Unable to compute the prediction using a neural network model. It can be an invalid input data or broken/unsupported model (error code: -1). [10/18/25 23:16:00] ERROR Exception in ASGI application

I blamed the model cache, started with a fresh one, no luck.
I tried with other models on the https://huggingface.co/immich-app repo, no luck

[MXMLN-sec]

Delete the constants.py file which you modified, then pull the git again (this should repull the file you just deleted), run the sync command and launch the ML instance (CoreML is now included in the base file)

Hi,
thank you for the support but I have not modified the constants.py.

[Xiseous]

With the latest version of the codebase, I'm having a new issue when immich hits immich_ml, after loading correctly the models (I've started from an empty cache)

2025-10-28 21:44:49.247 python[45642:11720851] Error getting visible function: (null) Compiler encountered XPC_ERROR_CONNECTION_INVALID (is the OS shutting down?) /AppleInternal/Library/BuildRoots/4~B5E4ugD7ttWrICQL0cKin8ziE5extnGgu--yePo/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShaders/MPSCore/Utility/MPSKernelDAG.mm:1378: failed assertion Error getting visible function: (null) Compiler encountered XPC_ERROR_CONNECTION_INVALID (is the OS shutting down?)'

Am I missing any dependency?

I'm getting the same issue,

2025-10-31 16:22:03.097 python[7213:83133] Error getting visible function:
(null) Compiler encountered XPC_ERROR_CONNECTION_INVALID (is the OS shutting down?)
/AppleInternal/Library/BuildRoots/d187755d-b9a3-11ef-83e5-aabfac210453/Library/Caches/com.apple.xbs/Sources/>MetalPerformanceShaders/MPSCore/Utility/MPSKernelDAG.mm:552: failed assertion `Error getting visible function:
(null) Compiler encountered XPC_ERROR_CONNECTION_INVALID (is the OS shutting down?)'

I have a feeling the instance has been updated to the a newer version MacOS 26 or something broke in the latest push, could be maybe related? pytorch/pytorch#114532

[Xiseous]

Delete the constants.py file which you modified, then pull the git again (this should repull the file you just deleted), run the sync command and launch the ML instance (CoreML is now included in the base file)

Hi, thank you for the support but I have not modified the constants.py.

Machine Specs (Model, RAM, CPU/Apple Silicon Chip)?
MacOS version?

[MXMLN-sec]

Delete the constants.py file which you modified, then pull the git again (this should repull the file you just deleted), run the sync command and launch the ML instance (CoreML is now included in the base file)

Hi, thank you for the support but I have not modified the constants.py.

Machine Specs (Model, RAM, CPU/Apple Silicon Chip)? MacOS version?

MacBook Air M4
10 Core CPU, 10‑Core GPU
16G RAM
512G SSD
macOS 26.0.1

I just pulled 20 Minutes ago with no change in the error.

[flsabourin]

I tried to run a test on the facial recognition model running the python script below.
Using the CPUExecutionProvider -> it works
Using the CoreMLExecutionProvider -> it fails
Now I still don't know how to solve it

import onnxruntime as ort
import numpy as np

print(ort.get_available_providers())

model_path = "~/.cache/immich_ml/facial-recognition/buffalo_l/detection/model.onnx"  # adjust

sess_cpu = ort.InferenceSession(
    model_path,
    providers=["CoreMLExecutionProvider"],
)

print("Inputs:")
for inp in sess_cpu.get_inputs():
    print(f" - {inp.name}: shape={inp.shape}, type={inp.type}")

# Build dummy inputs matching shapes
inputs = {}
for inp in sess_cpu.get_inputs():
    shape = [d if isinstance(d, int) else 1 for d in inp.shape]
    inputs[inp.name] = np.zeros(shape, dtype=np.float32)

# Try run
out = sess_cpu.run(None, inputs)
print("run OK, outputs:", len(out))

[delgurth]

I'm also having problems running the machine learning software on my M3 Mac. I'm still on macOs 15.6.1, and with python 3.12 I'm getting the following exception (after changing the loglevel to verbose, which didn't help me much unfortunately). Can someone tell me what I'm doing wrong?

❯ git pull
Already up to date.
❯ python -V
Python 3.12.12
❯ uv sync --extra cpu --python=3.12
Resolved 149 packages in 2ms
Bytecode compiled 9709 files in 339ms
❯ source .venv/bin/activate
❯ MACHINE_LEARNING_CACHE_FOLDER=~/.model-cache IMMICH_HOST=0.0.0.0 python -m immich_ml
[12/01/25 10:14:10] INFO     Starting gunicorn 23.0.0
[12/01/25 10:14:10] INFO     Listening at: http://0.0.0.0:3003 (81369)
[12/01/25 10:14:10] INFO     Using worker: immich_ml.config.CustomUvicornWorker
[12/01/25 10:14:10] INFO     Booting worker with pid: 81370
[12/01/25 10:14:10] DEBUG    Could not load ANN shared libraries, using ONNX:
                             dlopen(libmali.so, 0x0006): tried: 'libmali.so' (no
                             such file),
                             '/System/Volumes/Preboot/Cryptexes/OSlibmali.so'
                             (no such file),
                             '/Users/delgurth/.pyenv/versions/3.12.12/lib/libmali.
                             so' (no such file),
                             '/System/Volumes/Preboot/Cryptexes/OS/Users/delgurth/
                             .pyenv/versions/3.12.12/lib/libmali.so' (no such
                             file), '/opt/homebrew/lib/libmali.so' (no such
                             file),
                             '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/
                             lib/libmali.so' (no such file),
                             '/Users/delgurth/.pyenv/versions/3.12.12/lib/libmali.
                             so' (no such file),
                             '/System/Volumes/Preboot/Cryptexes/OS/Users/delgurth/
                             .pyenv/versions/3.12.12/lib/libmali.so' (no such
                             file), '/opt/homebrew/lib/libmali.so' (no such
                             file),
                             '/System/Volumes/Preboot/Cryptexes/OS/opt/homebrew/
                             lib/libmali.so' (no such file),
                             '/usr/lib/libmali.so' (no such file, not in dyld
                             cache), 'libmali.so' (no such file)
[12/01/25 10:14:10] DEBUG    RKNN is not available
objc[81370]: +[NSMutableString initialize] may have been in progress in another thread when fork() was called.
objc[81370]: +[NSMutableString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.

[fabriciorby]

I used export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES to bypass this error, but then couldn't escape the following ones lol

[DonSrinath]

If we can change immich in a way that it can use docker model runner it will be more easy.
(https://www.docker.com/blog/introducing-docker-model-runner/)

[plsnotracking]

#10454 (reply in thread), same issue as others.

2026-01-05 11:52:51.749 python[18697:1273054] 2026-01-05 11:52:51.783360 [E:onnxruntime:, sequential_executor.cc:572 ExecuteKernel] Non-zero status code returned while running 16946850393505866820_CoreML_16946850393505866820_32 node. Name:'CoreMLExecutionProvider_16946850393505866820_CoreML_16946850393505866820_32_32' Status Message: Error executing model: Unable to compute the prediction using a neural network model. It can be an invalid input data or broken/unsupported model (error code: -1).
2026-01-05 11:52:51.903 python[18697:1273054] 2026-01-05 11:52:51.937857 [E:onnxruntime:, sequential_executor.cc:572 ExecuteKernel] Non-zero status code returned while running 16946850393505866820_CoreML_16946850393505866820_32 node. Name:'CoreMLExecutionProvider_16946850393505866820_CoreML_16946850393505866820_32_32' Status Message: Error executing model: Unable to compute the prediction using a neural network model. It can be an invalid input data or broken/unsupported model (error code: -1).
2026-01-05 11:52:52.069 python[18697:1273054] 2026-01-05 11:52:52.103008 [E:onnxruntime:, sequential_executor.cc:572 ExecuteKernel] Non-zero status code returned while running 16946850393505866820_CoreML_16946850393505866820_32 node. Name:'CoreMLExecutionProvider_16946850393505866820_CoreML_16946850393505866820_32_32' Status Message: Error executing model: Unable to compute the prediction using a neural network model. It can be an invalid input data or broken/unsupported model (error code: -1).

Is this because OCR was introduced and now the old face recoginition model doesn't work?

[sebastianfredette]

Hey y'all. I'm not a developer, and I generally stay quiet about my crappy hobby code, but I wanted to post this here in hopes that it (A) helps some people on macOS and (B) inspires some community support around ML hardware acceleration on Apple Silicon. I (read: Claude) was able to throw this together today with quite bit of head-scratching.

It's a drop-in temporary replacement ('til they add official support) for immich-ML that runs on Apple's ML frameworks. It responds at the same API endpoints (and in the same format) as the official ML container, but uses MLX, Apple Vision, and CoreML to do the heavy lifting. I've only tested it on my M1 MacBook running Tahoe, but it seemed to work considerably faster than the official Immich-ML container running in Orbstack.

immich-ml-metal.

This is pretty much fully AI code, is not based on the original Immich-ML server, and I professionally advise none of you to use it. But maybe it could be considered a starting point. Or not.

[plsnotracking]

I just worked with the CPU, if anyone is interested, I mostly followed this: #10454 (reply in thread), then with a bit of smarts (hint: Claude), I kinda reached this

• start_immich_ml.sh, the changes I made were: disable CoreML, limit workers to 1, and OBJC_DISABLE_INITIALIZE_FORK_SAFETY

#!/bin/zsh
LOCKFILE="/tmp/immich_ml.lock"
if [ -e "$LOCKFILE" ] && kill -0 "$(cat $LOCKFILE)" 2>/dev/null; then
    echo "immich_ml is already running"
    exit 1
fi
echo $$ > "$LOCKFILE"
trap "rm -f $LOCKFILE" EXIT

export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES
export MACHINE_LEARNING_WORKERS=1

cd /<path-to-immich-repo>/machine-learning
git pull

# Disable CoreML
sed -i '' 's/"CoreMLExecutionProvider",/# "CoreMLExecutionProvider",/' \
  immich_ml/models/constants.py

/opt/homebrew/bin/uv sync --extra cpu --python=3.12
source .venv/bin/activate
python -m immich_ml

Then update the launch agent at ~/Library/LaunchAgents/com.user.immichml.daily.plist

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.user.immichml.daily</string>
    <key>ProgramArguments</key>
    <array>
        <string>/<immich-repo-full-path>/start_immich_ml.sh</string>
    </array>
    <key>StartCalendarInterval</key>
    <dict>
        <key>Hour</key>
        <integer>3</integer>
        <key>Minute</key>
        <integer>0</integer>
    </dict>
    <key>RunAtLoad</key>
    <true/>
    <key>EnvironmentVariables</key>
    <dict>
        <key>OBJC_DISABLE_INITIALIZE_FORK_SAFETY</key>
        <string>YES</string>
        <key>PATH</key>
        <string>/opt/homebrew/bin:/Users/dev/.local/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin</string>
    </dict>
    <key>StandardOutPath</key>
    <string>/tmp/immichml.log</string>
    <key>StandardErrorPath</key>
    <string>/tmp/immichml.err</string>
</dict>
</plist>

Then

launchctl unload ~/Library/LaunchAgents/com.user.immichml.daily.plist
launchctl load ~/Library/LaunchAgents/com.user.immichml.daily.plist

[ruimarinho]

What improvements do you see over running the ML server inside docker?

[plsnotracking]

I never tried running inside of docker but the baremetal one seems to be chugging along just fine.

I'm running 4 jobs (all limit to only 1 parallel job, my postgres can't any more of it):

• Smart Search: ViT-SO400M-16-SigLIP2-512__webli
• Generate Thumbnails
• Face Detection & Facial Recognition: antelopev2
• Ocr: PP-OCRv5_server (Chinese/Japanese/English)

I'm at about 28ish% CPU, with 8.4 GB RAM used up by the python process.

I wish the CoreML stuff works out eventually, this is more than I'd like but for now this is how I make do.

[ruimarinho]

If you're disabling CoreML, there's no advantage of running outside Docker. You're just adding complexity for the same CPU-only result, right?

[Xiseous]

Yes

…

On Tue, Feb 24, 2026 at 17:51 Rui Marinho ***@***.***> wrote: If you're disabling CoreML, there's no advantage of running outside Docker. You're just adding complexity for the same CPU-only result, right? — Reply to this email directly, view it on GitHub <#10454 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/APJSP63AUMOJAEFCZ2HLEC34NTPZ7AVCNFSM6AAAAABJPOHMBOVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTKOJRGY4TQOI> . You are receiving this because you commented.Message ID: ***@***.***>

[epheterson]

Hey, I made a project that maximizes Apple Silicon usage for dockerized Immich, offloading ML, transcoding and thumbnails from the CPU (and fixed the face bug in immich-ml-metal). I regenerated my entire library with it and it looks good, would love if others can check it out and give feedback or ideas to improve it.

https://github.com/epheterson/immich-apple-silicon