[BUG] redis (sentinel): ERR unknown command `eval`, with args beginning with:

deekerman commented

2026-02-19 23:16:30 -05:00

Owner

Originally created by @uhthomas on GitHub (May 23, 2023).

The bug

Hi, I'm using redis and sentinel and Immich won't start.

/usr/src/app/node_modules/redis-parser/lib/parser.js:179
    return new ReplyError(string)
           ^

ReplyError: ERR unknown command `eval`, with args beginning with: `--[[   Adds a job to the queue by doing the following:     - Increases the job counter if needed.     - Creates a new job key wi`,
    at parseError (/usr/src/app/node_modules/redis-parser/lib/parser.js:179:12)
    at parseType (/usr/src/app/node_modules/redis-parser/lib/parser.js:302:14) {
  command: {
    name: 'eval',
    args: [
      '--[[\n' +
        '  Adds a job to the queue by doing the following:\n' +
        '    - Increases the job counter if needed.\n' +
        '    - Creates a new job key with the job data.\n' +
        '\n' +
        '    - if delayed:\n' +
        '      - computes timestamp.\n' +
        '      - adds to delayed zset.\n' +
        "      - Emits a global event 'delayed' if the job is delayed.\n" +
        '    - if not delayed\n' +
        '      - Adds the jobId to the wait/paused list in one of three ways:\n' +
        '         - LIFO\n' +
        '         - FIFO\n' +
        '         - prioritized.\n' +
        '      - Adds the job to the "added" list so that workers gets notified.\n' +
        '\n' +
        '    Input:\n' +
        "      KEYS[1] 'wait',\n" +
        "      KEYS[2] 'paused'\n" +
        "      KEYS[3] 'meta-paused'\n" +
        "      KEYS[4] 'id'\n" +
        "      KEYS[5] 'delayed'\n" +
        "      KEYS[6] 'priority'\n" +
        '\n' +
        '      ARGV[1]  key prefix,\n' +
        '      ARGV[2]  custom id (will not generate one automatically)\n' +
        '      ARGV[3]  name\n' +
        '      ARGV[4]  data (json stringified job data)\n' +
        '      ARGV[5]  opts (json stringified job opts)\n' +
        '      ARGV[6]  timestamp\n' +
        '      ARGV[7]  delay\n' +
        '      ARGV[8]  delayedTimestamp\n' +
        '      ARGV[9]  priority\n' +
        '      ARGV[10] LIFO\n' +
        '      ARGV[11] token\n' +
        ']]\n' +
        'local jobId\n' +
        'local jobIdKey\n' +
        'local rcall = redis.call\n' +
        '\n' +
        'local jobCounter = rcall("INCR", KEYS[4])\n' +
        '\n' +
        'if ARGV[2] == "" then\n' +
        '  jobId = jobCounter\n' +
        '  jobIdKey = ARGV[1] .. jobId\n' +
        'else\n' +
        '  jobId = ARGV[2]\n' +
        '  jobIdKey = ARGV[1] .. jobId\n' +
        '  if rcall("EXISTS", jobIdKey) == 1 then\n' +
        '    return jobId .. "" -- convert to string\n' +
        '  end\n' +
        'end\n' +
        '\n' +
        '-- Store the job.\n' +
        'rcall("HMSET", jobIdKey, "name", ARGV[3], "data", ARGV[4], "opts", ARGV[5], "timestamp", ARGV[6], "delay", ARGV[7], "priority", ARGV[9])\n' +
        '\n' +
        '-- Check if job is delayed\n' +
        'local delayedTimestamp = tonumber(ARGV[8])\n' +
        'if(delayedTimestamp ~= 0) then\n' +
        '  local timestamp = delayedTimestamp * 0x1000 + bit.band(jobCounter, 0xfff)\n' +
        '  rcall("ZADD", KEYS[5], timestamp, jobId)\n' +
        '  rcall("PUBLISH", KEYS[5], delayedTimestamp)\n' +
        'else\n' +
        '  local target\n' +
        '\n' +
        '  -- Whe check for the meta-paused key to decide if we are paused or not\n' +
        '  -- (since an empty list and !EXISTS are not really the same)\n' +
        '  local paused\n' +
        '  if rcall("EXISTS", KEYS[3]) ~= 1 then\n' +
        '    target = KEYS[1]\n' +
        '    paused = false\n' +
        '  else\n' +
        '    target = KEYS[2]\n' +
        '    paused = true\n' +
        '  end\n' +
        '\n' +
        '  -- Standard or priority add\n' +
        '  local priority = tonumber(ARGV[9])\n' +
        '  if priority == 0 then\n' +
        '      -- LIFO or FIFO\n' +
        '    rcall(ARGV[10], target, jobId)\n' +
        '  else\n' +
        '    -- Priority add\n' +
        '    rcall("ZADD", KEYS[6], priority, jobId)\n' +
        '    local count = rcall("ZCOUNT", KEYS[6], 0, priority)\n' +
        '\n' +
        '    local len = rcall("LLEN", target)\n' +
        '    local id = rcall("LINDEX", target, len - (count-1))\n' +
        '    if id then\n' +
        '      rcall("LINSERT", target, "BEFORE", id, jobId)\n' +
        '    else\n' +
        '      rcall("RPUSH", target, jobId)\n' +
        '    end\n' +
        '\n' +
        '  end\n' +
        '\n' +
        '  -- Emit waiting event (wait..ing@token)\n' +
        '  rcall("PUBLISH", KEYS[1] .. "ing@" .. ARGV[11], jobId)\n' +
        'end\n' +
        '\n' +
        'return jobId .. "" -- convert to string\n',
      '6',
      'immich_bull:search-queue:wait',
      'immich_bull:search-queue:paused',
      'immich_bull:search-queue:meta-paused',
      'immich_bull:search-queue:id',
      'immich_bull:search-queue:delayed',
      'immich_bull:search-queue:priority',
      'immich_bull:search-queue:',
      '',
      'search-index-assets',
      '{}',
      '{"attempts":3,"removeOnComplete":true,"removeOnFail":false,"delay":0,"timestamp":1684872218171}',
      '1684872218171',
      '0',
      '0',
      '0',
      'LPUSH',
      'd1178744-332e-4a2f-bd8b-266684bc2fa7'
    ]
  }
}

Node.js v18.16.0

The OS that Immich Server is running on

Kubernetes

Version of Immich Server

1.57.1

Version of Immich Mobile App

N/A

Platform with the issue

Server
Web
Mobile

Your docker-compose.yml content

N/A

Your .env content

N/A

Reproduction steps

Use Redis Sentinel.

Additional information

github.com/uhthomas/automata@898f231a69/k8s/unwind/immich

Originally created by @uhthomas on GitHub (May 23, 2023). ### The bug Hi, I'm using redis and sentinel and Immich won't start. ``` /usr/src/app/node_modules/redis-parser/lib/parser.js:179 return new ReplyError(string) ^ ReplyError: ERR unknown command `eval`, with args beginning with: `--[[ Adds a job to the queue by doing the following: - Increases the job counter if needed. - Creates a new job key wi`, at parseError (/usr/src/app/node_modules/redis-parser/lib/parser.js:179:12) at parseType (/usr/src/app/node_modules/redis-parser/lib/parser.js:302:14) { command: { name: 'eval', args: [ '--[[\n' + ' Adds a job to the queue by doing the following:\n' + ' - Increases the job counter if needed.\n' + ' - Creates a new job key with the job data.\n' + '\n' + ' - if delayed:\n' + ' - computes timestamp.\n' + ' - adds to delayed zset.\n' + " - Emits a global event 'delayed' if the job is delayed.\n" + ' - if not delayed\n' + ' - Adds the jobId to the wait/paused list in one of three ways:\n' + ' - LIFO\n' + ' - FIFO\n' + ' - prioritized.\n' + ' - Adds the job to the "added" list so that workers gets notified.\n' + '\n' + ' Input:\n' + " KEYS[1] 'wait',\n" + " KEYS[2] 'paused'\n" + " KEYS[3] 'meta-paused'\n" + " KEYS[4] 'id'\n" + " KEYS[5] 'delayed'\n" + " KEYS[6] 'priority'\n" + '\n' + ' ARGV[1] key prefix,\n' + ' ARGV[2] custom id (will not generate one automatically)\n' + ' ARGV[3] name\n' + ' ARGV[4] data (json stringified job data)\n' + ' ARGV[5] opts (json stringified job opts)\n' + ' ARGV[6] timestamp\n' + ' ARGV[7] delay\n' + ' ARGV[8] delayedTimestamp\n' + ' ARGV[9] priority\n' + ' ARGV[10] LIFO\n' + ' ARGV[11] token\n' + ']]\n' + 'local jobId\n' + 'local jobIdKey\n' + 'local rcall = redis.call\n' + '\n' + 'local jobCounter = rcall("INCR", KEYS[4])\n' + '\n' + 'if ARGV[2] == "" then\n' + ' jobId = jobCounter\n' + ' jobIdKey = ARGV[1] .. jobId\n' + 'else\n' + ' jobId = ARGV[2]\n' + ' jobIdKey = ARGV[1] .. jobId\n' + ' if rcall("EXISTS", jobIdKey) == 1 then\n' + ' return jobId .. "" -- convert to string\n' + ' end\n' + 'end\n' + '\n' + '-- Store the job.\n' + 'rcall("HMSET", jobIdKey, "name", ARGV[3], "data", ARGV[4], "opts", ARGV[5], "timestamp", ARGV[6], "delay", ARGV[7], "priority", ARGV[9])\n' + '\n' + '-- Check if job is delayed\n' + 'local delayedTimestamp = tonumber(ARGV[8])\n' + 'if(delayedTimestamp ~= 0) then\n' + ' local timestamp = delayedTimestamp * 0x1000 + bit.band(jobCounter, 0xfff)\n' + ' rcall("ZADD", KEYS[5], timestamp, jobId)\n' + ' rcall("PUBLISH", KEYS[5], delayedTimestamp)\n' + 'else\n' + ' local target\n' + '\n' + ' -- Whe check for the meta-paused key to decide if we are paused or not\n' + ' -- (since an empty list and !EXISTS are not really the same)\n' + ' local paused\n' + ' if rcall("EXISTS", KEYS[3]) ~= 1 then\n' + ' target = KEYS[1]\n' + ' paused = false\n' + ' else\n' + ' target = KEYS[2]\n' + ' paused = true\n' + ' end\n' + '\n' + ' -- Standard or priority add\n' + ' local priority = tonumber(ARGV[9])\n' + ' if priority == 0 then\n' + ' -- LIFO or FIFO\n' + ' rcall(ARGV[10], target, jobId)\n' + ' else\n' + ' -- Priority add\n' + ' rcall("ZADD", KEYS[6], priority, jobId)\n' + ' local count = rcall("ZCOUNT", KEYS[6], 0, priority)\n' + '\n' + ' local len = rcall("LLEN", target)\n' + ' local id = rcall("LINDEX", target, len - (count-1))\n' + ' if id then\n' + ' rcall("LINSERT", target, "BEFORE", id, jobId)\n' + ' else\n' + ' rcall("RPUSH", target, jobId)\n' + ' end\n' + '\n' + ' end\n' + '\n' + ' -- Emit waiting event (wait..ing@token)\n' + ' rcall("PUBLISH", KEYS[1] .. "ing@" .. ARGV[11], jobId)\n' + 'end\n' + '\n' + 'return jobId .. "" -- convert to string\n', '6', 'immich_bull:search-queue:wait', 'immich_bull:search-queue:paused', 'immich_bull:search-queue:meta-paused', 'immich_bull:search-queue:id', 'immich_bull:search-queue:delayed', 'immich_bull:search-queue:priority', 'immich_bull:search-queue:', '', 'search-index-assets', '{}', '{"attempts":3,"removeOnComplete":true,"removeOnFail":false,"delay":0,"timestamp":1684872218171}', '1684872218171', '0', '0', '0', 'LPUSH', 'd1178744-332e-4a2f-bd8b-266684bc2fa7' ] } } Node.js v18.16.0 ``` ### The OS that Immich Server is running on Kubernetes ### Version of Immich Server 1.57.1 ### Version of Immich Mobile App N/A ### Platform with the issue - [X] Server - [ ] Web - [ ] Mobile ### Your docker-compose.yml content ```YAML N/A ``` ### Your .env content ```Shell N/A ``` ### Reproduction steps ```bash Use Redis Sentinel. ``` ### Additional information https://github.com/uhthomas/automata/tree/898f231a69cffdbff6d6e1d74bc4d3d0ed0441ba/k8s/unwind/immich

deekerman closed this issue

2026-02-19 23:16:31 -05:00

deekerman commented

2026-02-19 23:16:31 -05:00

Author

Owner

@alextran1502 commented on GitHub (May 23, 2023):

@samip5 Do you have much experience in this?

@alextran1502 commented on GitHub (May 23, 2023): @samip5 Do you have much experience in this?

deekerman commented

2026-02-19 23:16:31 -05:00

Author

Owner

@uhthomas commented on GitHub (May 23, 2023):

I was thinking maybe a bad Redis version, but this is the image I'm using redis:6.2.6-alpine. Maybe I should try redis 7?

@uhthomas commented on GitHub (May 23, 2023): I was thinking maybe a bad Redis version, but this is the image I'm using `redis:6.2.6-alpine`. Maybe I should try redis 7?

deekerman commented

2026-02-19 23:16:32 -05:00

Author

Owner

@jrasm91 commented on GitHub (May 23, 2023):

Not sure what I'm looking at here, but none of this looks familiar. The error is most likely not an immich bug, but related to however you are deploying it, which looks custom.

@jrasm91 commented on GitHub (May 23, 2023): Not sure what I'm looking at here, but none of this looks familiar. The error is most likely not an immich bug, but related to however you are deploying it, which looks custom.

deekerman commented

2026-02-19 23:16:32 -05:00

Author

Owner

@alextran1502 commented on GitHub (May 23, 2023):

We are using Redis 6.2 for the official compose file.

@alextran1502 commented on GitHub (May 23, 2023): We are using Redis 6.2 for the official compose file.

deekerman commented

2026-02-19 23:16:32 -05:00

Author

Owner

@uhthomas commented on GitHub (May 23, 2023):

We are using Redis 6.2 for the official compose file.

ah :(

@uhthomas commented on GitHub (May 23, 2023): > We are using Redis 6.2 for the official compose file. ah :(

deekerman commented

2026-02-19 23:16:32 -05:00

Author

Owner

@uhthomas commented on GitHub (May 23, 2023):

Not sure what I'm looking at here, but none of this looks familiar. The error is most likely not an immich bug, but related to however you are deploying it, which looks custom.

It should just be standard redis/sentinel.

@uhthomas commented on GitHub (May 23, 2023): > Not sure what I'm looking at here, but none of this looks familiar. The error is most likely not an immich bug, but related to however you are deploying it, which looks custom. It should just be standard redis/sentinel.

deekerman commented

2026-02-19 23:16:32 -05:00

Author

Owner

@samip5 commented on GitHub (May 23, 2023):

@samip5 Do you have much experience in this?

I don't sorry, but that sounds like an issue with how the config is encoded.

@uhthomas Could you please show us the base64 encoded redis sentinel config you're passing to immich in env? (I'm sorry, but I cannot read the cue things in your repo)

@samip5 commented on GitHub (May 23, 2023): > @samip5 Do you have much experience in this? I don't sorry, but that sounds like an issue with how the config is encoded. @uhthomas Could you please show us the base64 encoded redis sentinel config you're passing to immich in env? (I'm sorry, but I cannot read the cue things in your repo)

deekerman commented

2026-02-19 23:16:33 -05:00

Author

Owner

@uhthomas commented on GitHub (May 23, 2023):

@samip5

I'm not quite sure exactly what you're looking for, but I hope this helps.

❯ k -n immich get deploy immich-server -ojson | jq '.spec.template.spec.containers[].env[] | select(.name | contains("REDIS"))'
{
  "name": "REDIS_HOSTNAME",
  "value": "rfs-redis"
}
{
  "name": "REDIS_PORT",
  "value": "26379"
}

❯ k -n immich get cm rfr-redis -ojson | jq '.data["redis.conf"]' -r
slaveof 127.0.0.1 6379
port 6379
tcp-keepalive 60
save 900 1
save 300 10
user pinger -@all +ping on >pingpass

❯ k -n immich get cm rfs-redis -ojson | jq '.data["sentinel.conf"]' -r
sentinel monitor mymaster 127.0.0.1 6379 2
sentinel down-after-milliseconds mymaster 1000
sentinel failover-timeout mymaster 3000
sentinel parallel-syncs mymaster 2

@uhthomas commented on GitHub (May 23, 2023): @samip5 I'm not quite sure exactly what you're looking for, but I hope this helps. ```sh ❯ k -n immich get deploy immich-server -ojson | jq '.spec.template.spec.containers[].env[] | select(.name | contains("REDIS"))' { "name": "REDIS_HOSTNAME", "value": "rfs-redis" } { "name": "REDIS_PORT", "value": "26379" } ``` ```sh ❯ k -n immich get cm rfr-redis -ojson | jq '.data["redis.conf"]' -r slaveof 127.0.0.1 6379 port 6379 tcp-keepalive 60 save 900 1 save 300 10 user pinger -@all +ping on >pingpass ``` ```sh ❯ k -n immich get cm rfs-redis -ojson | jq '.data["sentinel.conf"]' -r sentinel monitor mymaster 127.0.0.1 6379 2 sentinel down-after-milliseconds mymaster 1000 sentinel failover-timeout mymaster 3000 sentinel parallel-syncs mymaster 2 ```

deekerman commented

2026-02-19 23:16:33 -05:00

Author

Owner

@uhthomas commented on GitHub (May 23, 2023):

Strange https://github.com/OptimalBits/bull/issues/1026, I guess I should try to figure out why "eval" is missing.

@uhthomas commented on GitHub (May 23, 2023): Strange https://github.com/OptimalBits/bull/issues/1026, I guess I should try to figure out why "eval" is missing.

deekerman commented

2026-02-19 23:16:33 -05:00

Author

Owner

@samip5 commented on GitHub (May 23, 2023):

@samip5

I'm not quite sure exactly what you're looking for, but I hope this helps.

❯ k -n immich get deploy immich-server -ojson | jq '.spec.template.spec.containers[].env[] | select(.name | contains("REDIS"))'
{
  "name": "REDIS_HOSTNAME",
  "value": "rfs-redis"
}
{
  "name": "REDIS_PORT",
  "value": "26379"
}

❯ k -n immich get cm rfr-redis -ojson | jq '.data["redis.conf"]' -r
slaveof 127.0.0.1 6379
port 6379
tcp-keepalive 60
save 900 1
save 300 10
user pinger -@all +ping on >pingpass

❯ k -n immich get cm rfs-redis -ojson | jq '.data["sentinel.conf"]' -r
sentinel monitor mymaster 127.0.0.1 6379 2
sentinel down-after-milliseconds mymaster 1000
sentinel failover-timeout mymaster 3000
sentinel parallel-syncs mymaster 2

That seems like it's not correct sentinel config for immich, as it's expecting a base64 encoded url for it in REDIS_URL env variable.

Please see https://github.com/immich-app/immich/blob/main/docker/example.env#L21L24

@samip5 commented on GitHub (May 23, 2023): > @samip5 > > I'm not quite sure exactly what you're looking for, but I hope this helps. > > ```shell > ❯ k -n immich get deploy immich-server -ojson | jq '.spec.template.spec.containers[].env[] | select(.name | contains("REDIS"))' > { > "name": "REDIS_HOSTNAME", > "value": "rfs-redis" > } > { > "name": "REDIS_PORT", > "value": "26379" > } > ``` > > ```shell > ❯ k -n immich get cm rfr-redis -ojson | jq '.data["redis.conf"]' -r > slaveof 127.0.0.1 6379 > port 6379 > tcp-keepalive 60 > save 900 1 > save 300 10 > user pinger -@all +ping on >pingpass > ``` > > ```shell > ❯ k -n immich get cm rfs-redis -ojson | jq '.data["sentinel.conf"]' -r > sentinel monitor mymaster 127.0.0.1 6379 2 > sentinel down-after-milliseconds mymaster 1000 > sentinel failover-timeout mymaster 3000 > sentinel parallel-syncs mymaster 2 > ``` That seems like it's not correct sentinel config for immich, as it's expecting a base64 encoded url for it in REDIS_URL env variable. Please see https://github.com/immich-app/immich/blob/main/docker/example.env#L21L24

deekerman commented

2026-02-19 23:16:33 -05:00

Author

Owner

@uhthomas commented on GitHub (May 23, 2023):

I don't think that's it.

github.com/immich-app/immich@e2bd7e1e08/server/libs/infra/src/infra.config.ts (L18-L19)

@uhthomas commented on GitHub (May 23, 2023): I don't think that's it. https://github.com/immich-app/immich/blob/e2bd7e1e085e7d8828fe3f9e201afd99e1eaf170/server/libs/infra/src/infra.config.ts#L18-L19

deekerman commented

2026-02-19 23:16:34 -05:00

Author

Owner

@samip5 commented on GitHub (May 23, 2023):

I don't think that's it.

github.com/immich-app/immich@e2bd7e1e08/server/libs/infra/src/infra.config.ts (L18-L19)

github.com/immich-app/immich@e2bd7e1e08/server/libs/infra/src/infra.config.ts (L9)

It's expecting it to be normal redis.

@samip5 commented on GitHub (May 23, 2023): > I don't think that's it. > > https://github.com/immich-app/immich/blob/e2bd7e1e085e7d8828fe3f9e201afd99e1eaf170/server/libs/infra/src/infra.config.ts#L18-L19 https://github.com/immich-app/immich/blob/e2bd7e1e085e7d8828fe3f9e201afd99e1eaf170/server/libs/infra/src/infra.config.ts#L9 It's expecting it to be normal redis.

deekerman commented

2026-02-19 23:16:34 -05:00

Author

Owner

@bo0tzz commented on GitHub (May 23, 2023):

To configure redis sentinel please use the REDIS_URL env var.

@bo0tzz commented on GitHub (May 23, 2023): To configure redis sentinel please use the `REDIS_URL` env var.

deekerman commented

2026-02-19 23:16:34 -05:00

Author

Owner

@samip5 commented on GitHub (May 23, 2023):

Hi, @uhthomas you might be interested our hobbyist communify of people running Kubernetes at home, which you don't seem to be apart of yet.

https://k8s-at-home.com

@samip5 commented on GitHub (May 23, 2023): Hi, @uhthomas you might be interested our hobbyist communify of people running Kubernetes at home, which you don't seem to be apart of yet. https://k8s-at-home.com

deekerman commented

2026-02-19 23:16:35 -05:00

Author

Owner

@uhthomas commented on GitHub (May 23, 2023):

Just following up, thank you so much for the help and apologies for my misunderstanding / lack of understanding with respect to redis and sentinels. The URL worked as expected.

github.com/uhthomas/automata@0549cc9fbd/k8s/unwind/immich/immich_server/deployment_list.cue (L64-L73)

@uhthomas commented on GitHub (May 23, 2023): Just following up, thank you so much for the help and apologies for my misunderstanding / lack of understanding with respect to redis and sentinels. The URL worked as expected. https://github.com/uhthomas/automata/blob/0549cc9fbdf656e969b6ba963a34cc6e94402bd7/k8s/unwind/immich/immich_server/deployment_list.cue#L64-L73

deekerman commented

2026-02-19 23:16:35 -05:00

Author

Owner

@uhthomas commented on GitHub (May 23, 2023):

Hi, @uhthomas you might be interested our hobbyist communify of people running Kubernetes at home, which you don't seem to be apart of yet.

https://k8s-at-home.com

Thanks! I'll take a look :)

@uhthomas commented on GitHub (May 23, 2023): > Hi, @uhthomas you might be interested our hobbyist communify of people running Kubernetes at home, which you don't seem to be apart of yet. > > https://k8s-at-home.com Thanks! I'll take a look :)

Rows
Columns

[BUG] redis (sentinel): ERR unknown command eval, with args beginning with: #884

The bug

The OS that Immich Server is running on

Version of Immich Server

Version of Immich Mobile App

Platform with the issue

Your docker-compose.yml content

Your .env content

Reproduction steps

Additional information

[BUG] redis (sentinel): ERR unknown command `eval`, with args beginning with: #884