Vapor use queues radis connect error

I followed the tutorial to connect the radis, the radis started with Docker.
But running Vapor, it keeps prompting Radis connection failure

docker-compose.yml

# Docker Compose file for Vapor
#
# Install Docker on your system to run and test
# your Vapor app in a production-like environment.
#
# Note: This file is intended for testing and does not
# implement best practices for a production deployment.
#
# Learn more: https://docs.docker.com/compose/reference/
#
#   Build images: docker-compose build
#      Start app: docker-compose up app
# Start database: docker-compose up db
# Run migrations: docker-compose run migrate
#       Stop all: docker-compose down (add -v to wipe db)
#
version: '3.7'

volumes:
  db_data:

x-shared_environment: &shared_environment
  LOG_LEVEL: ${LOG_LEVEL:-debug}
  DATABASE_HOST: db
  DATABASE_NAME: vapor_database
  DATABASE_USERNAME: vapor_username
  DATABASE_PASSWORD: vapor_password
  
services:
  app:
    image: spm_mirror_server:latest
    build:
      context: .
    environment:
      <<: *shared_environment
    depends_on:
      - db
    ports:
      - '8080:8080'
    # user: '0' # uncomment to run as root for testing purposes even though Dockerfile defines 'vapor' user.
    command: ["serve", "--env", "production", "--hostname", "0.0.0.0", "--port", "8080"]
  migrate:
    image: spm_mirror_server:latest
    build:
      context: .
    environment:
      <<: *shared_environment
    depends_on:
      - db
    command: ["migrate", "--yes"]
    deploy:
      replicas: 0
  revert:
    image: spm_mirror_server:latest
    build:
      context: .
    environment:
      <<: *shared_environment
    depends_on:
      - db
    command: ["migrate", "--revert", "--yes"]
    deploy:
      replicas: 0
  db:
    image: postgres:14-alpine
    volumes:
      - db_data:/var/lib/postgresql/data/pgdata
    environment:
      PGDATA: /var/lib/postgresql/data/pgdata
      POSTGRES_USER: vapor_username
      POSTGRES_PASSWORD: vapor_password
      POSTGRES_DB: vapor_database
    ports:
      - '5432:5432'

  redis:
    image: redis:latest
    ports:
      - '6379:6379'

Vapor Config file

import Fluent
import FluentPostgresDriver
import Vapor
import QueuesRedisDriver

// configures your application
public func configure(_ app: Application) throws {
    // uncomment to serve files from /Public folder
    // app.middleware.use(FileMiddleware(publicDirectory: app.directory.publicDirectory))

    app.databases.use(.postgres(
        hostname: Environment.get("DATABASE_HOST") ?? "localhost",
        port: Environment.get("DATABASE_PORT").flatMap(Int.init(_:)) ?? PostgresConfiguration.ianaPortNumber,
        username: Environment.get("DATABASE_USERNAME") ?? "vapor_username",
        password: Environment.get("DATABASE_PASSWORD") ?? "vapor_password",
        database: Environment.get("DATABASE_NAME") ?? "vapor_database"
    ), as: .psql)

    
    try app.routes.register(collection: MirrorController())
    
    app.migrations.add(CreateMirror())
    app.migrations.add(CreateMirrorStack())
    app.migrations.add(CreateMirrorRequest())
    app.migrations.add(UpdateMirror001())
    app.migrations.add(UpdateMirror002())
    try app.autoMigrate().wait()
    
    app.logger.logLevel = .info
    // register routes
    try routes(app)
    
    try app.queues.use(.redis(url: "redis://127.0.0.1:6379"))
    app.queues.add(MirrorJob())
    app.queues.add(StartMirrorJob())
    app.queues.add(UpdateMirrorJob())
    app.queues.add(WaitMirrorJob())
    try app.queues.startInProcessJobs(on: .default)

    app.queues.schedule(TimeJob())
    .daily()
    .at(.noon)
    let config = try MirrorConfigration()
    let job = MirrorJob.PayloadData(config: config)
    Task {
        let isRunning = await mirrorJobStatus.isRunning
        guard !isRunning else {
            return
        }
        await mirrorJobStatus.start()
        do {
            try await app.queues.queue.dispatch(MirrorJob.self, job)
        } catch (let e) {
            let hook = WeiXinWebHooks(app: app, url: config.wxHookUrl)
            hook.sendContent(e.localizedDescription, in: app.client)
        }
        await mirrorJobStatus.stop()
    }
}

Vapor log

[ NOTICE ] Server starting on http://127.0.0.1:8080
[ INFO ] Dispatched queue job [job_id: E0FE4D4A-EE76-4AA7-9D43-00AC0F25212C, job_name: MirrorJob, queue: default]
[ INFO ] Dequeing job [job_id: E0FE4D4A-EE76-4AA7-9D43-00AC0F25212C, job_name: MirrorJob, queue: default]
[ INFO ] MirrorJob: [job_id: E0FE4D4A-EE76-4AA7-9D43-00AC0F25212C]
[ ERROR ] Job run failed: RedisConnectionPoolError(baseError: RediStack.RedisConnectionPoolError.BaseError.timedOutWaitingForConnection)
[ ERROR ] Job run failed: RedisConnectionPoolError(baseError: RediStack.RedisConnectionPoolError.BaseError.timedOutWaitingForConnection)
[ ERROR ] Job run failed: RedisConnectionPoolError(baseError: RediStack.RedisConnectionPoolError.BaseError.timedOutWaitingForConnection)
[ ERROR ] Job run failed: RedisConnectionPoolError(baseError: RediStack.RedisConnectionPoolError.BaseError.timedOutWaitingForConnection)
[ ERROR ] Job run failed: RedisConnectionPoolError(baseError: RediStack.RedisConnectionPoolError.BaseError.timedOutWaitingForConnection)
[ ERROR ] Job run failed: RedisConnectionPoolError(baseError: RediStack.RedisConnectionPoolError.BaseError.timedOutWaitingForConnection)
[ ERROR ] Job run failed: RedisConnectionPoolError(baseError: RediStack.RedisConnectionPoolError.BaseError.timedOutWaitingForConnection)
[ ERROR ] Job run failed: RedisConnectionPoolError(baseError: RediStack.RedisConnectionPoolError.BaseError.timedOutWaitingForConnection)
[ ERROR ] Job run failed: RedisConnectionPoolError(baseError: RediStack.RedisConnectionPoolError.BaseError.timedOutWaitingForConnection)
[ ERROR ] Job run failed: RedisConnectionPoolError(baseError: RediStack.RedisConnectionPoolError.BaseError.timedOutWaitingForConnection)
[ ERROR ] Job run failed: RedisConnectionPoolError(baseError: RediStack.RedisConnectionPoolError.BaseError.timedOutWaitingForConnection)
[ ERROR ] Job run failed: RedisConnectionPoolError(baseError: RediStack.RedisConnectionPoolError.BaseError.timedOutWaitingForConnection)
[ ERROR ] Job run failed: RedisConnectionPoolError(baseError: RediStack.RedisConnectionPoolError.BaseError.timedOutWaitingForConnection)
[ ERROR ] Job run failed: RedisConnectionPoolError(baseError: RediStack.RedisConnectionPoolError.BaseError.timedOutWaitingForConnection)
[ ERROR ] Job run failed: RedisConnectionPoolError(baseError: RediStack.RedisConnectionPoolError.BaseError.timedOutWaitingForConnection)
[ ERROR ] Job run failed: RedisConnectionPoolError(baseError: RediStack.RedisConnectionPoolError.BaseError.timedOutWaitingForConnection)
[ ERROR ] Job run failed: RedisConnectionPoolError(baseError: RediStack.RedisConnectionPoolError.BaseError.timedOutWaitingForConnection)
[ ERROR ] Job run failed: RedisConnectionPoolError(baseError: RediStack.RedisConnectionPoolError.BaseError.timedOutWaitingForConnection)
[ ERROR ] Job run failed: RedisConnectionPoolError(baseError: RediStack.RedisConnectionPoolError.BaseError.timedOutWaitingForConnection)
[ ERROR ] Job run failed: RedisConnectionPoolError(baseError: RediStack.RedisConnectionPoolError.BaseError.timedOutWaitingForConnection)

Radis log

1:C 08 Apr 2022 02:52:46.212 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo

1:C 08 Apr 2022 02:52:46.212 # Redis version=6.2.6, bits=64, commit=00000000, modified=0, pid=1, just started

1:C 08 Apr 2022 02:52:46.212 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf

1:M 08 Apr 2022 02:52:46.213 * monotonic clock: POSIX clock_gettime

1:M 08 Apr 2022 02:52:46.213 * Running mode=standalone, port=6379.

1:M 08 Apr 2022 02:52:46.214 # Server initialized

1:M 08 Apr 2022 02:52:46.214 * Ready to accept connections

1:M 08 Apr 2022 02:53:03.040 # Possible SECURITY ATTACK detected. It looks like somebody is sending POST or Host: commands to Redis. This is likely due to an attacker attempting to use Cross Protocol Scripting to compromise your Redis instance. Connection aborted.

1:M 08 Apr 2022 03:52:47.023 * 1 changes in 3600 seconds. Saving...

1:M 08 Apr 2022 03:52:47.024 * Background saving started by pid 26

26:C 08 Apr 2022 03:52:47.028 * DB saved on disk

26:C 08 Apr 2022 03:52:47.028 * RDB: 0 MB of memory used by copy-on-write

1:M 08 Apr 2022 03:52:47.127 * Background saving terminated with success

1:M 08 Apr 2022 04:59:26.685 * 1 changes in 3600 seconds. Saving...

1:M 08 Apr 2022 04:59:26.686 * Background saving started by pid 27

27:C 08 Apr 2022 04:59:26.692 * DB saved on disk

27:C 08 Apr 2022 04:59:26.692 * RDB: 0 MB of memory used by copy-on-write

1:M 08 Apr 2022 04:59:26.790 * Background saving terminated with success

1:M 08 Apr 2022 05:59:27.040 * 1 changes in 3600 seconds. Saving...

1:M 08 Apr 2022 05:59:27.040 * Background saving started by pid 28

28:C 08 Apr 2022 05:59:27.044 * DB saved on disk

28:C 08 Apr 2022 05:59:27.045 * RDB: 0 MB of memory used by copy-on-write

1:M 08 Apr 2022 05:59:27.142 * Background saving terminated with success

1:signal-handler (1649400138) Received SIGTERM scheduling shutdown...

1:M 08 Apr 2022 06:42:19.027 # User requested shutdown...

1:M 08 Apr 2022 06:42:19.027 * Saving the final RDB snapshot before exiting.

1:M 08 Apr 2022 06:42:19.032 * DB saved on disk

1:M 08 Apr 2022 06:42:19.032 # Redis is now ready to exit, bye bye...

1:C 08 Apr 2022 06:42:19.719 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo

1:C 08 Apr 2022 06:42:19.719 # Redis version=6.2.6, bits=64, commit=00000000, modified=0, pid=1, just started

1:C 08 Apr 2022 06:42:19.719 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf

1:M 08 Apr 2022 06:42:19.720 * monotonic clock: POSIX clock_gettime

1:M 08 Apr 2022 06:42:19.722 * Running mode=standalone, port=6379.

1:M 08 Apr 2022 06:42:19.723 # Server initialized

1:M 08 Apr 2022 06:42:19.723 * Loading RDB produced by version 6.2.6

1:M 08 Apr 2022 06:42:19.723 * RDB age 0 seconds

1:M 08 Apr 2022 06:42:19.723 * RDB memory usage when created 0.77 Mb

1:M 08 Apr 2022 06:42:19.724 # Done loading RDB, keys loaded: 2, keys expired: 0.

1:M 08 Apr 2022 06:42:19.724 * DB loaded from disk: 0.001 seconds

1:M 08 Apr 2022 06:42:19.724 * Ready to accept connections

Looking at the Radis log the connection was refused due to security

Which tutorial did you follow? As you're running the app and Redis in compose you need to set the Redis hostname in your Vapor config to the name of your Redis container (redis)

I modified the code as follows

    app.redis.configuration = try RedisConfiguration(hostname: "0.0.0.0")
    try app.queues.use(.redis(url: "redis://0.0.0.0:6379"))

[ ERROR ] Job run failed: RedisConnectionPoolError(baseError: RediStack.RedisConnectionPoolError.BaseError.timedOutWaitingForConnection)

That's not what I meant, you should configure it to be

app.redis.configuration = try RedisConfiguration(hostname: "redis")
    try app.queues.use(.redis(url: "redis://redis:6379"))

But that's only when working in Docker, you should really pass it in via an environment variable

I am currently running in Docker, the current Container list is as follows
NAMES
c380ba88b2c6 spm_mirror_server:latest "./Run serve --env p…" 46 minutes ago Up 46 minutes 0.0.0.0:8080->8080/tcp spm_mirror_server-app-1
ee04656587ff redis:latest "docker-entrypoint.s…" About an hour ago Up 53 minutes 0.0.0.0:6379->6379/tcp spm_mirror_server-radis-1
df4cdda11ed2 postgres:14-alpine "docker-entrypoint.s…" 7 days ago Up 6 hours 0.0.0.0:5432->5432/tcp spm_mirror_server-db-1

Is the corresponding name spm_mirror_server-app-1

The compose network sets the hostname to the name of the service defined in your docker-compose.yml. See Networking in Compose | Docker Documentation for more details

That error is thrown to avoid infinite waiting for a connection to become available. Check the RedisConfiguration and maybe increase some of the defaults to give more time.

@0xTim @Mordil

I use redis client to connect via 127.0.0.1 redis is connectable. But on Vapo r via 127.0.0.1 :link: timeout

RedisConfiguration has some properties that are set to control the amount of time RediStack (Vapor) will wait to create a connection for you.

Try changing those values to 1 second and see if that improves what you're seeing.

I deployed on the server and didn't see any errors. I'm not sure if it's OK

@josercc if you're running the Vapor app inside Docker compose you need to use redis as the hostname if it's connecting to a Redis container in the compose stack. If you're running Vapor locally (outside of Docker) and Redis is running in Docker (and exposed) then it's 127.0.0.1. This is why using environment variables is a good way forward

@Mordil @0xTim

Thank you for your help and reply, it's working fine now. thank you very much.

Which tutorial did you follow? As you're running the app and Redis in compose you need to season new smoker in your Vapor config to the name.

I follow Vapor: Redis → Overview Here is the tutorial setup. I set it through redis: and run the normal link in the server docker. I can start redis through local docker or directly brew servers start redis through 127.0.0.1: unable to connect. I try to download some connection terminals of redis, which can connect and operate normally. I don't know whether it's the problem of vapor or the local network

I started redis through docker. I developed links through Xcode through 127.0.0.1. It is normal for redis to set and obtain values. However, when some queues are not completed, the next time you start automatic execution, you will be prompted with a link error

Hi. josercc, were you able to get this working where Redis is in Docker and your application is running locally in XCode? No matter what I try I am still getting this RedisConnectionPoolError that you describe. Thanks.

I didn't solve it. There was no error report because the task didn't start. I made a test error. Using redis will still explode this error. At present, I use the community fluent engine to start the task and give up using redis

One of my colleague, did have the same errors logs (but not me on the same project but Intel Mac) on M1 Pro macOS 13 beta the other day with Xcode Intel version, we thought that maybe something with the cross-architecture was making the connection to timeout because other that the queues was working

My error report is on the 19 year old Mac Pro. I don't know whether it is due to the influence of agent software such as clash pro. At present, I replace it with the fluent Library of the community, and everything is normal

Jose,

Thank you for this. I am using the fluent driver now as well and seems to work.