Version 1.18 crashes for me due to mailer.HOST config #22274

nunoperalta · 2022-12-30T05:37:24Z

Description

Hello,

In a loop, 1.18 keeps crashing forever:

Server listening on :: port 22.
Server listening on 0.0.0.0 port 22.
2022/12/30 05:11:07 cmd/web.go:106:runWeb() [I] Starting Gitea on PID: 15
2022/12/30 05:11:07 ...s/setting/setting.go:613:deprecatedSetting() [E] Deprecated fallback `[server]` `LFS_CONTENT_PATH` present. Use `[lfs]` `PATH` instead. This fallback will be removed in v1.19.0
2022/12/30 05:11:07 cmd/web.go:160:runWeb() [I] Global init
2022/12/30 05:11:07 ...s/setting/setting.go:613:deprecatedSetting() [E] Deprecated fallback `[server]` `LFS_CONTENT_PATH` present. Use `[lfs]` `PATH` instead. This fallback will be removed in v1.19.0
2022/12/30 05:11:07 routers/init.go:116:GlobalInitInstalled() [I] Git Version: 2.36.3, Wire Protocol Version 2 Enabled (home: /data/gitea/home)
2022/12/30 05:11:07 routers/init.go:117:GlobalInitInstalled() [I] AppPath: /usr/local/bin/gitea
2022/12/30 05:11:07 routers/init.go:118:GlobalInitInstalled() [I] AppWorkPath: /app/gitea
2022/12/30 05:11:07 routers/init.go:119:GlobalInitInstalled() [I] Custom path: /data/gitea
2022/12/30 05:11:07 routers/init.go:120:GlobalInitInstalled() [I] Log path: /data/gitea/log
2022/12/30 05:11:07 routers/init.go:121:GlobalInitInstalled() [I] Configuration file: /data/gitea/conf/app.ini
2022/12/30 05:11:07 routers/init.go:122:GlobalInitInstalled() [I] Run Mode: Prod
Received signal 15; terminating.

Once I downgrade to 1.17, all is working again.

I'm not sure what to do, or how to investigate further.

MySQL is running fine:

2022-12-30T05:10:29.580620Z 0 [System] [MY-010931] [Server] /usr/sbin/mysqld: ready for connections. Version: '8.0.31'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  MySQL Community Server - GPL.

Also, I'm not sure how to fix the LFS_CONTENT_PATH issue.
What I currently have is:

LFS_START_SERVER = true
LFS_CONTENT_PATH = /data/git/lfs
LFS_JWT_SECRET   = ...

Do I just move the LFS_CONTENT_PATH like this?

[lfs]
PATH = /data/git/lfs

and leave the other LFS_* as they are?

Thank you very much.

Gitea Version

1.18

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

No response

Git Version

Bundled in Docker

Operating System

Bundled in Docker

How are you running Gitea?

services:
  server:
    container_name: gitea_server
    image: gitea/gitea:latest
    environment:
      - USER_UID=...
      - USER_GID=...
      - DB_TYPE=mysql
      - DB_HOST=db:3306
      - RUN_MODE=prod
      - ROOT_URL="https://..../"
    env_file:
      - ./db.env
    restart: always
    networks:
      - gitea
    volumes:
      - /.../data:/data
    ports:
      - "127.0.0.1:...:3000"
    depends_on:
      - db
  db:
    container_name: gitea_db
    image: mysql:8.0
    restart: always
    env_file:
      - ./db.env
    networks:
      - gitea
    volumes:
      - /.../mysql:/var/lib/mysql
    cap_add:
      # mbind: Operation not permitted
      - SYS_NICE

networks:
  gitea:
    external: false

Database

MySQL

The text was updated successfully, but these errors were encountered:

lunny · 2022-12-30T06:17:58Z

I tested in my macOS with gitea/gitea:latest which is point to 1.18.0 currently. It's work.

docker pull gitea/gitea:latest
docker run -p 3000:3000 gitea/gitea:latest

You can add the new section in your app.ini and old content could be kept and be removed after this work.

[lfs]
PATH = /data/git/lfs

rafipiccolo · 2022-12-30T07:22:00Z

same bug for me, but i cant revert to version 1.17
because i now have this error :

Server listening on :: port 22.
Server listening on 0.0.0.0 port 22.
Your database (migration version: 231) is for a newer Gitea, you can not use the newer database for this old Gitea release (224).
Gitea will exit to keep your database safe and unchanged. Please use the correct Gitea release, do not change the migration version manually (incorrect manual operation may lose data).
Received signal 15; terminating.

stuck as hell :) I'm considering restoring a backup or waiting for patch.

techknowlogick · 2022-12-30T07:48:53Z

signal 15 is SIGTERM, which means Gitea isn't exiting something else is telling it to exit. Are you able to provide "trace logs" for what is happening prior to exit?

rafipiccolo · 2022-12-30T08:08:13Z

how would you activate "trace logs" for gitea with docker ?

the thing is, the containers dies right after start so i believe it is killed from inside the image. (same second)

the only mecanism that could kill the container from outside that i know of is with docker swarm : where a faulty healthcheck would kill the unhealthy container (also it would leave it about 1:30 to reach healtyness before sending a sigterm)

nunoperalta · 2022-12-30T08:16:46Z

It does take a few seconds to terminate, though.

How do I do a "trace logs"? I'm not finding anything about "docker trace logs". The only thing I find is the normal docker logs which I've already provided on the original post above.

btw, I also had a look at the mailer breaking changes, but I believe there's nothing to be changed on my side:

[mailer]
ENABLED = true
HOST    = ...
FROM    = ...
USER    = ...
PASSWD  = ...

wolfogre · 2022-12-30T10:15:55Z

@nunoperalta @rafipiccolo Could you please just set the log level to Trace and restart the Gitea to see what it outputs?

Update the conf file like:

[log]
LEVEL = Trace

I tried to figure it out but didn't find anything. I believe more logs would be helpful.

rafipiccolo · 2022-12-30T10:37:09Z

thanks for your comment.
i noticed a fresh install with version 1.18 works just fine.

from what i see, it is indeed a problem with a previous configuration.
ps: i never changed configs, they are all autogenerated by gitea itself
therefore i would expect gitea to update it automatically to not crash upon updates.

you comment led me to this file : /gitea/conf/app.ini

[log]
ROOT_PATH = /data/gitea/log
MODE      = file
LEVEL     = trace

for me the logs stay the same as nunoperalta

but i see this in /gitea/log/gitea.log

2022/12/30 10:30:55 ...dules/setting/log.go:335:newLogService() [I] Gitea Log Mode: File(File:trace)
2022/12/30 10:30:55 ...dules/setting/log.go:249:generateNamedLogger() [I] Router Log: Console(console:trace)
2022/12/30 10:30:55 ...les/setting/cache.go:76:newCacheService() [I] Cache Service Enabled
2022/12/30 10:30:55 ...les/setting/cache.go:91:newCacheService() [I] Last Commit Cache Service Enabled
2022/12/30 10:30:55 ...s/setting/session.go:73:newSessionService() [I] Session Service Enabled
2022/12/30 10:30:55 ...s/setting/setting.go:613:deprecatedSetting() [E] Deprecated fallback `[mailer]` `HOST` present. Use `[mailer]` `SMTP_ADDR` instead. This fallback will be removed in v1.19.0
2022/12/30 10:30:55 ...es/setting/mailer.go:75:newMailService() [F] Invalid mailer.HOST (in-v3.mailjet.com): address in-v3.mailjet.com: missing port in address

rafipiccolo · 2022-12-30T10:41:33Z

OK, i happened to edit the file, or configure a mail server via the UX. It was a long time ago, I don't remember.
i needed to replace HOST with SMTP_ADDR, and add SMTP_PORT

which does it :

[mailer]
ENABLED   = true
SMTP_ADDR = in-v3.mailjet.com
SMTP_PORT = 587
FROM      = xxx@xxx.fr
USER      = xxx
PASSWD    = xxxx

now it work again.

please dont break gitea for a mail configuration :)

nunoperalta · 2022-12-30T10:49:35Z

Thanks.

Strange... I don't see any extra logs since I put LEVEL = trace :(

Server listening on :: port 22.
Server listening on 0.0.0.0 port 22.
2022/12/30 10:39:58 cmd/web.go:106:runWeb() [I] Starting Gitea on PID: 15
2022/12/30 10:39:58 cmd/web.go:160:runWeb() [I] Global init
2022/12/30 10:39:58 routers/init.go:116:GlobalInitInstalled() [I] Git Version: 2.36.3, Wire Protocol Version 2 Enabled (home: /data/gitea/home)
2022/12/30 10:39:58 routers/init.go:117:GlobalInitInstalled() [I] AppPath: /usr/local/bin/gitea
2022/12/30 10:39:58 routers/init.go:118:GlobalInitInstalled() [I] AppWorkPath: /app/gitea
2022/12/30 10:39:58 routers/init.go:119:GlobalInitInstalled() [I] Custom path: /data/gitea
2022/12/30 10:39:58 routers/init.go:120:GlobalInitInstalled() [I] Log path: /data/gitea/log
2022/12/30 10:39:58 routers/init.go:121:GlobalInitInstalled() [I] Configuration file: /data/gitea/conf/app.ini
2022/12/30 10:39:58 routers/init.go:122:GlobalInitInstalled() [I] Run Mode: Prod
Received signal 15; terminating.

nunoperalta · 2022-12-30T10:54:10Z

Ah, @rafipiccolo - seems it was what fixed it for me too!

I did look at the Breaking Changes notice, and somehow I did NOT understand what was said there 😅🤦🏼‍♂️

The combined mailer.HOST option has been deprecated in favor of the new mailer.SMTP_ADDR and mailer.SMTP_PORT options.

I guess it's my fault.
Would be nice if the logs said that, the same way they complained about LFS :)

rafipiccolo · 2022-12-30T11:02:42Z

my bad too, i'll read it next time,
But i didn't expect a breaking change on a minor release.

nunoperalta · 2022-12-30T11:04:36Z

Yeah exactly. Especially when it says "deprecated" and not "removed".

wolfogre · 2022-12-30T13:21:32Z

Strange... I don't see any extra logs since I put LEVEL = trace :(

Server listening on :: port 22.
Server listening on 0.0.0.0 port 22.
2022/12/30 10:39:58 cmd/web.go:106:runWeb() [I] Starting Gitea on PID: 15
2022/12/30 10:39:58 cmd/web.go:160:runWeb() [I] Global init
2022/12/30 10:39:58 routers/init.go:116:GlobalInitInstalled() [I] Git Version: 2.36.3, Wire Protocol Version 2 Enabled (home: /data/gitea/home)
2022/12/30 10:39:58 routers/init.go:117:GlobalInitInstalled() [I] AppPath: /usr/local/bin/gitea
2022/12/30 10:39:58 routers/init.go:118:GlobalInitInstalled() [I] AppWorkPath: /app/gitea
2022/12/30 10:39:58 routers/init.go:119:GlobalInitInstalled() [I] Custom path: /data/gitea
2022/12/30 10:39:58 routers/init.go:120:GlobalInitInstalled() [I] Log path: /data/gitea/log
2022/12/30 10:39:58 routers/init.go:121:GlobalInitInstalled() [I] Configuration file: /data/gitea/conf/app.ini
2022/12/30 10:39:58 routers/init.go:122:GlobalInitInstalled() [I] Run Mode: Prod
Received signal 15; terminating.

@nunoperalta

Let me guess, your Gitea config is:

[log]
MODE      = file

So most logs will output to the file, but you check logs from console with docker logs, so you can only see very few logs that have been printed to console at the very beginning.

If I'm right, you can find the error logs in the file. It's not your fault, I think Gitea can do better.

nunoperalta · 2022-12-30T13:32:04Z

@wolfogre - Perfect. Thank you!

I changed now to:

MODE      = console
LEVEL     = warn

It now shows also another deprecated config, that I also fixed :)
Cheers!

Even if the log mode is `file`, there are still few logs printed to the console at the very beginning. That's fine but confusing. Someone will think the console is the only place to find logs, and get nothing helpful. See #22274 (comment). There should be a reminder that there are no more logs to the console. And to avoid log loss, we should add configured loggers first, then remove console logger if there's no `console` in the mode. Tests with `MODE = file`: Before: <img width="1792" alt="image" src="https://user-images.githubusercontent.com/9418365/210079862-d591677f-347e-46ed-a548-bb2ddbb0885c.png"> After: <img width="1792" alt="image" src="https://user-images.githubusercontent.com/9418365/210080002-d66cc418-6888-4909-b370-d03f5986ef41.png"> Co-authored-by: Lunny Xiao <xiaolunwen@gmail.com> Co-authored-by: delvh <dev.lh@web.de>

Fix #22274 This PR will allow `HOST` without port. Then a default port will be given in future steps.

Fix go-gitea#22274 This PR will allow `HOST` without port. Then a default port will be given in future steps.

Fix #22274 Backport #22280 This PR will allow `HOST` without port. Then a default port will be given in future steps.

nunoperalta added the type/bug label Dec 30, 2022

lunny mentioned this issue Dec 30, 2022

Allow HOST has no port #22280

Merged

wolfogre mentioned this issue Dec 30, 2022

Reminder for no more logs to console #22282

Merged

markshep mentioned this issue Dec 30, 2022

Unneccessary SMTP_ADDR lookup when using "PROTOCOL = sendmail" #22287

Closed

lunny added this to the 1.18.1 milestone Jan 9, 2023

wxiaoguang changed the title ~~Version 1.18 crashes for me (Docker)~~ Version 1.18 crashes for me due to mailer.HOST config Jan 11, 2023

zeripath closed this as completed in #22280 Jan 11, 2023

zeripath pushed a commit that referenced this issue Jan 11, 2023

Allow HOST has no port (#22280)

2220e5d

Fix #22274 This PR will allow `HOST` without port. Then a default port will be given in future steps.

lunny added a commit to lunny/gitea that referenced this issue Jan 12, 2023

Allow HOST has no port (go-gitea#22280)

990a8c4

Fix go-gitea#22274 This PR will allow `HOST` without port. Then a default port will be given in future steps.

lunny mentioned this issue Jan 12, 2023

Allow HOST has no port (#22280) #22409

Merged

lunny added a commit that referenced this issue Jan 12, 2023

Allow HOST has no port (#22280) (#22409)

426c0ad

Fix #22274 Backport #22280 This PR will allow `HOST` without port. Then a default port will be given in future steps.

go-gitea locked and limited conversation to collaborators May 3, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Version 1.18 crashes for me due to mailer.HOST config #22274

Version 1.18 crashes for me due to mailer.HOST config #22274

nunoperalta commented Dec 30, 2022

lunny commented Dec 30, 2022

rafipiccolo commented Dec 30, 2022

techknowlogick commented Dec 30, 2022

rafipiccolo commented Dec 30, 2022 •

edited

Loading

nunoperalta commented Dec 30, 2022

wolfogre commented Dec 30, 2022 •

edited

Loading

rafipiccolo commented Dec 30, 2022

rafipiccolo commented Dec 30, 2022 •

edited

Loading

nunoperalta commented Dec 30, 2022 •

edited

Loading

nunoperalta commented Dec 30, 2022 •

edited

Loading

rafipiccolo commented Dec 30, 2022

nunoperalta commented Dec 30, 2022

wolfogre commented Dec 30, 2022 •

edited

Loading

nunoperalta commented Dec 30, 2022

Version 1.18 crashes for me due to mailer.HOST config #22274

Version 1.18 crashes for me due to mailer.HOST config #22274

Comments

nunoperalta commented Dec 30, 2022

Description

Gitea Version

Can you reproduce the bug on the Gitea demo site?

Log Gist

Screenshots

Git Version

Operating System

How are you running Gitea?

Database

lunny commented Dec 30, 2022

rafipiccolo commented Dec 30, 2022

techknowlogick commented Dec 30, 2022

rafipiccolo commented Dec 30, 2022 • edited Loading

nunoperalta commented Dec 30, 2022

wolfogre commented Dec 30, 2022 • edited Loading

rafipiccolo commented Dec 30, 2022

rafipiccolo commented Dec 30, 2022 • edited Loading

nunoperalta commented Dec 30, 2022 • edited Loading

nunoperalta commented Dec 30, 2022 • edited Loading

rafipiccolo commented Dec 30, 2022

nunoperalta commented Dec 30, 2022

wolfogre commented Dec 30, 2022 • edited Loading

nunoperalta commented Dec 30, 2022

rafipiccolo commented Dec 30, 2022 •

edited

Loading

wolfogre commented Dec 30, 2022 •

edited

Loading

rafipiccolo commented Dec 30, 2022 •

edited

Loading

nunoperalta commented Dec 30, 2022 •

edited

Loading

nunoperalta commented Dec 30, 2022 •

edited

Loading

wolfogre commented Dec 30, 2022 •

edited

Loading