Hi guys,

so recently we had a bit of an influx of users on beehaw.org, which lead to some support questions due to the site being unstable.

I looked at the nginx config and introduced rate limiting and we scaled up from 2→4 CPU cores and 4→8GB RAM. That alleviated these issues a bit, but we still have a load of ~2.0 with just a few hundred/thousand users online.

In my experience with web services this is not a lot of users. What I found out is that a magick process is getting OOM killed frequently, that pict-rs often calls exiftool which causes lots of load and that basically the server.js from Lemmy-UI is not fast enough dealing with requests from nginx.

How do I debug where the bottleneck is? Do you already have an idea on how to reduce the load? We’re still running 0.17.2, but upgraded from pict-rs 0.3.0 to 0.3.1. Is it safe to upgrade to pict-rs 0.3.3 or even 0.4.0? Can you configure pict-rs to not use exiftool? I wouldn’t want to renice it manually if there’s a good way to optimise requests/s.

pinging @dessalines@lemmy.ml @nutomic@lemmy.ml @makotech222@lemmy.ml @CannotSleep420@lemmygrad.ml