Search

Items tagged with: SysAdmin


Nach mehreren Tagen Messung möchte ich unsere Untersuchung zum ungewöhnlich
anwachsenden PHP-FPM-Pool abschließen.

Der wichtigste Befund ist inzwischen recht eindeutig:
Von 196.466 ausgewerteten Slowlog-Ereignissen enthielten 195.069 einen
curl_exec()-Aufruf, also rund 99,3 Prozent.

Zusätzlich konnten wir einen betroffenen PHP-FPM-Prozess während eines realen
Ereignisses mit ss und strace beobachten. Dabei wartete der Prozess über viele Sekunden wiederholt in poll() auf Daten eines externen HTTPS-Sockets. Über ss ließsich derselbe File-Descriptor derselben PID eindeutig einer externen Verbindung zuordnen. Gleichzeitig bestanden mehrere weitere PHP-FPM-Verbindungen zu denselben wenigen Gegenstellen.

MariaDB und Redis antworteten im selben Zeitraum dagegen sehr schnell. Der Engpass
lag also nicht primär im lokalen System, sondern im Warten auf externe HTTP-
Kommunikation.

Nach der schrittweisen Anpassung der Wartezeiten und der FPM-Reserve hat sich das
Verhalten deutlich beruhigt. Auch der Load Average ist wieder auf ein normales Niveau
zurückgegangen. Die Zahl der Slowlog-Ereignisse fiel ab dem 4. September massiv ab.

Der Slowlog ist deshalb inzwischen wieder deaktiviert.

Mein persönliches Fazit aus der Untersuchung geht inzwischen über die konkrete
Friendica-Instanz hinaus:

Eine föderierte Instanz muss nicht nur selbst stabil laufen. Sie muss auch so mit
langsamen, fehlerhaften oder zeitweise unerreichbaren Gegenstellen umgehen können,
dass deren Zustand die eigene Funktionsfähigkeit möglichst wenig beeinflusst.

Ich habe die Ergebnisse deshalb noch einmal ausführlicher in einem Fachartikel
zusammengefasst:

Föderation braucht Fehlertoleranz an den Instanzgrenzen

Hier der vollständige Artikel

Dabei verzichte ich bewusst auf konkrete Grenzwerte oder eine allgemeine „Best-Practice-
Konfiguration“. Die Werte hängen zu stark von Instanzgröße, Nutzung, Federation-
Volumen und Systemumgebung ab. Entscheidend ist aus meiner Sicht nicht das Kopieren
einzelner Parameter, sondern das Verständnis der Zusammenhänge und die Messung auf
der eigenen Instanz.

#Friendica #Fediverse #Sysadmin #ActivityPub #Federation


"AI-generated commands on your production RHEL systems" — sounds terrifying until you see the guardrails.

The MCP server for RHEL now has guarded command execution: a gatekeeper AI validates every command, write operations require human approval, and systemd-run sandboxes the blast radius.

redhat.com/en/blog/dynamic-tro…

#RHEL #MCP #AI #SysAdmin #Linux


There is a person I've been helping for a while who runs two instances, one #GoToSocial and one #snac.

Yesterday, they suffered severe file system corruption on their device, and I lent them a hand with the recovery. For GoToSocial, they restored the DB from the previous backup and got it running again. For snac, although a few files were lost, no action was necessary... it just started back up and did what it had to do, bringing the instance back to full operation.

This goes to show that when software is well-designed and intentionally kept simple, it's also easier to get back up and running when something goes wrong. In this case, snac's "files-only" approach demonstrated excellent resilience.

Thanks, @grunfink !

#snac2 #snac #ThankYouTuesday #Efficiency #IT #SysAdmin #OwnYourData


The media in this post is not displayed to visitors. To view it, please go to the original post.

hey, folks, i've been facing an issue with #FursonaDirectory that i'm not able to remedy with my current knowledge. i'm getting spammed every day with signups to the site and it's getting ridiculous. i'm going to have to manually clean up the database with some SQL queries again at this rate. how do people stop these from coming in on their own #LinkStack instances?

for reference, i have fail2ban and crowdsec setup but have no idea how to leverage them for this problem. very new to this. 😓

#LazyWeb #SysAdmin #SelfHosting


A few months ago, a client asked me to shave about €200 a month off our agreement. "We’re trying to cut back on expenses", he said, "to boost our cash flow." I was hesitant, but hey, I agreed.

Fast forward to this morning: one of his team members calls me asking for help because - and I quote - "both Claude and ChatGPT are giving me answers that just don't feel right."

I asked for a little context, only to find out they had upgraded to the top-tier plans for both AIs so they could "solve problems on their own without bothering me."

So, I dropped my client a line: we either revert to our original agreement, or we wrap up this collaboration effective immediately.

Asking me for a discount just to throw far more money at AI subscriptions - only to call me anyway when the bots fail - is, to say the least, a slap (or slop?) in the face.

#IT #SysAdmin


The media in this post is not displayed to visitors. To view it, please go to the original post.

#TechTipThursday

The best time to learn a new Linux tool is when nothing is on fire.
Pick one command you use often and go one level deeper this week.

Already use grep? Try grep -r, grep -v, grep -E.
Already use find? Add -exec to actually do something with the results.
Already use systemctl? Try systemctl list-dependencies.

You don't need a course. You need 20 minutes and a test machine. man is still the best documentation that exists.

#Linux #SysAdmin #LearningLinux #TechTip


The media in this post is not displayed to visitors. To view it, please go to the original post.

For my shop machine, I'm now actively testing Debian 14 Testing w/ Xlibre and SonicDE. So far, so good. I use this machine for client work and it also gets to be my guinea pig for testing drawing tablet workflows. Both native Xlibre (via wacom) and Gaomon's proprietary driver work. I'm impressed with how snappy SonicDE is - first time using it today. Great work folks‼️

@sonicdesktop @XLibreDev #xlibre #freesoftware #opensource #debian #sysadmin #floss #sonicde


A client asked for a server install for a specific CRM developed by one of Italy's biggest software houses. They’re dropping Windows Server 2022 support in a few months, even though the OS itself has a much longer lifecycle.

We looked into Linux support: Rocky Linux 9 and Ubuntu 24.04 are "certified", but only until April 2027. Since we'd rather not reinstall everything in less than a year, we asked for a path that guarantees official support beyond 2027.

The "support" team replied with a canned response, attaching a 2023 document where every single distribution is listed as EoL since 2025. 🤡

And then people ask me why these "software giants" are the primary cause of my receding hairline...

#SysAdmin #TechLife #EnterpriseSoftware #ITProblems #CRM #IT #OwnYourData