After the upgrade caused NIS issues, I’ve discovered some users are locked out of access to web and ftp services here. If you’re reading this you are probably okay but if you’ve got a weather station or something else automatically doing periodic logins to ftp, you may be blocked. If this is the case please initiate a support ticket with your IP address or e-mail support with your IP address and indicate that you can not reach the web and/or ftp server and I will manually remove the block for you.
We upgraded our main web server to Ubuntu 18.10 Cosmic Cuttlefish early this morning. I did not become aware until later this morning that the upgrade broke NIS which is how the system propagates information about users between machines. This has been corrected as of about 10:45 AM this morning the 22nd of October, 2018.
I managed to get everything working on new mail servers based upon Ubuntu 18.04 and for me at least spam filtering is working much better. Prior to the upgrade I was getting between 1/3rd and 1/4th of spam in my INBOX rather than spam box, now I am getting 1/25 spams in my INBOX the rest to the spam box and so far no false positives.
The new servers bring new versions of just about all of the spam control facilities, new versions of spamassassin, postgres, dkim, spf milter, clamav, etc. It’s still older version of postfix, procmail, and smartlist because I can not get the newer versions to play together well owing to the fact that they each want to operate in their own chroot jail which unfortunately doesn’t give them access to each other which they need to function. I think eventually I will be able to get the newer version of postfix working but the new procmail is too broken.
The shell server julinux.yellow-snow.net is now back in operation. After going down a lot of rabbit holes, I thought to ask systemd the status of nis, and low and behold it was disabled. I re-enabled it and now all is well. So when you do an upgrade of ubuntu or a Ubuntu-derived distribution that is something to lookout for.
Julinux kicked out a new release based upon Ubuntu-Mate 18.10, but this is about three months premature and predictably it broke things, most notably NIS. Without NIS users can not authenticate on this machine. So for now that machine is down. I’ll try to troubleshoot NIS and if I can’t fix it revert to the 18.04 version.
I have made several changes to mail server configurations and DNS that should reduce spam and viruses.
- No mail will be accepted to email@example.com via any of the mail servers.
- SPF records are now published for mail.eskimo.com. This should prevent forgeries since SPF records are checked on all of our servers.
- The issue which prevented the clamav anti-virus database from being updated has been repaired.
- Fail2ban was not starting on mail allowing that machine to be used for brute force password guessing attacks. This has been fixed.
I have restored mail.eskimo.com from backups, re-applied all of the updates between the time it was backed up and current, and still it continues to operate. I do not know what broke the init system but it is presently all current again and yet running okay. I will be taking the system down again about 20 minutes around midnight to make a backup with all the updates in place.
The client mail server is up but I will need to reboot after bringing it current again with updates to make sure they didn’t break it, and if one of them did, a few more to determine which one. I suspect an issue with clamav as it was the last thing to start before init explodes.
The client mail server is currently down being restored from backup. For reasons I am unable to identify, the init start up script fails to parse the entire /etc.rc.d/rc5.d directory stopping just before postfix even though if I run the scripts they run fine.
I am restoring the server from backups which I hope will resolve the problem. Expected downtime is about 20 minutes. Mail should be functional by 8:20PM Pacific Time.
Something broke the init system of mail.eskimo.com such that necessary things are not being started on the client mail server when it boots. No cron jobs ran for some period of time and this caused the anti-virus database not to get updated. This would have made it possible for a virus to propagate from one infected customers computer to another. Mail may be interrupted for short intervals as I will need to perform a number of reboots to test and get the start-up logic straightened out.