Show newer

Emergency reboot on Longhorn to update server. (Emergency as it relates to immediate performance and reliability)

Longhorn server is presently unresponsive. Working upstream to resolve.

Acadia upgraded on memory to prevent repeat event.

Show thread

Rebooting Acadia, OOM. Wasn't expecting that considering how few tenants are on the box.

We tried to alert Microsoft to a huge portion of their Hotmail IPs having no reverse DNS:

They asked us to open a ticket. Couldn't make this stuff up.

Large portions of Microsoft's range, if not all of it, seem to be lacking valid reverse DNS records. Can't reach anyone at Microsoft who can use this, but expect delivery issues with them due to failure to maintain their infrastructure.

QuadraNet is having a significant spam problem, moving around to new IPs constantly. We just blacklisted and will review individual removal requests.

Today we experienced an issue with outbound mail for roughly an hour. The details are too extensive to go into here, but have been posted in our channel at

This issue doesn't seem to be what it initially appeared to be. As we continue to work on it, this seems to have occurred when multiple multi domain certs were made. For example, selected Domain1 and made a cert for all domains, selected Domain2 and made a cert for all domains, selected Domain3 and made a cert for all domains. Now all 3 domains have 3 Dovecot SNI certs in conf files for each domain.

Show thread

We believe we may have identified an issue with Dovecot's handling of multi-domain SSL certs. If you have one cert signed at once for more than one of your domains (for custom mail subdomain, for IMAP/POP/SMTP), Dovecot may continue to serve an expired certificate even after successful renewal. Investigating.

Planned maintenance on Eagle right now. Any inbound mail sent to the server during this time will be received when it comes back online.

“Gmail spam filters are better” some say. Mostly true but…

Banshee server is set for emergency migration as soon as possible. We're looking at hardware failure but impossible to tell how much time we have. As long as it's functioning, we're going to keep it in production and aim for a seamless migration. This is an older cPanel box, not equal to our current hardware standards. It will be post-migration.

Rebooting Banshee again to satisfy vendor in troubleshooting KVM connectivity.

False alarm on Banshee outage. Looks like it got bumped while plugging in KVM.

We've kicked into overdrive an effort to fight back against botnets that attack our customers. The number of IPs blocked that have no connection to customers or anything they could want is rising every second. Zero false positives, this is very surgical work.

We've limited exim connections to the Pixel server to 30 simultaneous connections per IP, to mitigate a DDOS attack from Amazon SES.

Show older

Legal content allowed, don't be a dick.