News

migration complete

DNS records have been updated, and Banjo is the new employees.org server.

If there are any problems, send email to admin@employees.org

mailing lists migrated

All 63 mailing lists that owners said were active have been migrated to the new server. The only change change will be the hostname in the administrative URL. The list's email addresses will remain the same before, during, and after the migration. When we update the DNS records for www to point to Banjo (estimated in September), the hostname for the administrative URL will again be www.employees.org.

Accounts with bad passwords have been locked

We recently cracked all of our passwords against the recently-disclosed password list described in the New York Times (and other places).

We have locked all of the accounts that were found to have password contained on that list. If you are having trouble logging in, your account may have been one of the ones that was reset, and you should have received an e-mail about it.

Unplanned outage - 2010-Jan-16

We are still investigating the cause of the Saturday morning outage. At this time, the only thing we know is that the system had to be hard-reset to bring it back up as it would not respond to the console.

Also, perl was unintentionally upgraded while trying to fix a problem with the zope/plone server. If you have any problems with any perl modules, please let the admins know.

Willers New Year's DOS attack

Willers was hit from out-of-control user app eating all of swap and a 60Mbps DoS attach. As of Jan 5th, things should be back to normal. Bookmark this URL for offline status info:

http://status.employees.org

Willers Downtime 7PM PDT Oct 7th, 2009

unplanned outage, 22 Sep

On 22-Sep, a UPS failure at 5:58am PDT at our colocation facility caused all systems to lose power. Power was restored at 6:42am. However, willers.employees.org did not reboot successfully due to a corrupted CMOS. It was manually brought up at 3:30pm, disks fsck'd, and is now operating normally.

We suspect the CMOS was corrupted due to a dead CMOS battery, which we plan to replace soon.

Planned outage - July 31st, 2009 11pm PDT

Willers will be down for file system maintenance at 11PM PDT, July 21st, 2009.
This may take a few hours.

planned June 26, 2009 outage

Employees.Org will be powered down the afternoon and evening of Friday, June 26, 2009. During this time the cabinet will be re-cabled and new hardware will be installed in the replacement system -- which will finally allow us to begin migrating to faster system with mirrored disks.

July 14-15 outage

At about 5pm PT on Monday, July 14th, Employees.Org encountered errors on its system disk that caused the system to wedge. We rebooted the system at 11:30am PT on Tuesday, July 15th.

Normally, we would communicate outage information by posting to http://status.employees.org -- however, that system was also down for unrelated reasons.

We apologize for the outage.

We have new hardware which we will be bringing live within the next several months.

Syndicate content