So I'm about to roll out a new site design. I want to do a little bit of link validation before doing the switch over. So I started writting a basic crawler that'll request the page, report the status, and then check that page for links and follow them. Nothing super special. Well our fucking server admins are retards. They have a 404 handler page but it doesn't actually send the 404 header. Instead it sends the 200 OK header.