Introduction
Towards the end of August or beginning of September, The CerebralRift went KABOOM! Some of you may have noticed a sudden “dump” of old articles to Twitter, Facebook and other social networks. I set off a small bomb in the process of gathering up the shards of a horrendously broken website, and didn’t realize the resulting updates were shared. And boy howdy! I made a lot of updates.
(Okay, “shards of a horrendously broken website” is a bit hyperbolic, yet the problems were serious.. Indeed there were a series of small and large explosions happening on the site.)
So, what happened? What broke? Why did it explode? In this series of articles I’ll walk through what happened, how I fixed it, and why it was necessary.
This is only the first article. There are a few topics that came up during this process that I want to either discuss or rant about. I also have new plans for bringing this site back with a new and (I think, at least) very exciting idea. Those topics will be covered in separate posts throughout this month.
Lighting the fuse… KABOOM?
The situation came about because of changes I made in early 2018. I moved CerebralAudio (the netlabel I started in 2015) to it’s own domain. Before relocating CerebralAudio, I started pruning back the old articles by removing:
- The vaping articles
- Cryptocurrency articles
- CerebralMix Podcast posts
The reason? The vaping and cryptocurrency content was an experiment that didn’t play well or fit with the primary focus of The CerebralRift. While they did gain some traction, they didn’t improve the viewer-ship or audience overall. So, I decided that I would move it off to another website: Vapes-n-Coins. (It’s just an archive for the old content. I don’t have any plans to revive my interest in those topics.)
As for CerebralMix… I wanted to leave all that content on line. I really did. However the podcasting plug-in has a nasty habit of breaking. In the best case, the shows would just not be available. In the worst case it would take down the whole site. So, the last time it went KABOOM, I decided the plug-in had to go. When the podcasting plug-in went away, the content became inaccessible. (It will come back eventually, but I will save that topic for another article.)
Bomb Detection
I mentioned moving CerebralAudio to its own domain. While I was doing that I installed a two tools to monitor the site:
- A redirection plug-in to handle the removed product pages.
- A link scanning tool to find any missed links from the removed articles, and moving CerebralAudio.
All of this was good, and the site was still up and running, if somewhat dormant since I hadn’t published any new articles (I started a bunch of them. but couldn’t find the time to finish them).
I thought these tools might uncover issues due to moving CerebralAudio. While they did uncover a few problems, they uncovered much larger issues. The issues that put CerebralRift on the path to going KABOOM! Boy, am I glad I was wrong about just finding a few issues.
Ch-ch-ch-ch-changes…
My hosting service introduced a new storage system, and told all customers we had to switch to it. I looked at migrating my data to the new system, and at first I thought it would mostly impact CerebralAudio… Then I remembered (a) the CerebralMix podcasts are on that storage, and (b) the backups for CerebralRift were on there as well. That meant I needed to take two actions: (1) update my backups, and (2) make certain there weren’t references to the old podcasts on here anymore.
Since it had been seven or eight months since CerebralAudio moved to its new home, I checked the status of the redirect plug-in. Some pages were still seeing a lot of traffic (in the hundreds of thousands of “hits”), and some were seeing moderate traffic (in the tens of thousands of hits), while the majority were seeing little traffic. But, the interesting part was all the hits were recent, within a few days.
Closer examination proved what I already thought: the indexing bots accounted for most of the traffic, and old tweets or links accounted for the rest. The breakdown was something like 93-94 percent were bots, 1-2 percent from old tweets and links. 3-4 percent I couldn’t determine (I suspect hackers and spammers). The redirects seemed to no longer be of use.
The stunning part came when I checked the broken link scanner: there were over 3000 links on the site, and 1300-1500 of them were broken. Those numbers seemed quite insane. When I had CerebralMix on the website, I could understand those numbers: every show had 20-30 links, which easily amounted to 2000-3000 links. But those posts were gone, not just unpublished, they were physically removed from the site. The remaining content (the reviews, news and editorials) wouldn’t have nearly that many links, and there was less than 500 articles. So what the heck was going on?
The broken links were the KABOOM! And they were affecting the website in more ways than I had anticipated.
An Aside About Website Performance
I’m sure by this point many of you are yawning… “This is a bunch of technical mumbo jumbo BS. Why are you writing this? Why should I care?”
Honestly, I expect quite a few people don’t care about any of this stuff. But I think it’s a community service to anyone who runs a website, and especially to those in the Creative Commons and Netlabel communities who haven’t kept track of their websites. (And I know, from experience, there are more than a few of you…) This is kind of like being an inventor, or writing software. Everyone wants to create the new and shiny. Almost no one wants to perform site maintenance: it’s boring, annoying and even painful. But there are reasons it really needs to be done:
- Broken content negatively affects user experience: no one likes looking at a website with broken images, videos, music players, and links.
- Broken content can affect your website performance.
- Search engines rank on both broken content and performance.
There are more reasons to care, and a lot of related topics to cover in this area. But that can be discussed another time, in a place where it is more appropriate.
And then there’s…
I knew by this point I needed to make a bigger plan for doing maintenance on the website. I continued auditing the site:
Plug-ins: I mentioned above that part of what started this whole thing was my backups plug-in. It is a nifty plug-in that never gave me any problems. But it had an annual cost to it, and I now had a free option available that is better. While looking at the other plug-ins, I realized I could remove a bunch of them. I also remembered starting to work on replacing one plug-in, but hadn’t finished the (rather large) job. Several plug-ins had been installed for specific use, but weren’t being used at all. And a couple plug-ins I suspected were hurting site performance. I determined all the work on plug-ins needed to be completed.
Images: after removing the boat load of old posts earlier in the year, I hadn’t audited the images. I found literally hundreds of images that were not needed any more. To further complicate things, I had used several plug-ins to generate web-friendly thumbnails and other versions of the images. This left a major, space consuming mess on my site. I suspected the bloat of also impacting the website’s performance, and therefore needed to be addressed.
Content: I found more content I needed to remove, mostly because it was not within the topic focus of the site (old, rambling, blog style posts). Some content I needed to move. A large pile of articles needed reformatting (to address several issues, including plug-in updates I determined needed completion). I also found pages that I hadn’t updated after moving CerebralAudio to its own site. I could see this was a monumental task.
Finally, I audited the appearance of the site. It really hadn’t been addressed since moving CerebralAudio. There were still menu items that were specifically for CerebralAudio that didn’t need to be part of the site anymore (they were linking to the new site, but they really didn’t fit what I wanted on this site now). I also realized that some things weren’t right given the changes that I had made (for example, social media links). And in general, the design didn’t fit the focus of the website. This isn’t just a visual thing, it’s a function to provide content and make it more accessible.
So with the audit completed I knew what I needed to do, and it was a monster of a task. And there was no chance of it being a simple, one pass, operation.
Making the changes
I started with the plug-ins first. Why? Primarily because of the backups. If I was going to be making a lot of changes to the website I wanted rock solid backups. The new backup system would deliver that with near real-time backups. And, the old backups plug-in license was due for renewal. Best to get rid of it now.
I ripped out around half of the plug-ins during the process. As noted some of them were not in use at all, and some impacted the site performance. Others were made redundant by the new plug-in I had installed to handle backups. The ones I left were in two categories: (a) served some useful function, or (b) removing them would break the content. Another pass would be needed after addressing other issues.
Next I updated the appearance of the site. But, this wasn’t just for the sake of changing its look. The site, it’s content and it’s structure were undergoing a re-focus process. The site is returning to a more periodical / magazine style. The content focus is going back to where it started (really never left), but with a new scope and mission in mind (more about this in a future article). So, updating the structure and navigation parts of the site was necessary, and removing some of the dependencies on the redirection plug-in (which was still installed at this point) was also needed.
Next came the hard, grueling work. It also lead to several discoveries, some of which I want to rant about…in the next article.
Defused?
At this point I was making a start on defusing the bomb that had gone off on The CerebralRift. No more KABOOM! But I still needed to address the content issues. And that is where I made a discovery that answered a big question that was still nagging at me in the back of my mind: where the hell were all those broken links? Get ready for a rant later in this series after I answer this question.
