SEO,a taste of why our site redesign is a big job

mclemore

Administrator
Staff member

Donor 15 years: 2010-2024
Joined
Apr 3, 2001
Messages
5,360
Reaction score
1,940
Location
Pasadena, California
A taste of why our site redesign is a big job...

We migrated a forum we support (https://forums.lymphoma.com) from this system to our new system. And of course one thing led to another. Besides work on how the main site communicates with the forums, other issues came up...

SEO, and Forum Traffic.

When we migrated from our current system to our new system, we instantly lost 88% of page views in Google, 83% of our traffic from Google, and 60% of our page coverage in Google. So Google showed our forums a whole lot less. As a minor consolation, the click through rate on organic traffic went up, as did our average impression on the page.

Now we upgraded the forums at the exact same time that Google changed their systems, so multiple things were going on. Even though we had proper redirects set-up to new URLs, google still dropped a lot of our pages. Now our new forums were much faster (which Google likes) and mobile friendly unlike the old ones (which Google also likes), though neither way enough to 'save us'.

After a lot of research and looking, it's become clear that Google has been downplaying (a) forums the last two years, and (b) especially content they think is old regardless of whether it is on a forum system or not.

So let's look at the forums here, at forums.arcade-museum.com, which we have not been upgraded yet. In the last 30 days, Google has decided to include an extra 2,000 pages from the main site (yeah)--the 30 days before that were flat for us. However, in the last 60 days, Google has dropped 60,000 pages from these forums from our index. It's been a dramatic downward trend for quite a while now. Google used to include all our forum pages. Now, they include only about 5% of them. So the forums are getting slaughtered by Google regardless of whether we changes systems or URLs.

Due to the loss of traffic, we have been exploring with what it takes to make Google happy. Since page speed affects both user experience and Google, we've experimented with making our pages as fast as possible. For example, for https://www.ace.com/, we've got the page to load really fast. Google's PageSpeed Insights tool consistently gives it a 96-99 (usually 97-98) out of 100 on mobile and perfect 100 on desktop. However, we've learned that these PageSpeed scores don't correlate well to Google 'Core Web Vitals' scores, especially if Core Web Vitals traffic is coming from international users.

BTW... to answer the question... making the site mobile friendly and faster isn't helping keep the forums from being slaughtered. However, once Google reached equilibrium, it doesn't seem to be deleting more pages now (nor adding any).

For people interested in web sites and SEO, it's worth noting that we are using a CDN (content delivery network) for the main arcade web site but not for many other pages and not for other sites (ie: forums.lymphoma.com or ace.com). There are reasons for each case. Beyond our basic, CDNs are great for web sites that don't have their servers tuned or are underpowered. For web sites that are tuned, CDNs provide a plus for static content (images, video), but provide a serious negative for dynamically generated web pages. The answer (which used to have more negatives than it may now) may be to separate static content (ie: images & video) onto a sub-domain. We are going to pause working on this site for a few days and maybe experiment more with CDNs and speed for international visitors on another (already rebuilt) site.

To make a long story... well... no longer that it already is... we are going to continue to migrate to the new system, and soon. And we will continue to focus on site speed and user experience. It's a long and hard process though.

Back to Google... it is clear that each site has a certain crawl budget from Google. We are also going to try and get Google to focus on areas for which it considers us an authority (ie: arcade games), and not so much on other areas. Previously, we moved the 'Chat About Almost Anything' section private, keeping it away from Google and non-logged in users. I'm now going to do the same thing for the Jokes, Laughs & Video section. They are great sections for our members, but not areas of strength for us in Google's eyes.
 
Good stuff. Thanks for the update. Sounds like you guys are working hard.

One question: Will there be a '3-day warning', or something similar, before the forums go down, for the official transition? This would be very helpful for those of us who run businesses here, and rely on this forum for communication, payments, etc. Especially if the upgrade is going to be near the end of the month. (Personally I'd prefer it was after the first of the month, if there's any choice in the matter, FWIW).

Also, I don't know what the state of things is, but is there a single thread we should be subscribed to, to make sure we see all upgrade-related notices? I know there are multiple about the upgrade, and I don't want to miss anything.
 
This is something you really want to focus on and take your time with. make sure the redirects are sending the proper status codes, canonical urls, meta info etc. You should take a hit with google but it should bounce back. Keep in mind you are dealing with separate things here. One being the forum change and the other being different SEO formats not just involving what search engines would consider repeated/duplicate content. Webmasters fail to realize that when you migrate to new software you are forcing search engines not only to spider and recognize what it would consider repeat content but also new tags, key words, markup language etc.

I helped migrate a forum some years back. The site was forum+portal addon. I migrated the website to invision forums which at the time was king (xenforo had just released their first version and Kier Darby was immediately sued by Vbulletin - nobody thought they would survive the lawsuit).

Invision's SEO at the time was controlled by an official dedicated plugin that would "optimize" your forum. We discovered their IPBSEO was a complete mess and after repeated attempts to get them to fix these issues, they actually banned us from their website and even the owner slagged us off on his twitter account - to Kier Darby of Xenforo no less.

We noticed that there were some issues with their SEO generating proper canonical urls which made a massive mess of things. A few months passed by and they had now wrapped their IPBSEO plugin into their forum. No plugin required. Immediately multiple webmasters were complaining their their search rank tanked and their organic search traffic and adsense revenue had fallen upwards of 2/3. A few months after that they quietly admitted their mistake in a new version changelog but not after they permanently ruined multiple forums.

even if you jump Gung ho into this migration, you want to keep an eye on all of this.
 
Updating websites is a pain to begin with - then add if you're trying to keep traffic routed towards it via Google, etc. That (as you've mentioned) is a completely new pain.

I had a site I upgraded and made a change that dropped multiple peoples' at work off first page of searches. They pitched a hissy fit (these are research-type pages). Google's answer (15 years ago) was "that'll straighten out in 30-45 days". The users, of course, wouldn't accept that answer (they did, btw, reappear at the top).

What did they want me to do, lay siege to Google and modify their algorithm?!?

Thank goodness I am no longer in the content-management service anymore.
 
This is all great info and I assume the primary reason that google traffic is so important is the ad revenue to fund the site, so that's worth mentioning (since we probably don't want to pay a subscription and/or lose this forum.

When you mentioned that only 5% of pages are getting indexed, I wondered if we should do anything (when posting?) to either tag or improve the chances of certain threads/posts of getting indexed. I know we have a ton of posts about a popeye still being for sale that belong in the 95%, but should we try to game the system to use our 5% wisely and/or convince their crawlers to index more?

I assume the canonical threads about FAQ's get linked back to often enough over time that they continue to be indexed (and Google doesn't think they're "old"), but should we do something else?

And then an ethical question: is it wrong to try to get all of the zaxxon sucks threads indexed....:001_stongue:
 
Back
Top Bottom