Board timeouts
0

20 posts in this topic

Architecht   
  • Administrator
  • 6,307 posts
  • Location: Chicago area
  • Kudos Thread
Board timeouts

Hi all,

Many have already seen these, but in case you haven't - we are currently seeing brief slow downs and timeouts on the boards on a somewhat regular basis. We filed a ticket with the board hosts on Wednesday 4/12 to resolve this when their frequency seemed to go up.

Initially we needed to isolate the problem to the board host, as they suspected at first that our CDN (network infrastructure that improves performance) was the problem. We didn't think it was, but we needed to prove that to them.

We also began tracking the outages by polling the boards with an automated tool once per minute in order to establish to the hosts the frequency / urgency of the situation. We're seeing something like 8 of these timeouts or slow downs per day right now lasting between 1 and 5 minutes.

Tech support at the board host asked us to uninstall a third party plugin, and we did some trouble shooting with other plugins to try to resolve the issue, but those efforts didn't resolve the problem.

We have done some diagnostics to try to isolate the problem and focus the troubleshooting efforts. They've been investigating database load, but we believe the problem to be at a higher level (at the web server or network level for those technically inclined, because even a TCP handshake fails).

At this point they have escalated the ticket to higher level resources on their end and are continuing to pursue it.

We're sorry for the inconvenience these slow downs are causing, and we're working to get this resolved as quickly as possible.

 

Share this post


Link to post
Share on other sites


Jeffro.   
  • Member
  • 50,020 posts
  • Location: Jerkfropolis
3 hours ago, Architecht said:

Initially we needed to isolate the problem to the board host, as they suspected at first that our CDN

Damnit! I knew it was the Canadians  

BowGIeCLedzY0QDXTxCP.png

Edited by Jerkfro

Share this post


Link to post
Share on other sites
Sqeggs   
  • Member: Seasoned Veteran
  • 19,658 posts

I'd say the frequency of slowdowns is approaching, if not surpassing, the worst days on the old boards.  I know there have been several times in the past week when I've given up after waiting a dog's age (translation:  30 seconds or so!) for a post or PM to go through.

Share this post


Link to post
Share on other sites
sacentaur   
  • Member: Seasoned Veteran
  • 13,310 posts
  • Location: The Human Zoo
1 hour ago, Sqeggs said:

I'd say the frequency of slowdowns is approaching, if not surpassing, the worst days on the old boards.  I know there have been several times in the past week when I've given up after waiting a dog's age (translation:  30 seconds or so!) for a post or PM to go through.

Agreed - just endured yet another slowdown, this login was a last attempt after repeated failures.

Hopefully this issue can be resolved sooner rather than later.

Share this post


Link to post
Share on other sites
Silver Surfer   
  • Member: Seasoned Veteran
  • 34,828 posts
On 4/18/2017 at 11:21 AM, Jerkfro said:

Damnit! I knew it was the Canadians  

BowGIeCLedzY0QDXTxCP.png

:acclaim:You guys south of the border blink slower so it doesn't effect you as much. 

Edited by Silver Surfer

Share this post


Link to post
Share on other sites
Sqeggs   
  • Member: Seasoned Veteran
  • 19,658 posts

Just got 504'd twice in succession. :pullhair:

Share this post


Link to post
Share on other sites
Architecht   
  • Administrator
  • 6,307 posts
  • Location: Chicago area
  • Kudos Thread

Invision has made an adjustment that they think will let certain queries run faster. I don't know if that's truly the source of the problem, and I haven't had enough time for monitoring to confirm/deny improvement, but I wanted to update everyone that progress is continuing.

I am continuing to run independent monitoring so that if this doesn't resolve things, we will know it.

 

Share this post


Link to post
Share on other sites
Ricksneatstuff   
On 4/18/2017 at 5:24 PM, Sqeggs said:

I'd say the frequency of slowdowns is approaching, if not surpassing, the worst days on the old boards.  I know there have been several times in the past week when I've given up after waiting a dog's age (translation:  30 seconds or so!) for a post or PM to go through.

+ 1. I think this is the first time I have ever even brought up one negative thing about the boards. Hopefully you guys can resolve this issue :wishluck:

Share this post


Link to post
Share on other sites
G.A.tor   
  • Member: Seasoned Veteran
  • 57,178 posts
  • Location: The Gator Nation

I've gotten 3 diff 504 errors in last hour, fyi

Share this post


Link to post
Share on other sites
Azkaban   
  • Member
  • 5,778 posts
  • Location: 8.5, are you SERIOUS
  • Kudos Thread
On ‎4‎/‎18‎/‎2017 at 4:24 PM, Sqeggs said:

I'd say the frequency of slowdowns is approaching, if not surpassing, the worst days on the old boards.  I know there have been several times in the past week when I've given up after waiting a dog's age (translation:  30 seconds or so!) for a post or PM to go through.

for me it's no comparison, never had problems like this on the old boards. Missed out on some sweet books this week in a thread :pullhair:

Share this post


Link to post
Share on other sites
BlowUpTheMoon   
  • Member: Seasoned Veteran
  • 33,126 posts
  • Location: Buffalo, NY

The Boards were inaccessible from 20:50 (UTC) until 21:04 (UTC) on 4/20.  I posted the ten screenshots in the "I hate the new boards" thread in Comics General.

error.png

 

error10.jpg

Share this post


Link to post
Share on other sites
Marwood & I   
15 hours ago, Marwood & I said:

 

All quiet on the pictures of time outs front.  Fingers crossed, the problem has gone away and we can all get back to posting high quality comic related observations free from the terrors of Billy Buffering ?

Share this post


Link to post
Share on other sites
Architecht   
  • Administrator
  • 6,307 posts
  • Location: Chicago area
  • Kudos Thread

Just an update for everyone.

It looks like we have turned the corner on this issue. We did have an outage on the morning of 4/22. This was actually not an outage due to the board software, but was at a different level of network infrastructure. Sorry for that inconvenience, but the good news is that the more intractable problem of regular board timeouts caused by database load seem to be behind us. Other than the known issue on 4/22, we've had pretty clear sailing according to the web monitors.

 

Share this post


Link to post
Share on other sites
divad   
  • Member: Seasoned Veteran
  • 38,377 posts
  • Location: bolinas lagoon

And here I thought this thread was about posters getting sent to the corner with a dunce cap . . . :)

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
0

  • Welcome! Sign up and jump in!

    Membership and use of the boards is FREE! The knowledge available on these boards about comic collecting is an amazing resource for less experienced collectors. It's also a great place to talk about expert comic collecting topics with other experienced collectors. We hope that you find what you came looking for here, but more than that, we hope you find your new comic collecting home.

    Don't let the friendly banter and close knit community put you off. You're only a few posts from being part of the gang!

    After you sign up, introduce yourself! We'd love to know more about your collecting.

  • Upcoming Events

    • 28 January 2018 03:00 PM Until 10:00 PM
      0  
      Orlando Toy and Comic Con is coming to a new location on Jan 28th 2018.

      Location:
      The Florida Hotel
      1500 Sand Lake Rd Orlando FL 32809 (in the Florida Mall)
       
      Featured guests include: Tony Moore, Nick Bradshaw, Ryan Stegman, Ryan Ottley, Billy Tucci, Frank Tieri and more.
       
      This is Florida's best 1 day show!
      www.orlandotoyandcomiccon.com
       
       
       
    • 28 January 2018 03:00 PM Until 09:00 PM
      0  
      On January 28th join us from 10-4 at the Hasbrouck Heights Holiday Inn, 283 Rt 17 South (directly on the highway).for the ONLY comicon in Bergen County, NJ! Access from Rts 3, 46, 80, the GSP and the NJ Tpk is quick and easy!
      Admission is only $3, or buy 5 show passes that can be used at either of our hotel shows for only $10 (Admission is still free for kids under 12 and non-collecting partners). Parking is on site, easy and free.
      We'll have tons of comics as always! Your favorite dealers will be there, including some that ONLY do this show… and some new guys!
      Our notorious no minimum bid auction is at noon.  
      More info is on our website at http://www.njcomicbookshows.com
      “Like us” on Facebook at NJ Comic Book Shows
      NJ’s ONLY monthly comicons since 1991!
      Seeya at the show!
      JP
      Other upcoming shows:
      Clifton 2/11
      Old Bridge 2/18
      Clifton 3/11