Anyone else running into these problems? I have been encountering all of this since late Sunday night. Pages taking minutes or longer to even load. Really painful when trying to edit campaigns. In other cases, it just simply craps out the 502 Bad Gateway error for the site. Over the last 2 or so days, there have been maybe 20 minute blocks where site seems running just fine. Then back to slowness and not loading.
It looks like you're new here. If you want to get involved, click one of these buttons!
Comments
Came here to say the same
Hi guys! @Lordmisha @EugeneGM, I went ahead and reported this to my team - we're so sorry you're experiencing this but we're on to find a solution for you ASAP!
Obsidian Portal Support Druid | [email protected]
Let us know if you're still seeing issues, or saw them after 3am CST on May 28th.
One of the members has a very ambitious Campaign, and the number of links that needed processing on Save was locking up the database. The Campaign has just reached the threshold where it was too much for the existing code to handle. We made some changes to the code to process that better, so it shouldn't be an issue anymore.
Obsidian Portal Developer
Well, that seemd to fix things for a bit. But as of late last night and today (Saturday) the problems are starting back up. Not quite as often but often enough to be frustrating when trying to edit a campaign.
The slowness and timing out errors is still ongoing, a bit less frequently then original complaint, but still none-the-less.
Our sincerest apologies @LordMisha! I'll let the team know and we'll get back with you, thank you for your patience.
Obsidian Portal Support Druid | [email protected]
@LordMisha, thank you for letting us know!
We made some further changes over the weekend, and a few more tweaks this week. It's looking good at this point from this side ... we haven't seen any errors that would indicate slowness.
If you see any issues after today, please do let us know again!
Obsidian Portal Developer
Hello, I had the same issues for the past two weeks, and it's still ongoing as of today ; it takes a few minutes to load a page, and it ends up most of the time with a 502 error.
I'm so sorry to hear this @polandais, we apologize for the late reply! I'll have our team look into this.
Obsidian Portal Support Druid | [email protected]
Howdy @PoLaNdAiS!
We sent you a personal message to follow up on this bug, please give your email inbox a check!
Obsidian Portal Support Druid | [email protected]
Okay, so this problem is back for me. I have been encountering it since late Saturday night. Not trying to throw shade at anyone but this has become a semi-re-occuring problem that is interfering with what little free time I have trying to develop and edit campaigns.
Any chance this can be addressed? Or if it is similar to the previous problem, how come it doesn't seem to be addressed until someone points it out on the forums.
In fact, my page I was working on is still on a 502 error.
This is happening for me right now making it tough to work on my campaign. Judging on when OB has answered my emails in the past I guess they're on US west coast time or so. Don't expect a quick resolution if you're on a European time zone.
How the Hell did my name become dsadas!?
Hey friends! @LordMisha @dsadas Our sincerest apologies for the delay in response, and yes we are on US time, some of us are on West coast, and others on East coast.
Our team was notified about this, and we did see a spike in bot activity that would have caused this issue - we did address previously when it was brought to us a few days ago, are you still having the same issues now? @LordMisha I'll send this over to my team so we can keep an eye on things!
Obsidian Portal Support Druid | [email protected]
Any chance this can be addressed? Or if it is similar to the previous problem, how come it doesn't seem to be addressed until someone points it out on the forums.
@LordMisha, very valid questions.
502 errors don't show up on our side in the error logs, and they don't show up for everyone at the same time. It basically means that the server is being overwhelmed and is probably serving someone else's request instead of yours, whereas when things are working fine, everyone's requests are getting served without issue.
The problem back in June was a process that was taking too long when a page with a lot of links was Saved. When that was happening, the server was getting overwhelmed trying to finish the process, and so some members were seeing 502 errors, but when the process eventually failed, it would also throw a 500 error, which does show up on our side in the error logs. So, in June, we could see the 500 error and were able to track it back to the long running process. And we fixed the code so that process was handled differently and won't overwhelm the server.
For this current 502 issue, there aren't any errors in the error logs, so we don't know about the 502 issues until a member reports they are having issues.
We initially thought this round of issues was due to bots scanning/scraping, so we turned on the "are you human" Cloudflare checking on the Campaigns for not-logged-in visitors. But that doesn't seem to have completely fixed the issue. We can still see odd spikes in traffic.
So, we're still working on what is causing the server overwhelm.
Obsidian Portal Developer
We've kicked off some bots that were aggressively crawling the site. Please continue to let us know if you are seeing 502 errors or just slowness.
Obsidian Portal Developer