Re:Re:Re:Major Outage

Bigjimny has suffered a major outage in the last two days.

I have work through a lot of scenarios and found nothing (I know what is stopping working but I dont know why).

I have turned on a lot of tracing diagnostics which may make it slow (and of course it could crash at any time)

Martin

As ever thank you for your efforts

Well done and thank you.

Hope you manage to find a good resolution.

Nice work Martin, this forum is an amazing resource, cheers from oz

Sent from my SM-A336E using Tapatalk

Hi Martin,

We really appreciate all of the hard work you put in, hope it gets fixed soon and you can go back to looking forwards to Xmas.

I have no direct experience of this website platform, but do have experience of other related systems, not sure if I could help? There might be other IT professionals on here with a lot more experience.

Anyway the message is don't be afraid to ask for help, I am quite happy if you want to contact me if you need to pass anything by me?

In my experience things like this that were working just fine and sudenly stop are nearly always as a result of upgrades somewhere, sometimes in the core offering.. Usually others experience the same problem so the forum is full of postings.

Robin

Robin.

Its the mysql database server (in this case, MariaDb). It loses all knowledge of the attached databases roughly 15 minutes after booting. It loses them to the extent that it will not recover the connections unless the entire drive volume is restored from backup.

I have turned on SQL logging in the hope of trapping whats going on as the only thing in the syslog is that MariaDb is commanded to shutdown. However, inevitably, since turning on logging it has behaved just fine.

I don't have any experience with MariaDb, but do have some experience of MySQL. My last problem was that as MySQL filled up queries would take so long to run that they would time out. This left errors in the tables and eventually I had to rebuild the tables to get rid of the accumulated errors.

The only solution we found was to archive much of the database off, year old data wasn't of much use in any case.

I did a quick search for MariaDb shut down after 15 minutes, there are a few entries on Serverfault, one person found a cron job which was shutting MariaDb down, another found that he had previously used MySQL and there were settings in that which conflicted with MariaDb.

I don't suppose the syslog tells you which process shut down MariaDb? If it isn't by name, it might be by PID and you could look up the processes with ps.

I don't suppose you have a DB tidy up/clean running as a cron job do you?

Hope that helps

Robin

Robin,

Thanks, the tables are all clean with no errors

I have also looked at Cron to see if something was shutting it down but couldn't see anything.

It has been happily running MariaDb for years now. In previous years MariaDb had run out of memory so I increased the VM memory as the first attempt fix. The large database sizes seem to be no issue if it has the memory.

However it's only today that I have turned on logging ( it eats disk space) so it's a matter of waiting to catch it doing it, but now it's suddenly stable

Martin

Sent from my moto g42 using Tapatalk

Typical, it won't fail when you want it to. It will probably go down Xmas or New Year's Eve just to be awkward.

Robin

wrote:

Typical, it won't fail when you want it to. It will probably go down Xmas or New Year's Eve just to be awkward.

Robin

I am sure it will. (If nothing else the log files will fill the disk and bring it to a halt!)

Sods law.

I removed the diagnostic logging yesterday. Today it crashed.......