502 bad gateway after software update

Hiya. I briefly took down one of my Mattermost instances to update the software, using the standard docker-compose down, git pull, docker-compose build, docker-compose up -d sequence, and everything appeared to work as expected. Except that on reloading the app I started getting nginx 502 bad gateway errors. I’ve attempted to restart both the load balancer and the server behind it, I’ve tried redeploying the load balancer, I’ve reissued the SSL cert (just in case), and nothing is helping. Any ideas about what else I should poke at?

Okay, next question: after a LOT of poking around, it appears that the update overwrote my config file with a default version, which would be a problem. Is there any chance of recovering a pre-update version?

Further update: I got in and edited the config.json file to include several of the settings that I knew had disappeared, including the SiteURL and SMTP info. Those settings require a server restart to take effect… and after restarting the settings were wiped again.

I am beginning to think that I’m going to need to take more drastic action, such as starting a new instance entirely and trying to move the database container from the old instance to the new one. Problem #1 with that plan is that I am not 100% sure that the SQL user/password aren’t among the config settings that have been wiped… and I’m not sure how I can recover them.

All of which is to say that I really need some help here. Any guidance would be hugely appreciated.

Hi Kathleen,

We do have backups we could pull from. It’s sounding like that might be a better approach. For a more immediate response I would recommend a support ticket as these forums are not monitored with the same regularity as our support ticketing system.

Sounds good – huge thanks, @timmmmyboy!