Upgrading AWS Ubuntu Server

MikeJones · 2020-10-27T22:31:48+00:00

I am running a Ubuntu 16.04.7 LTS EC2 Instance on AWS and I am having issues upgrading it. I have tried: sudo apt update sudo apt upgrade sudo reboot And...

Upgrading AWS Ubuntu Server

Abhi

Were you able to move smoothly from 16.04 to 18.04 ? I think the following link and answer by user535733 might be helpful. The person replied is a part of the Ubuntu team ( I think so )

https://askubuntu.com/questions/1263794/how-to-upgrade-from-ubuntu-16-04-lts-to-20-04-in-virtual-machine

MikeJones

Abhi nope, still stuck.

I asked for help on the official Ubuntu Forum and someone asked me to run this command:

apt-cache policy landscape-common:amd64
Which I did and gave them the output, but not sure what next step is yet.

Abhi

From what I can see you will have to upgrade to 18.04 first and then upgrade to 20.04. I assume that is what you have done so far. If no, kindly do update.

It would be easier if you can fresh install 20.04 and avoid all these troubles. That way you have an error-free install.

MikeJones

Abhi I have not upgraded to 18.04 yet. I think thats where my issue lies.

Abhi It would be easier if you can fresh install 20.04 and avoid all these troubles. That way you have an error-free install.

If I do that I have to reconfigure my entire nginx settings and somehow import all my flarum data into the new server.... I think that seems just as much of a hassle, no?

Abhi

I understand your trouble. From the error data you posted, the issues seems to be broken packages / unmet dependencies. The issue is at the point from where you are trying to updating. ie, something is already broken on your server and from there you are trying an upgrade. Is there any chance you have a snapshot a bit older than the point from which you are trying ? Yes, it would loose you a few forum posts. But maybe a day or two old, will is cost you too much. I would suggest give it a try. I really hoped that one of those guides might have pulled you out.

PS : Did you try sudo apt autoremove ( backup please ) ? What was the result ?

Abhi

There might be hope

https://www.rosehosting.com/blog/how-to-fix-broken-packages-on-ubuntu-16-04-and-debian-9/

http://www.iasptk.com/ubuntu-fix-broken-package-best-solution/

I have not tried these, if interested, take a look

MikeJones

So I think my issue with upgrading might be this:

Reinst Failed because of python3-zope.interface:amd64 MarkKeep python3-twisted [ amd64 ]

Any more thoughts on this?

phenomlab

MikeJones Mike, this seems like the artefact of a removed PPA relating to Certbot. Have you removed any repositories recently before attempting the upgrade ?

Abhi

Could be because of partially upgrade packages.

I haven't tried :

sudo apt-get install --fix-broken

remove all Python and install it afresh

sudo apt-get purge python* && sudo apt-get autoclean && sudo apt-get install python*

MikeJones

phenomlab I dont believe so but is there anyway to check/fix?

phenomlab

MikeJones What is the result of sudo apt install certbot python3-certbot-nginx

MikeJones

phenomlab

I got this:

certbot is already the newest version (0.31.0-1+ubuntu16.04.1+certbot+1).
python3-certbot-nginx is already the newest version (0.31.0-1+ubuntu16.04.1+cert               bot+1).
python3-certbot-nginx set to manually installed.
The following packages were automatically installed and are no longer required:
  javascript-common jsonlint libargon2-0 libjs-excanvas libvpx3 libzip5
  linux-aws-headers-4.4.0-1054 linux-aws-headers-4.4.0-1055
  linux-aws-headers-4.4.0-1057 linux-aws-headers-4.4.0-1060
  linux-aws-headers-4.4.0-1061 linux-aws-headers-4.4.0-1062
  linux-aws-headers-4.4.0-1065 linux-aws-headers-4.4.0-1066
  linux-aws-headers-4.4.0-1069 linux-aws-headers-4.4.0-1070
  linux-aws-headers-4.4.0-1072 linux-aws-headers-4.4.0-1074
  linux-aws-headers-4.4.0-1075 linux-aws-headers-4.4.0-1077
  linux-aws-headers-4.4.0-1079 linux-aws-headers-4.4.0-1083
  linux-aws-headers-4.4.0-1084 linux-aws-headers-4.4.0-1085
  linux-aws-headers-4.4.0-1087 linux-aws-headers-4.4.0-1088
  linux-aws-headers-4.4.0-1090 linux-aws-headers-4.4.0-1092
  linux-aws-headers-4.4.0-1094 mercurial mercurial-common php-cli-prompt
  php-composer-semver php-composer-spdx-licenses php-json-schema
  php-symfony-console php-symfony-filesystem php-symfony-finder
  php-symfony-process
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 40 not upgraded.
ubuntu@ip-172-31-93-25:~$

MikeJones

So good news, I figured it out! I had to uninstall python3 and python-certbot-nginx and then I was able to upgrade!

Unfortunately now that that problem is solved, I am onto the next one! NGINX doesnt seem to be running: https://askubuntu.com/questions/1294969/upgraded-from-ubuntu-version-16-to-20-and-nginx-is-not-working

phenomlab

MikeJones So good news, I figured it out! I had to uninstall python3 and python-certbot-nginx and then I was able to upgrade!

That was going to be my next suggestion 🙂 You beat me to it - that's what happens when you "snooze" (I wish) I suppose 🙂

In relation to NGINX not starting, what's the output of journalctl -xe and anything in journalctl | tail ?

This is a bit old, but may still be useful
https://www.digitalocean.com/community/questions/can-t-start-nginx-job-for-nginx-service-failed

MikeJones

phenomlab

For journalctl -xe:

`-- A start job for unit session-70.scope has finished successfully.

-- The job identifier is 3818.
Nov 24 23:50:02 ip-172-31-66-31 systemd[7236]: Reached target Main User Target.
-- Subject: A start job for unit UNIT has finished successfully
-- Defined-By: systemd

-- Support: http://www.ubuntu.com/support

-- A start job for unit UNIT has finished successfully.

-- The job identifier is 1.
Nov 24 23:50:02 ip-172-31-66-31 systemd[7236]: Startup finished in 52ms.
-- Subject: User manager start-up is now complete
-- Defined-By: systemd

-- Support: http://www.ubuntu.com/support

-- The user manager instance for user 1000 has been started. All services queued
-- for starting have been started. Note that other services might still be starting

-- up or be started at any later time.

-- Startup of the manager took 52538 microseconds.
Nov 24 23:50:21 ip-172-31-66-31 sshd[7278]: rexec line 16: Deprecated option UsePrivilegeSeparation
Nov 24 23:50:21 ip-172-31-66-31 sshd[7278]: rexec line 19: Deprecated option KeyRegenerationInterval
Nov 24 23:50:21 ip-172-31-66-31 sshd[7278]: rexec line 20: Deprecated option ServerKeyBits
Nov 24 23:50:21 ip-172-31-66-31 sshd[7278]: rexec line 31: Deprecated option RSAAuthentication
Nov 24 23:50:21 ip-172-31-66-31 sshd[7278]: rexec line 38: Deprecated option RhostsRSAAuthentication
Nov 24 23:50:21 ip-172-31-66-31 sshd[7278]: reprocess config line 31: Deprecated option RSAAuthenti>
Nov 24 23:50:21 ip-172-31-66-31 sshd[7278]: reprocess config line 38: Deprecated option RhostsRSAAu>
Nov 24 23:50:21 ip-172-31-66-31 sshd[7278]: Invalid user jenkins from 104.248.243.134 port 41200
Nov 24 23:50:21 ip-172-31-66-31 sshd[7278]: Received disconnect from 104.248.243.134 port 41200:11:>
Nov 24 23:50:21 ip-172-31-66-31 sshd[7278]: Disconnected from invalid user jenkins 104.248.243.134 >
lines 2040-2070/2070 (END)
`

and for journalctl | tail:

ubuntu@ip-172-31-66-31:~$ journalctl | tail Nov 24 23:50:21 ip-172-31-66-31 sshd[7278]: rexec line 16: Deprecated option UsePrivilegeSeparation Nov 24 23:50:21 ip-172-31-66-31 sshd[7278]: rexec line 19: Deprecated option KeyRegenerationInterval Nov 24 23:50:21 ip-172-31-66-31 sshd[7278]: rexec line 20: Deprecated option ServerKeyBits Nov 24 23:50:21 ip-172-31-66-31 sshd[7278]: rexec line 31: Deprecated option RSAAuthentication Nov 24 23:50:21 ip-172-31-66-31 sshd[7278]: rexec line 38: Deprecated option RhostsRSAAuthentication Nov 24 23:50:21 ip-172-31-66-31 sshd[7278]: reprocess config line 31: Deprecated option RSAAuthentication Nov 24 23:50:21 ip-172-31-66-31 sshd[7278]: reprocess config line 38: Deprecated option RhostsRSAAuthentication Nov 24 23:50:21 ip-172-31-66-31 sshd[7278]: Invalid user jenkins from 104.248.243.134 port 41200 Nov 24 23:50:21 ip-172-31-66-31 sshd[7278]: Received disconnect from 104.248.243.134 port 41200:11: Normal Shutdown, Thank you for playing [preauth] Nov 24 23:50:21 ip-172-31-66-31 sshd[7278]: Disconnected from invalid user jenkins 104.248.243.134 port 41200 [preauth] ubuntu@ip-172-31-66-31:~$

phenomlab

MikeJones not the greatest of help...
What's the output of systemctl status nginx.service ?

phenomlab

I've been bitten by this before also. Check your nginx.conf fine to ensure you haven't missed a semicolon at the end of a line or perhaps have an unclosed bracket.

MikeJones

phenomlab

`ubuntu@ip-172-31-66-31:~$ systemctl status nginx.service
● nginx.service - A high performance web server and a reverse proxy server
Loaded: loaded (/lib/systemd/system/nginx.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Tue 2020-11-24 15:32:07 UTC; 9h ago
Docs: man:nginx(8)
Process: 4734 ExecStartPre=/usr/sbin/nginx -t -q -g daemon on; master_process on; (code=exited,>

Nov 24 15:32:07 ip-172-31-66-31 systemd[1]: Starting A high performance web server and a reverse pr>
Nov 24 15:32:07 ip-172-31-66-31 nginx[4734]: nginx: [emerg] open() "/etc/nginx/sites-enabled/exampl>
Nov 24 15:32:07 ip-172-31-66-31 nginx[4734]: nginx: configuration file /etc/nginx/nginx.conf test f>
Nov 24 15:32:07 ip-172-31-66-31 systemd[1]: nginx.service: Control process exited, code=exited, sta>
Nov 24 15:32:07 ip-172-31-66-31 systemd[1]: nginx.service: Failed with result 'exit-code'.
Nov 24 15:32:07 ip-172-31-66-31 systemd[1]: Failed to start A high performance web server and a rev>
lines 1-12/12 (END)
`

I don think I am missing any semicolons in the nginx.conf files. I didn't edit them at all since I upgraded.

phenomlab

MikeJones can you provide your nginx.conf file ?

MikeJones

phenomlab

MikeJones And the content of include /etc/nginx/conf.d/*.conf;