Upgrading Openstack using Packstack

I wanted to test Juno in more real world environment, since i'm not that reckless, i decided to experiment with my own little home server so i just put my own data at risk.
This headless desktop computer lives in a closet and runs Fedora 20 with OpenStack Havana. I didn't want to reinstall from scratch so i thought that upgrading to Juno would be a nice experiment (Kids, do try this at home and not in a datacenter with your clients important data).
First of all upgrade the system

# yum upgrade -y

Once it finishes install the latest Juno RDO repo.

# yum install -y https://repos.fedorapeople.org/repos/openstack/openstack-juno/rdo-release-juno-1.noarch.rpm

This will add the rdo-release.repo file to your /etc/yum.repos.d directory.
In this step i just could have run yum upgrade again and the latest openstack packages would have installed but there are some configurations that packstack performs that the havana version didn't do (or needed) so i rather do it this way. Besides i wanted to test how packstack will behave in this use case.

Since i had the Havana packstack version installed i upgraded to the Juno version.

# yum install -y openstack-packstack

This upgrades packstack as well as it's dependencies: openstack-packstack-puppet and openstack-puppet-modules.
Once it finished i ran packstack using my old answer file:

# packstack  -d --answer-file=packstack-answers-20130821-004155.txt

I wasn't expecting this to work at first because my answer file is more than a year old, but it did until it got to the mysql plugin:

ERROR : Error appeared during Puppet run: 192.168.1.68_mysql.pp
Error: Execution of '/usr/bin/yum -d 0 -e 0 -y install mariadb-galera-server' returned 1: Error: mariadb-galera-server conflicts with 1:mariadb-server-5.5.39-1.fc20.x86_64
You will find full trace in log /var/tmp/packstack/20141029-174821-1lLOIx/manifests/192.168.1.68_mysql.pp.log

This is because the mariadb-server was previously installed by packstack Havana, i just uninstalled it and ran packstack again:

[root@cloud ~(keystone_admin)]# systemctl stop mariadb
[root@cloud ~(keystone_admin)]# systemctl disable mariadb
rm '/etc/systemd/system/multi-user.target.wants/mariadb.service'
rm '/etc/systemd/system/multi-user.target.wants/mysqld.service'
# yum erase -y mariadb-server
packstack  -d --answer-file=packstack-answers-20130821-004155.txt
packstack  -d --answer-file=packstack-answers-20130821-004155.txt
Welcome to Installer setup utility

Installing:
Clean Up                                             [ DONE ]
Setting up ssh keys                                  [ DONE ]
Discovering hosts' details                           [ DONE ]
Adding pre install manifest entries                  [ DONE ]
Preparing servers                                    [ DONE ]
Adding AMQP manifest entries                         [ DONE ]
Adding MySQL manifest entries                        [ DONE ]
Adding Keystone manifest entries                     [ DONE ]
Adding Swift Keystone manifest entries               [ DONE ]
Adding Swift builder manifest entries                [ DONE ]
Adding Swift proxy manifest entries                  [ DONE ]
Adding Swift storage manifest entries                [ DONE ]
Adding Swift common manifest entries                 [ DONE ]
Adding MongoDB manifest entries                      [ DONE ]
Adding Ceilometer manifest entries                   [ DONE ]
Adding Ceilometer Keystone manifest entries          [ DONE ]
Adding post install manifest entries                 [ DONE ]
Installing Dependencies                              [ DONE ]
Copying Puppet modules and manifests                 [ DONE ]
Applying 192.168.1.68_prescript.pp
Applying 10.3.235.114_prescript.pp
10.3.235.114_prescript.pp:                           [ DONE ]
192.168.1.68_prescript.pp:                           [ DONE ]
Applying 10.3.235.114_amqp.pp
Applying 192.168.1.68_mysql.pp
10.3.235.114_amqp.pp:                                [ DONE ]
192.168.1.68_mysql.pp:                               [ DONE ]
Applying 192.168.1.68_keystone.pp
192.168.1.68_keystone.pp:                            [ DONE ]
Applying 192.168.1.68_ring_swift.pp
192.168.1.68_ring_swift.pp:                          [ DONE ]
Applying 192.168.1.68_swift.pp
192.168.1.68_swift.pp:                               [ DONE ]
Applying 10.3.235.114_mongodb.pp
10.3.235.114_mongodb.pp:                             [ DONE ]
Applying 192.168.1.68_ceilometer.pp
192.168.1.68_ceilometer.pp:                          [ DONE ]
Applying 192.168.1.68_postscript.pp
Applying 10.3.235.114_postscript.pp
10.3.235.114_postscript.pp:                          [ DONE ]         
192.168.1.68_postscript.pp:                          [ DONE ]         
Applying Puppet manifests                            [ DONE ]
Finalizing                                           [ DONE ]

 **** Installation completed successfully ******


Additional information:
 * Deprecated parameter has been used in answer file. Please use parameter CONFIG_CONTROLLER_HOST next time. This parameter deprecates following parameters: ['CONFIG_CEILOMETER_HOST', 'CONFIG_CINDER_HOST', 'CONFIG_GLANCE_HOST', 'CONFIG_HORIZON_HOST', 'CONFIG_HEAT_HOST', 'CONFIG_KEYSTONE_HOST', 'CONFIG_NAGIOS_HOST', 'CONFIG_NEUTRON_SERVER_HOST', 'CONFIG_NEUTRON_LBAAS_HOSTS', 'CONFIG_NOVA_API_HOST', 'CONFIG_NOVA_CERT_HOST', 'CONFIG_NOVA_VNCPROXY_HOST', 'CONFIG_NOVA_SCHED_HOST', 'CONFIG_OSCLIENT_HOST', 'CONFIG_SWIFT_PROXY_HOSTS'].
 * Deprecated parameter has been used in answer file. Please use parameter CONFIG_COMPUTE_HOSTS next time. This parameter deprecates following parameters: ['CONFIG_NOVA_COMPUTE_HOSTS'].
 * Deprecated parameter has been used in answer file. Please use parameter CONFIG_NETWORK_HOSTS next time. This parameter deprecates following parameters: ['CONFIG_NEUTRON_L3_HOSTS', 'CONFIG_NEUTRON_DHCP_HOSTS', 'CONFIG_NEUTRON_METADATA_HOSTS', 'CONFIG_NOVA_NETWORK_HOSTS'].
 * Deprecated parameter has been used in answer file. Please use parameter CONFIG_SWIFT_STORAGES next time. This parameter deprecates following parameters: ['CONFIG_SWIFT_STORAGE_HOSTS'].
 * Time synchronization installation was skipped. Please note that unsynchronized time on server instances might be problem for some OpenStack components.
 * The installation log file is available at: /var/tmp/packstack/20141029-181742-ESZKio/openstack-setup.log
 * The generated manifests are available at: /var/tmp/packstack/20141029-181742-ESZKio/manifests

Apparently everything went well, i checked the system status using top and…

Tasks: 277 total,   6 running, 268 sleeping,   3 stopped,   0 zombie
%Cpu0  : 83.3 us,  0.0 sy,  0.0 ni, 16.7 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu1  : 85.7 us, 14.3 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu2  :100.0 us,  0.0 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
%Cpu3  : 85.7 us, 14.3 sy,  0.0 ni,  0.0 id,  0.0 wa,  0.0 hi,  0.0 si,  0.0 st
KiB Mem:  15904096 total,  6865632 used,  9038464 free,   192632 buffers
KiB Swap:  4095996 total,    69764 used,  4026232 free,  4173952 cached

  PID USER      PR  NI    VIRT    RES    SHR S  %CPU %MEM     TIME+ COMMAND                                                                                                 
31259 mysql     20   0 1731240  83920  10124 S 105.7  0.5   8:10.90 mysqld                                                                                                  
 1981 keystone  20   0  401476  43724   5816 S  39.6  0.3   3:35.85 keystone-manage                                                                                         
 7968 keystone  20   0  401096  43324   5816 R  39.6  0.3   0:56.46 keystone-manage                                                                                         
 3832 keystone  20   0  401476  43720   5816 S  26.4  0.3   2:56.38 keystone-manage                                                                                         
 4737 keystone  20   0  401476  43720   5816 S  26.4  0.3   2:16.55 keystone-manage                                                                                         
 5751 keystone  20   0  401496  43832   5816 R  26.4  0.3   1:41.61 keystone-manage                                                                                         
 7702 keystone  20   0  401352  43584   5816 R  26.4  0.3   1:15.26 keystone-manage                                                                                         
 8015 keystone  20   0  401100  43324   5816 S  26.4  0.3   0:37.85 keystone-manage                                                                                         
 8083 keystone  20   0  400964  43056   5816 S  26.4  0.3   0:19.59 keystone-manage                                                                                         
 8137 keystone  20   0  400720  42796   5816 R  26.4  0.3   0:03.38 keystone-manage                                                                                         
 1108 keystone  20   0  401488  43724   5816 S  13.2  0.3   4:13.95 keystone-manage                    

The keystone-manage token_flush command was taking a lot of time to finish and since it's being executed every minute it was causing this behaviuour. The problem was that i had 2827 in the token table of the keystone database, i deleted then manually and everything went ok.

This next section is not needed if you're not experimenting with a modified answer file, but well this trick can be useful…
Turns out that i had CONFIG_HORIZON_INSTALL=n in the answer file and horizon wasn't installed so i changed it to CONFIG_HORIZON_INSTALL=y and this time just run packstack with the –dry-run option:

 packstack  -d --dry-run --answer-file=packstack-answers-20130821-004155.txt
Welcome to Installer setup utility

Installing:
Clean Up                                             [ DONE ]
Setting up ssh keys                                  [ DONE ]
Discovering hosts' details                           [ DONE ]
Adding pre install manifest entries                  [ DONE ]
Preparing servers                                    [ DONE ]
Adding AMQP manifest entries                         [ DONE ]
Adding MySQL manifest entries                        [ DONE ]
Adding Keystone manifest entries                     [ DONE ]
Adding Horizon manifest entries                      [ DONE ]
Adding Swift Keystone manifest entries               [ DONE ]
Adding Swift builder manifest entries                [ DONE ]
Adding Swift proxy manifest entries                  [ DONE ]
Adding Swift storage manifest entries                [ DONE ]
Adding Swift common manifest entries                 [ DONE ]
Adding MongoDB manifest entries                      [ DONE ]
Adding Ceilometer manifest entries                   [ DONE ]
Adding Ceilometer Keystone manifest entries          [ DONE ]
Adding post install manifest entries                 [ DONE ]
Installing Dependencies                              [ DONE ]
Copying Puppet modules and manifests                 [ DONE ]
Applying Puppet manifests                            [ DONE ]
Finalizing                                           [ DONE ]

 **** Installation completed successfully ******


Additional information:
 * Deprecated parameter has been used in answer file. Please use parameter CONFIG_CONTROLLER_HOST next time. This parameter deprecates following parameters: ['CONFIG_CEILOMETER_HOST', 'CONFIG_CINDER_HOST', 'CONFIG_GLANCE_HOST', 'CONFIG_HORIZON_HOST', 'CONFIG_HEAT_HOST', 'CONFIG_KEYSTONE_HOST', 'CONFIG_NAGIOS_HOST', 'CONFIG_NEUTRON_SERVER_HOST', 'CONFIG_NEUTRON_LBAAS_HOSTS', 'CONFIG_NOVA_API_HOST', 'CONFIG_NOVA_CERT_HOST', 'CONFIG_NOVA_VNCPROXY_HOST', 'CONFIG_NOVA_SCHED_HOST', 'CONFIG_OSCLIENT_HOST', 'CONFIG_SWIFT_PROXY_HOSTS'].
 * Deprecated parameter has been used in answer file. Please use parameter CONFIG_COMPUTE_HOSTS next time. This parameter deprecates following parameters: ['CONFIG_NOVA_COMPUTE_HOSTS'].
 * Deprecated parameter has been used in answer file. Please use parameter CONFIG_NETWORK_HOSTS next time. This parameter deprecates following parameters: ['CONFIG_NEUTRON_L3_HOSTS', 'CONFIG_NEUTRON_DHCP_HOSTS', 'CONFIG_NEUTRON_METADATA_HOSTS', 'CONFIG_NOVA_NETWORK_HOSTS'].
 * Deprecated parameter has been used in answer file. Please use parameter CONFIG_SWIFT_STORAGES next time. This parameter deprecates following parameters: ['CONFIG_SWIFT_STORAGE_HOSTS'].
 * Time synchronization installation was skipped. Please note that unsynchronized time on server instances might be problem for some OpenStack components.
 * To access the OpenStack Dashboard browse to http://192.168.1.68/dashboard .
Please, find your login credentials stored in the keystonerc_admin in your home directory.
 * The installation log file is available at: /var/tmp/packstack/20141029-201826-heukRa/openstack-setup.log
 * The generated manifests are available at: /var/tmp/packstack/20141029-201826-heukRa/manifests

and then run the horizon puppet manifest manually:

puppet apply --modulepath=/usr/share/openstack-puppet/modules /var/tmp/packstack/20141029-201826-heukRa/manifests/192.168.1.68_horizon.pp

In the end it wasn't as painfull as i thought and now i have my Juno setup to play with, next, i'm gonna configure nova to use Docker as hypervisor but that's for another post ;)