User Tools

Site Tools



This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
system:vicki_debian_lenny_to_squeeze [2012-02-14T13:40:57+0000]
michael_paoli added autostart and confirmation bits
system:vicki_debian_lenny_to_squeeze [2013-02-17T20:00:48+0000] (current)
michael_paoli typo/wordo fix
Line 28: Line 28:
 ===== draft/​outline of upgrade plans/​procedure & background: ===== ===== draft/​outline of upgrade plans/​procedure & background: =====
 <​file>​ <​file>​
-# hostname --fqdn; pwd -P; expand -t 2 < 0001_a_general_plan_or_outline+# hostname --fqdn; pwd -P; expand -t 2 < 0010_a_general_plan_or_outline 
 SF-LUG & BALUG: System OS upgrades *soon*(?) - volunteer(s)?​ SF-LUG & BALUG: System OS upgrades *soon*(?) - volunteer(s)?​
Line 76: Line 78:
          ​removed from LVM; md1 data wiped to all binary zeros)          ​removed from LVM; md1 data wiped to all binary zeros)
      o install Debian 6.0.4 amd64, using beginning area(s) of disks      o install Debian 6.0.4 amd64, using beginning area(s) of disks
-       ​(md[01] (sd[ab]12) and area preceeding ​sd[ab]1 (boot blocks, MBR,+       ​(md[01] (sd[ab]12) and area preceding ​sd[ab]1 (boot blocks, MBR,
        ​partition table) - partition layout to be preserved, all data on        ​partition table) - partition layout to be preserved, all data on
-       ​all ​partions ​to be preserved except sd[ab]12 via md[01] will be+       ​all ​partitions ​to be preserved except sd[ab]12 via md[01] will be
        used for /boot and LVM2 respectively)        used for /boot and LVM2 respectively)
        ​general architecture layout mostly quite as before (everything        ​general architecture layout mostly quite as before (everything
Line 114: Line 116:
 references: references:
 http://​​debian-security-announce/​2011/​msg00238.html http://​​debian-security-announce/​2011/​msg00238.html
 </​file>​ </​file>​
Line 143: Line 146:
 ===== disk layout, etc. details: ===== ===== disk layout, etc. details: =====
 <​file>​ <​file>​
-# hostname --fqdn; pwd -P; more 000[23]* | expand -t 2+# hostname --fqdn; pwd -P; more 00[23]* | expand -t 2
 /​root/​upgrades/​5.0_lenny_to_6.0_squeeze /​root/​upgrades/​5.0_lenny_to_6.0_squeeze
 ::::::::::::::​ ::::::::::::::​
 ::::::::::::::​ ::::::::::::::​
 //hard drive partitions, we have: //hard drive partitions, we have:
Line 244: Line 247:
 //excepting extended partition, all logical and non-zero length primary //excepting extended partition, all logical and non-zero length primary
-//​partitions paired up between the sda and sdb devices ​partisions ​as md+//​partitions paired up between the sda and sdb devices ​partitions ​as md
 //raid1 devices: //raid1 devices:
 # mdadm --verbose --examine --scan # mdadm --verbose --examine --scan
Line 305: Line 308:
 //​apparently /​dev/​md[7-9] and /dev/md10 not in use (free/​available) //​apparently /​dev/​md[7-9] and /dev/md10 not in use (free/​available)
 ::::::::::::::​ ::::::::::::::​
 ::::::::::::::​ ::::::::::::::​
 //before: //before:
Line 337: Line 340:
 [[ip_addresses|IP Addresses]] [[ip_addresses|IP Addresses]]
 <​file>​ <​file>​
-# hostname --fqdn; pwd -P; expand < 0005_networking+# hostname --fqdn; pwd -P; expand < 0050_networking
 /​root/​upgrades/​5.0_lenny_to_6.0_squeeze /​root/​upgrades/​5.0_lenny_to_6.0_squeeze
Line 418: Line 421:
         dns-search         dns-search
 $ $
 </​file>​ </​file>​
 ===== xen --> qemu-kvm (specific example sflug): ===== ===== xen --> qemu-kvm (specific example sflug): =====
 <​file>​ <​file>​
-# hostname --fqdn; pwd -P; expand -t 2 < 0006_vicki_sflug_xen2qemu-kvm+# hostname --fqdn; pwd -P; expand -t 2 < 0060_vicki_sflug_xen2qemu-kvm
 /​root/​upgrades/​5.0_lenny_to_6.0_squeeze /​root/​upgrades/​5.0_lenny_to_6.0_squeeze
Line 431: Line 434:
 o sflug is Debian GNU/Linux 5.0.9 i386 (excepting some slight bits that o sflug is Debian GNU/Linux 5.0.9 i386 (excepting some slight bits that
   may predate that - e.g. some existing low-level bits quite   may predate that - e.g. some existing low-level bits quite
-  ​interedependent ​with existing Xen host domU, e.g. kernel - but+  ​interdependent ​with existing Xen host domU, e.g. kernel - but
   nevertheless,​ even those bits are at least Debian GNU/Linux 5.0.x   nevertheless,​ even those bits are at least Debian GNU/Linux 5.0.x
   i386)   i386)
Line 450: Line 453:
     filesystem, as applicable)     filesystem, as applicable)
   o UUIDs should be unique, so we adjust accordingly   o UUIDs should be unique, so we adjust accordingly
-  o use e2fstune ​to change UUID of target root (/) filesystem,+  o use tune2fs ​to change UUID of target root (/) filesystem,
   o use mkswap to recreate target swap   o use mkswap to recreate target swap
   o use blkid to determine UUID and label(s) of the above targets   o use blkid to determine UUID and label(s) of the above targets
Line 477: Line 480:
               --disk path=/​dev/​vg-sflug/​sflug-sda,​format=raw,​bus=scsi \               --disk path=/​dev/​vg-sflug/​sflug-sda,​format=raw,​bus=scsi \
               --wait=-1               --wait=-1
-    boot the (virtual) guest from CD into graphical recovery ​mode+    boot the (virtual) guest from CD into Graphical rescue ​mode
     ...     ...
     Device to use as root file system: /dev/sda1     Device to use as root file system: /dev/sda1
Line 538: Line 541:
     # virsh autostart sflug     # virsh autostart sflug
   o shutdown guest(s), reboot host, check that all properly comes up   o shutdown guest(s), reboot host, check that all properly comes up
 +===== vicki host install/"​upgrade"​ procedure outline + select details: ===== 
 +# hostname --fqdn; pwd -P; expand -t 2 < 0070_vicki_upgrade_steps 
 +vicki upgrade key steps (outline of general procedure - fair bit of 
 +detail, but skipping details of many of the more obvious/​routine steps) 
 +(remaining) pre steps: 
 +o remount /boot ro 
 +o backup /boot data 
 +o backup MBR (440 bytes) 
 +o shutdown VM guests 
 +o attach KVM 
 +o shutdown vicki 
 +install(/​merge/​upgrade) steps: 
 +o attach bootable USB image: debian-6.0.4-amd64-CD-1.iso 
 +o boot USB 
 +o Advanced options 
 +o Graphical expert install 
 +o ... 
 +o Configure locales, add all en_US* locales 
 +o default locale: en_us.UTF-8 
 +o ... 
 +o Load installer components from CD, include: 
 +  o cfdisk-udeb 
 +  o choose-mirror 
 +  o multipath 
 +  o openssh 
 +  o parted-udeb 
 +  o partman-reiserfs 
 +  o reiserfs-modules 
 +o Configure the network 
 +  o Auto-configure network with DHCP: 
 +    o no 
 +  o IP address: 
 +  o Netmask: 
 +  o Gateway: 
 +  o Name server addresses: 
 +  o Hostname: vicki 
 +  o Domain name: 
 +o Set up users and passwords 
 +  o Allow login as root: yes (will change that later) 
 +  o Create a normal user account now?: No 
 +o Configure the clock 
 +  o ... 
 +  o Pacific 
 +o Partition disks 
 +  o Manual 
 +    o Configure software RAID 
 +    o Keep current partition layout and conigure RAID?: Yes 
 +    o Create MD device 
 +    o RAID1 
 +    o Number of active devices for the RAID1 array: 2 
 +    o Number of spare devices for the RAID1 array: 0 
 +    o select partition 1 from both disks 
 +    o Keep current partition layout and conigure RAID?: Yes 
 +    o Create MD device 
 +    o RAID1 
 +    o Number of active devices for the RAID1 array: 2 
 +    o Number of spare devices for the RAID1 array: 0 
 +    o select partition 2 from both disks 
 +    o Keep current partition layout and conigure RAID?: Yes 
 +    o Finish 
 +    o RAID1 device #0 #1 
 +      o Ext3 
 +      o /boot 
 +      o vicki-boot 
 +    o RAID1 device #1 #1 
 +      o physical volume for LVM 
 +    o Configure the Logical Volume Manager 
 +    o Write the changes to disks and configure LVM?: Yes 
 +    o Create volume group 
 +    o Volume group name: vicki 
 +    o /dev/md1 
 +    o Create logical volume 
 +      logical volume name & size 
 +      root / 1G 
 +      usr 2G 
 +      var 4G 
 +      home 2G 
 +      (we'll add swap later, not now: 
 +        swap1 512M 
 +        swap2 512M 
 +        swap3 512M 
 +        swap4 512M 
 +      ) 
 +    o Partition disks 
 +      LVM mount fs type 
 +      root / ext3 
 +      usr /usr ext3 
 +      var /var ext3 
 +      home /home ext3 
 +    o RAID1 device #0 #1 
 +      o Ext3 
 +      o /boot 
 +      o vicki-boot 
 +    o Finish partitioning and write changes to disk 
 +    (continue without swap) 
 +o Install base system 
 +after that, and before kernel, etc.: 
 +ID "​merging":​ 
 +At this point, target filesystems are mounted on/under /target, including 
 +/​etc/​{passwd,​shadow,​group,​gshadow} files (although root password isn't in 
 +there yet).  Here's where we start UID/GID reconciliation and alignment 
 +and merging in, etc. of IDs from "​old"​ vicki. 
 +merge/​reconcile /​etc/​{passwd,​shadow,​group,​gshadow} 
 +activate and use console session on tty2 (Ctrl+Alt+F2) 
 +start additional md (mdadm) devices, e.g.: 
 +~ # mdadm --assemble /dev/md2 /​dev/​sd[ab]5 
 +~ # mdadm --assemble /dev/md3 /​dev/​sd[ab]6 
 +~ # mdadm --assemble /dev/md4 /​dev/​sd[ab]7 
 +~ # mdadm --assemble /dev/md5 /​dev/​sd[ab]8 
 +~ # mdadm --assemble /dev/md6 /​dev/​sd[ab]9 
 +~ # mdadm --assemble /dev/md7 /​dev/​sd[ab]10 
 +~ # mdadm --assemble /dev/md8 /​dev/​sd[ab]11 
 +~ # mdadm --assemble /dev/md9 /​dev/​sd[ab]12 
 +~ # mdadm --assemble /dev/md10 /​dev/​sd[ab]13 
 +Scan for volume groups: 
 +~ # vgscan 
 +and activate: 
 +~ # vgchange -a y vg00 
 +~ # vgchange -a y vg-local 
 +~ # vgchange -a y vg-sflug 
 +~ # vgchange -a y vg-balug 
 +mount "​old"​ root filesystem and copy ID information to handy location, 
 +and while we're at it, ssh keys: 
 +~ # mkdir /tmp/mnt 
 +~ # mount -o ro /​dev/​vg00/​root /tmp/mnt 
 +~ # (umask 077 && mkdir /​target/​var/​tmp/​etc /​target/​var/​tmp/​etc/​ssh) 
 +~ # (cd /​tmp/​mnt/​etc && cp -p passwd shadow group gshadow /​target/​var/​tmp/​etc/​) 
 +~ # (cd /​tmp/​mnt/​etc/​ssh && cp -p *key* /​target/​var/​tmp/​etc/​ssh/​) 
 +unmount "​old"​ root filesystem and remove our temporary mountpoint:​ 
 +~ # umount /tmp/mnt && rmdir /tmp/mnt 
 +chroot into target we're building: 
 +~ # cd / && exec chroot /target 
 +optionally give us more "​friendly"​ shell, etc.: 
 +# cd / && exec /bin/bash --posix --login 
 +bash-4.1# PS1='# ' 
 +# set -o vi; FCEDIT=vi VISUAL=vi EDITOR=vi export FCEDIT VISUAL EDITOR 
 +mount /proc filesystem:​ 
 +# mount /proc 
 +VERY CAREFULLY merge in ID information from /​var/​tmp/​etc/​* files into 
 +corresponding /etc/* files: 
 +# vipw 
 +# pwconv 
 +# vipw -s 
 +# vigr 
 +# grpconv 
 +# vigr -s 
 +etc. as needed 
 +o any changes on NEW filesystem(s) and/or 
 +o any changes on OLD filesystem(s) 
 +At last dry run check on 2012-02-25, the following issues were found, 
 +and their recommended actions: 
 +login/UID conflicts/​issues:​ 
 +  new UID/GID on target 
 +  adjust any old data before allowing import, multichown IDspec: 
 +  107,​100,​107,​101 
 +  uid conflict (libuuid vs. old Debian-exim) 
 +  change Debian-exim to available <1000 UID 
 +  adjust any old data before allowing import, 
 +  multichown IDspec (replacing Debian-exim with new target UID, e.g. 125): 
 +  100,125 
 +group/GID conflicts/​issues:​ 
 +  new GID on target: 101 
 +  adjust any old data before allowing import, 
 +  ,,107,101 
 +  new GID on target: 102 
 +  adjust any old data before allowing import, 
 +  ,,101,102 
 +  gid conflict (vs. old Debian-exim) 
 +  change Debian-exim to available <1000 UID 
 +  adjust any old data before allowing import, 
 +  ,,102,125 
 +umount /proc filesystem from within chroot and exit chroot: 
 +# umount /proc && exit 
 +o continue with installation (kernel, etc.) 
 +o include non-free (in case we need it for, e.g. firmware) 
 +o Software selection 
 +  o Choose software to install: 
 +   ​select only: 
 +   x SSH server 
 +   x Standard system utilities 
 +   ​(we'​ll add other stuff later) 
 +o Is the system clock set to UTC?: Yes 
 +reboot to single user 
 +empty contents of /tmp 
 +edit /etc/fstab to use tmpfs for tmp: 
 +tmpfs /tmp tmpfs nosuid,​nodev 0 0 
 +mount /tmp 
 +recopy our "​old"​ ssh keys to /etc/ssh/ 
 +Should be able to login via ssh as "​regular"​ user and su to root, 
 +verify that, and if okay, disable root login via ssh 
 +start additional md (mdadm) devices, e.g.: 
 +# mdadm --assemble /dev/md2 /​dev/​sd[ab]5 
 +# mdadm --assemble /dev/md3 /​dev/​sd[ab]6 
 +# mdadm --assemble /dev/md4 /​dev/​sd[ab]7 
 +# mdadm --assemble /dev/md5 /​dev/​sd[ab]8 
 +# mdadm --assemble /dev/md6 /​dev/​sd[ab]9 
 +# mdadm --assemble /dev/md7 /​dev/​sd[ab]10 
 +# mdadm --assemble /dev/md8 /​dev/​sd[ab]11 
 +# mdadm --assemble /dev/md9 /​dev/​sd[ab]12 
 +# mdadm --assemble /dev/md10 /​dev/​sd[ab]13 
 +add them to /​etc/​mdadm/​mdadm.conf:​ 
 +# /​usr/​share/​mdadm/​mkconf | diff --ed /​etc/​mdadm/​mdadm.conf - 
 +... and apply those changes 
 +mount our CD image and configure into apt 
 +  sudo libvirt-bin libvirt-doc virtinst virt-viewer qemu-kvm ed nvi 
 +reconfigure network using br0 
 +convert sflug from Xen to qemu-kvm and configure to autostart 
 +change line in /​etc/​modules from: 
 +loop max_loop=256 
 +reboot vicki, confirm all comes up as expected 
 +time permitting, work on IMPI 
 +remainder can be handled remotely 
 </​file>​ </​file>​
 ===== checklist/​outline of things to bring onsite: ===== ===== checklist/​outline of things to bring onsite: =====
   * laptop(s) - Michael Paoli   * laptop(s) - Michael Paoli
-  * Ethernet cables - Michael Paoli+  * Ethernet cables ​(should bring at least 5) - Michael Paoli
   * 10/100/1000 Mbit Ethernet switch - Michael Paoli   * 10/100/1000 Mbit Ethernet switch - Michael Paoli
-  * "​Home"​ router (optional; ​Michael Paoli may bring) +  * "​Home"​ router (Linksys running OpenWrt ​Michael Paoli will bring) 
-  * portable power strip - Michael Paoli+  * portable power strips ​- Michael Paoli
   * off-line accessible copies of reference documentation,​ Michael Paoli:   * off-line accessible copies of reference documentation,​ Michael Paoli:
     * existing root passwords     * existing root passwords
Line 555: Line 817:
     * Debian GNU/Linux 6.0.4 "​Squeeze"​ - Official amd64 CD Binary-1 20120128-13:​42     * Debian GNU/Linux 6.0.4 "​Squeeze"​ - Official amd64 CD Binary-1 20120128-13:​42
     * Debian GNU/Linux 6.0.4 "​Squeeze"​ - Official amd64 NETINST Binary-1 20120129-00:​39     * Debian GNU/Linux 6.0.4 "​Squeeze"​ - Official amd64 NETINST Binary-1 20120129-00:​39
-===== to do: ===== + 
-  *prepare more general outline ​of procedure steps as necessary/appropriate +===== procedure "​details"​ - the "​short"​ list /outline (specifics, etc.) ===== 
-  *save needed ​upgrade ​related ​data (e.g. procedure steps/outlines) to handy locations accessible throughout ​upgrade +  * disable its IP: currently and from restart (not needed for, and highly useful ​to have 2nd IP freed for upgrade/​etc. procedures) 
-  *checklist/outline ​of things ​to bring onsite +  * Michael Paoli linux laptop - configure for network: 
-  *xen --> qemu-kvm ​conversion+    * down dhcp server, repoint dhcp server to config. for Servepath 
-    *work out, test, and document ​guest/qemu-kvm network conversion/setup bits (host bits documented)+    * reconfig br0: # ifconfig br0 netmask broadcast up 
 +    * # route add default gw 
 +    * DNS: 
 +    * restart dhcp server 
 +  * connect: 
 +    * Gig switch: Internet, vicki, Michael Paoli linux laptop, Linksys "​Internet"​ Ethernet port 
 +    * Linksys Ethernet ports 1-5 - additional laptop(s) (drive/​watch process, Internet via NAT/SNAT) 
 +  * (www.) - prepare to have services temporarily on Michael Paoli'​s laptop: 
 +    * have command set, but don't yet hit enter: # ifconfig br0:1 netmask broadcast up 
 +    * bring down sflug domU, and once down, hit enter on above command 
 +    * test connectivity/​functionality (sflug domU down, but DNS and http up on 
 +  * shutdown the balug VM 
 +  * continue with vicki "​upgrade"/​install as noted on this wiki page, and including also: 
 +  * before bringing sflug VM back up on Internet, on Michael Paoli'​s Linux laptop, down the IP​ 
 +    * # ifconfig br0:1 down 
 +===== vicki "​upgraded"​ to Debian GNU/Linux 6.0.4 amd64 ===== 
 +  * 2012-02-27 initial main phase of vicki being "​upgrade"​ completed (install + merge) 
 +  * 2012-03-01 completed UID/GID alignments - "​old"​ filesystems and their data to be merged, most or all other remnant cleanup and configuration bits, etc. 
 +===== balug VM upgraded to Debian GNU/Linux 6.0.4 ===== 
 +  * 2012-03-04--2012-03-05 balug VM was upgraded, remnant cleanup tasks completed relatively shortly thereafter 
 +  * balug VM log bits made available here: [[http://​​log.txt]] 
 +===== sflug VM upgrade ​to Debian GNU/Linux 6.0.4 ===== 
 +  * sflug VM log bits made available here: [[http://​​log.txt]] 
 +  * upgrade to be done [[http://​​pipermail/​sf-lug/​2012q1/​009242.html|soon]] ... [[http://​​pipermail/​sf-lug/​2012q1/​009246.html|"​done"​]] 
 +  * will be upgrading/​was upgraded, mostly per: 
 +    * **backup(s)** - since [www.] is (generally) "​down"​ when sflug is down, we'll mostly try to minimize the host's downtime. ​ So, for backup, backing up "​everything"​ is good,  but as this is a VM, and its storage on the hosting system is under LVM, we have some nice options for fast live backups - backups from which, state is at least "​recoverable",​ namely: 
 +      * from host (vicki), sflug'​s storage is an LVM LV, 
 +      * we can take an LV snapshot of that LV, create backup copy of snapshot, then release the snapshot, 
 +      * though that doesn'​t gives us fully consistent filesystem images (as the guest filesystem(s) are still live, mounted rw, and active at that time, 
 +      * it does give us fully recoverable point-in-time capture of the guest'​s disk image (as if one instantly removed power from the guest and then took a snapshot of its hard drive data), 
 +      * note that otherwise taking a copy of live filesystem ​(e.g. via dd) is NOT guaranteed to be recoverable,​ as data continues to change as the filesystem is being read, thus such an image would not be guaranteed to be recoverable,​ 
 +      * note also other methods could be used to ensure recoverable copy, e.g. by creating a mirror, "​breaking"​ (separating off) that mirror copy, and then using that (as that, again, provides a point-in-time snapshot of the entire filesystem or whatever data is mirrored),​ 
 +      * note also we need to pay due attention/care regarding UUIDs and such to avoid problems, as this LVM snapshot method will introduce some duplication of UUIDs, but note, however, in our scenario that is //mostly// a non-issue, as: 
 +        * from the host there are separate and unique LVs, and for the most part host wouldn'​t go poking within to look at, e.g. partitions within or other constructs within, thus host normally wouldn'​t see any of those further lower level UUIDs within, 
 +        * we won't present both "​original"​ and "​copy"​ to guest (sflugor any other single VM at the same time - if we did, that's where problems would occur, as host would see two hard drives, with identical UUIDs, and thus problems would be likely ​to occur, 
 +        * also, these backups are "​temporary"​ and for convenience - "just in case" ​upgrade ​goes horribly wrong or something like that - gives us quick convenient way to get back to "​precisely"​ where we were before we started the upgrade (specifically to the point of the moment the LV snapshot was taken), we probably won't need to use this backup at all, and will dispose of it (and its UUIDs - wipe the data), fairly shortly after we're confident all has gone quite well with the upgrade and we've no interest in potentially reverting to the pre-upgrade snapshot backup, 
 +        note also this is separate/independent ​of our more (semi-?​)regular backups, which tend to be filesystem based and avoid most all the UUID issues (and also tend to generally be much more space efficient) - in this particular pre-upgrade backup scenario, we're going for recoverability and speed and uptime over space efficiency of stored (or transmitted) backup. 
 +      * doing the backup - here's part of what doing the "​proposed"​ LV snapshot based backup looks like, in this case I do a "​real"​ pre-run (plus some additional checks) - when we're much closer to our upgrade time, will do another nice fresh backup (except some of that pre-work will already be done, so we can skip repeating some bits - we can also skip some of the test/​verification bits done here for demonstration purposes) 
 +//I show my comments on lines starting with // 
 +//I don't show all output of all commands (mostly reduced for brevity/​clarity) 
 +//from sflug (guest), we have relatively simplistic filesystem and swap setup, 
 +//one "​physical"​ (virtual from host, but effectively physical to guest) hard drive 
 +//with partition for root (/) filesystem, and swap 
 +# sfdisk -uS -l 
 +Disk /dev/sda: 1435 cylinders, 255 heads, 63 sectors/​track 
 +Units = sectors of 512 bytes, counting from 0 
 +   ​Device Boot    Start       ​End ​  #​sectors ​ Id  System 
 +/​dev/​sda1 ​           63  20980889 ​  ​20980827 ​ 83  Linux 
 +/​dev/​sda2 ​     20980890 ​ 22041179 ​   1060290 ​ 82  Linux swap / Solaris 
 +/​dev/​sda3 ​            ​0 ​        ​- ​         0   ​0 ​ Empty 
 +/​dev/​sda4 ​            ​0 ​        ​- ​         0   ​0 ​ Empty 
 +//and to confirm a bit - ignoring virtual filesystems,​ but our other filesystem(s) and swap: 
 +# mount -t notmpfs,​noproc,​nodevpts,​nosysfs,​nousbfs 
 +/dev/sda1 on / type ext3 (rw,​errors=remount-ro) 
 +# awk '​{if($2=="/"​)print;​};'​ /​etc/​fstab 
 +UUID=792a4837-91bc-4b36-bce7-a5a4208845bf / ext3 errors=remount-ro 0 1 #/​dev/​sda1 
 +# blkid /dev/sda1 
 +/dev/sda1: UUID="​792a4837-91bc-4b36-bce7-a5a4208845bf"​ SEC_TYPE="​ext2"​ TYPE="​ext3"​ 
 +# cat /​proc/​swaps 
 +Filename ​                               Type            Size    Used    Priority 
 +/​dev/​sda2 ​                              ​partition ​      ​530136 ​ 0       -1 
 +# awk '​{if($3=="​swap"​)print;​};'​ /​etc/​fstab 
 +UUID=e326206f-55a6-4fbb-8c81-62f8dfb30753 none swap sw 0 0 #/dev/sda2 LABEL=swap 
 +# blkid /dev/sda2 
 +/dev/sda2: TYPE="​swap"​ LABEL="​swap"​ UUID="​e326206f-55a6-4fbb-8c81-62f8dfb30753"​ 
 +//so, quite simple there and no surprises. ​ And, what does that look like from the host? 
 +# hostname 
 +# virsh list --all 
 + Id Name                 ​State 
 +  2 sflug                running 
 + 26 balug                running 
 +# virsh dumpxml sflug | fgrep '<​source dev='​ 
 +      <source dev='/​dev/​vg-sflug/​sda'/>​ 
 +//so, the storage for the sflug VM is on /​dev/​vg-sflug/​sda in VG /​dev/​vg-sflug,​ 
 +//how much space used and free (and PE size) for that VG? 
 +# vgdisplay /​dev/​vg-sflug 
 +  PE Size               4.00 MiB 
 +  Total PE              6904 
 +  Alloc PE / Size       2941 / 11.49 GiB 
 +  Free  PE / Size       3963 / 15.48 GiB 
 +//so, we can see that a fair bit less than half of the VG is used, 
 +//thus we know we've got ample room to complete our backup, 
 +//and also for our snapshot LV to be present; 
 +//snapshot LV doesn'​t have to consume same space as LV it's a snapshot of, 
 +//as it essentially only tracks changed data, 
 +//and since we'll do this backup relatively quickly, 
 +//and the sflug host won't be changing lots of data nor quickly at that time anyway, 
 +//we don't need much space for the snapshot. 
 +//So, let's do a good test run of our procedure:​ 
 +//we peek at LV names in the VG, to avoid any accidental name collision:​ 
 +# ls /​dev/​vg-sflug 
 +sda  sflug-disk-BACKUP ​ sflug-disk-BACKUP_2010-04-04 
 +//we examine size of our source LV: 
 +# lvdisplay /​dev/​vg-sflug/​sda 
 +  Current LE             ​2816 
 +//we create our target LV, 
 +//​we'​ll use -lenny suffix, 
 +//as that's codename of the "​old"​ Debian GNU/Linux 5.0.10 release we're upgrading from, 
 +//and less likely to cause confusion (e.g. backup, which backup, when, of what?) 
 +# lvcreate -l 2816 -n sda-lenny vg-sflug 
 +  Logical volume "​sda-lenny"​ created 
 +//we check on free PEs in our VG - we'll use them all for our snapshot (probably excessive),​ 
 +//but we only need them for a quite short while anyway, 
 +//so not an issue to be a bit excessive here for a quite short while anyway. 
 +//we reexamine our free PEs in the VG: 
 +# vgdisplay /​dev/​vg-sflug 
 +  Free  PE / Size       1147 / 4.48 GiB 
 +//we use them all for our snapshot LV: 
 +# lvcreate -l 1147 -n sda-snapshot -s /​dev/​vg-sflug/​sda 
 +  Logical volume "​sda-snapshot"​ created 
 +//I include use of time below, just to see how long it takes (dd also happens to provide similar information) 
 +//and also, especially when operating as superuser (root), we want to be sure our commands are correct, 
 +//Unix, and mostly likewise Linux, tends to presume one knows what one is doing, 
 +//and probably doubly (or more) so for superuser,​ 
 +//so carefully triple-check the command(s) before viciously striking the <​RETURN>​ key 
 +//(says me ;-) - has saved my butt on more than one occasion) 
 +//here we want to be particularly careful with where we're writing our data to, 
 +//what we remove, 
 +//and also that we in fact pick our correct data source 
 +# time dd if=/​dev/​vg-sflug/​sda-snapshot of=/​dev/​vg-sflug/​sda-lenny bs=4194304 
 +2816+0 records in 
 +2816+0 records out 
 +11811160064 bytes (12 GB) copied, 424.946 s, 27.8 MB/s 
 +real    7m4.957s 
 +user    0m0.016s 
 +sys     ​0m35.142s 
 +//a bit over 7 minutes - from a point-in-time snapshot, recoverable,​ 
 +//had we done that instead with source of the live rw mounted sda (well, containing sda1), 
 +//what we'd have had instead in our target data would have been an inconsistent mess with no guarantees of recoverability 
 +//and out of curiosity, just before removing it, 
 +//I peek at our snapshot to see how much space it has actually used: 
 +# lvdisplay /​dev/​vg-sflug/​sda-snapshot 
 +  Current LE             ​2816 
 +  COW-table LE           ​1147 
 +  Allocated to snapshot ​ 0.16% 
 +//as we can see, almost none (0.16%) 
 +//and removing our snapshot: 
 +# lvremove -f /​dev/​vg-sflug/​sda-snapshot 
 +  Logical volume "​sda-snapshot"​ successfully removed 
 +//removing the snapshot is important - if it's left around ("too long") and runs out of room, I/O on the LVs stops 
 +//(that can then be corrected by extending the snapshot LV, or removing it) 
 +//pretty much the same LV snapshot techniques are often used for live database (DB) backups, e.g.: 
 +//put the relevant database files/​devices in "​backup mode",​ 
 +//(done within the DB, and in many cases involves more than one which must be done together as a set), 
 +//snapshot them (as set), take database (or those files/​devices) out of backup mode, start backing up the snapshots,​ 
 +//proceed to next set, etc., 
 +//and also release each snapshot after it (or its set) has completed being backed up. 
 +//so, as exercise, we'll check the recoverability of our backup, 
 +//since it's just one journaled filesystem (and swap, that we don't care about the swap data), 
 +//fairly simple to test (we won't do this test when we do our backup "for real" - we know it works, 
 +//​we'​re just going to demonstrate it here. 
 +//And to note first, we're going to bend our UUID "​rule"​ slightly, 
 +//as here we're going to give host peek into the guest'​s UUIDs - but that's not an issue in this particular case, 
 +//as we only let it see one instance of this UUID, 
 +//as the other duplicate UUIDs are within the partitions of /​dev/​vg-sflug/​sda,​ 
 +//which we won't be having the host access (certainly at least not at the same time!) 
 +//from our partition information earlier from the guest (we could also examine it from the host), 
 +//we know how to construct a loop device to access just the first partition within our LV copy: 
 +# losetup -o `expr 512 '*' 63` --sizelimit `expr 512 '​*'​ 20980827` -f --show /​dev/​vg-sflug/​sda-lenny 
 +//does it match what we expect? 
 +# blkid /​dev/​loop0 
 +/dev/loop0: UUID="​792a4837-91bc-4b36-bce7-a5a4208845bf"​ TYPE="​ext3"​ 
 +//yes ... journaled filesystem, so, manner in which we copied it should be quite recoverable,​ 
 +//but rather than just try to mount it (which may replay journal and do relatively light checks), 
 +//​let'​s do heavier set of checks, and see what, if anything it complains about ... complaints should be 
 +//​relatively negligible and few (mostly about state not being set to clean, and perhaps a bit of journal 
 +//to replay - but that should be about it). 
 +# fsck.ext3 -f /​dev/​loop0 
 +e2fsck 1.41.12 (17-May-2010) 
 +/dev/loop0: recovering journal 
 +Clearing orphaned inode 278547 (uid=0, gid=1, mode=0100700,​ size=452) 
 +Pass 1: Checking inodes, blocks, and sizes 
 +Pass 2: Checking directory structure 
 +Pass 3: Checking directory connectivity 
 +Pass 4: Checking reference counts 
 +Pass 5: Checking group summary information 
 +/dev/loop0: ***** FILE SYSTEM WAS MODIFIED ***** 
 +/dev/loop0: 28306/​1310720 files (2.1% non-contiguous),​ 294444/​2621440 blocks 
 +//and as we'd expect, quite "​clean"​ (recoverable),​ 
 +//some bit(s) like Clearing orphaned inode or similar, may be expected, 
 +//e.g. if snapshot happens, and there are unlinked open files at the time, 
 +//then those files no longer exist in any directory,​ 
 +//but their inode (and storage, if any) hasn't yet been freed, 
 +//while that's normal for Unix/Linux if a PID still has the files open, 
 +//​that'​s no longer relevant after the fact when we're cleaning up a backup of the filesystem,​ 
 +//so that bit if housekeeping on the filesystem is still to be done (which would 
 +//normally happen rather quickly on live mounted filesystem after last PID having the unlinked 
 +//file open, closed the file (explicitly,​ or via other action such as exit)). 
 +//we could do more with that filesystem (e.g. testing, etc.) if we wished at this point, 
 +//but that more than suffices for our testing here, so time to free up that loop device: 
 +# losetup -d /​dev/​loop0 
 +    * **incorporating "​lessons learned"​** / glitches/​gottchas found and discovered from earlier similar upgrades (namely balug qemu-kvm ​VM guest on same host): 
 +      * from earlier similar upgrade done on balug qemu-kvm VM guest on same host, encountered **problem with reboot - namely upgrade procedure produced a flawed /​boot/​grub/​menu.lst file** - so we'll **carefully inspect (and adjust, as necessary/​appropriate) that file prior to performing that reboot step**. ​ From earlier with balug qemu-kvm VM guest: 
 +installation "​failed"​ around step 4.4.5 in: 
 +namely reboot failed - kernel couldn'​t find root filesystem,​ 
 +upon investigation,​ found that update procedure hadn't added initrd 
 +lines in /​boot/​grub/​menu.lst for the new kernel, I added those and was 
 +then able to proceed 
 +    * **relevant Debian documentation**:​ 
 +      * [[http://​​releases/​stable/​i386/​release-notes/​|Release Notes for Debian GNU/Linux 6.0 (squeeze)32-bit PC]] 
 +        * **especially:​ [[http://​​releases/​stable/​i386/​release-notes/​ch-upgrading.en.html|Upgrades from Debian 5.0 (lenny)]]** 
 +          * will do pre-upgrade steps in advance as feasible, have done the following:​ 
 +            * 4.1.1\\ output of:\\ # dpkg_--get-selections '​*'​\\ saved to:\\ /​root/​upgrades/​5.0_lenny_to_6.0_squeeze/​dpkg_--get-selections\\ from host (vicki), saved sflug VM configuration for reference, "just in case":​\\ # virsh dumpxml sflug > /​root/​upgrades/​5.0_lenny_to_6.0_squeeze/​sflug/​sflug.xml\\ will do fresh snapshot backup of balug VM (virtual) disk before upgrade 
 +            * 4.1.6 checked package splashy not installed 
 +            * 4.2 - should be good on no 3rd party packages, etc., system has been updated to the latest point release of lenny (5.0.10). 
 +            * 4.2.1 checked good: No packages are scheduled to be installed, removed, or upgraded. 
 +            * 4.2.2 good (/​etc/​apt/​preferences not present) 
 +            * 4.2.3:\\ looks good: # dpkg --audit\\ dpkg -l looks quite good, used: # dpkg -l | grep -v '^ii '\\ dpkg --get-selections '​*'​ also quite clean, examined: # dpkg --get-selections '​*'​ | awk '​{print $2;}' | sort | uniq -c\\ also clean: # aptitude search "​~ahold"​ 
 +            * 4.2.4 proposed-updates not in /​etc/​apt/​sources.list 
 +            * 4.2.5 - presumably not an issue (doesn'​t appear to be an issue) 
 +          * **done** (main upgrade proper sequence, and immediately preceding steps): 
 +            * did LV snapshot backup, from host (vicki) (target created earlier; also, this was set up to execute via at(1) at 5:40 P.M. on the day of the upgrade - 20 minutes prior to start of our upgrade maintenance window; and in earlier ​test runscompleted in under 10 minutes; checked that it completed okay, script ​and output on viki host under /​root/​upgrades/​5.0_lenny_to_6.0_squeeze/​sflug/​):​ 
 +              * # lvcreate -l 1147 -n sda-snapshot -s /​dev/​vg-sflug/​sda 
 +              * # time dd if=/​dev/​vg-sflug/​sda-snapshot of=/​dev/​vg-sflug/​sda-lenny bs=4194304 
 +              * # lvremove -f /​dev/​vg-sflug/​sda-snapshot 
 +              * and here's what that script that was executed looks like, and its output: 
 +# hostname; pwd -P; cat bkup; echo; cat bkup.out 
 +set -e 
 +umask 077 
 +cd / 
 +at 1740 << \__EOT__ 
 +set -e 
 +umask 077 
 +cd /​root/​upgrades/​5.0_lenny_to_6.0_squeeze/​sflug/​ 
 +exec >> bkup.out 2>&​1 
 +echo start `date -Iseconds` 
 +[ -b /​dev/​vg-sflug/​sda-lenny ] 
 +lvcreate -l 1147 -n sda-snapshot -s /​dev/​vg-sflug/​sda && { 
 +time dd if=/​dev/​vg-sflug/​sda-snapshot of=/​dev/​vg-sflug/​sda-lenny bs=4194304 || 2>&1 echo dd returned $? 
 +lvremove -f /​dev/​vg-sflug/​sda-snapshot 
 +echo end `date -Iseconds` 
 +start 2012-03-14T17:​40:​00-0700 
 +  Logical volume "​sda-snapshot"​ created 
 +2816+0 records in 
 +2816+0 records out 
 +11811160064 bytes (12 GB) copied, 421.079 s, 28.0 MB/s 
 +0.01user 34.83system 7:​01.09elapsed 8%CPU (0avgtext+0avgdata 19248maxresident)k 
 +23068824inputs+23068680outputs (1major+1242minor)pagefaults 0swaps 
 +  Logical volume "​sda-snapshot"​ successfully removed 
 +end 2012-03-14T17:​47:​04-0700 
 +            * present Debian GNU/Linux 6.0.4 "​Squeeze"​ - Official i386 CD Binary-1 20120128-12:​53 (or latest 6.0.x) from host to sflug guest,\\ we do this as a SCSI "​disk"​ (though SCSI CD-ROM would be even more preferred), as we can't hot remove virtual IDE devices from guest (though we can change virtual media ... but sometimes we want to present multiple ISO images concurrently,​ and it's preferable to be able to remove the devices when they'​re no longer needed or wanted), so we present it from host vicki thusly:\\ # virsh attach-disk sflug /var/​local/​pub/​mirrored/​​debian-cd/​6.0.4/​i386/​iso-cd/​debian-6.0.4-i386-CD-1.iso sdc --mode readonly\\ Note that that sometimes causes a slight booboo in the guest'​s configuration,​ which we'd fix via:\\ # virsh edit sflug\\ namely we'd want to change the driver name from file to qemu in the section for our added disk otherwise the guest will fail to restart once (virtually) powered off.  In our particular case this time around, virsh appeared to not introduce that issue (it used driver name phy) - probably "good enough",​ so didn't bother to "​correct"​/alter that at time of upgrade; we do however, still need to test full (virutal) power down and subsequent reboot of guest 
 +            * on guest, scan SCSI devices, create mount point (if not present), update /etc/fstab, and mount it: 
 +              * # (for tmp in /​sys/​class/​scsi_host/​host*/scan; do echo '- - -' > "​$tmp";​ done) 
 +              * # (umask 022 && mkdir /​media/​cdrom9) 
 +              * Even though we told the host to present the device to guest as sdc, it showed up on guest as sdb, so we went with that: 
 +              * add to /​etc/​fstab:​\\ /dev/sdb /​media/​cdrom9 udf,iso9660 ro,​nosuid,​nodev 0 0 
 +              * # mount /​media/​cdrom9 
 +            * 4.3 - continue per documentation\\ add new cdrom ISO to /​etc/​apt/​sources.list:​\\ # apt-cdrom -m -d=/​media/​cdrom add\\ and make other /​etc/​apt/​sources.list updates 
 +            * looking now at the above, and checking shell history, had actually intended to use /​media/​cdrom9 for the above apt-cdrom command, but didn't catch that mis-entry at the time ... so, ... apt and friends "of course"​ then wanted to use /​media/​cdrom instead of /​media/​cdrom9 ... at the time, just made symbolic link to cover that, and then continued from there: 
 +            * # ln -s /​media/​cdrom9 /​media/​cdrom 
 +            * ... 
 +            * 4.4.5 **before** first reboot, inspected /​boot/​grub/​menu.lst,​ and corrected/​edited if/as relevant, probably also increase timeout if a "too short" default timeout got placed in there - /​boot/​grub/​menu.lst actually looked fine - unlike the glitch we encountered when upgrading the balug VM, where the new kernel entries were missing their initrd entries. 
 +            * inspected legacy grub, and grub2 configurations - grub looked fine, tweaked grub2 - mostly via /​etc/​default/​grub and mostly modeling after quite similar from the balug VM guest'​s configuration with grub2, tested and legacy grub boot looked fine, chainload of grub2 worked and looked fine, booted via grub2 chainload, used the indicated procedure to then convert to update to grub2 (including MBR) and remove (but not purge) legacy grub (all handled pretty much automagically for us by running the one indicated program that did that for us - we just had to "​tell"​ it (via presented options) where grub2 was to "​install"​ (write its MBR) - the rest what highly automagic; tested reboot with direct grub2 without any "​user"​(/​admin) input being used on console (graphics or terminal) to ensure all came up fine multi-user without any input needed (to ensure unattended (re)boots should generally quite work well and as expected). 
 +            * ... 
 +            * we essentially completed through step 4.6.3, "​skipped"​ (deferred) 4.7 (the part before 4.7.1), and completed step 4.7.1, and then deferred subsequent steps (4.8 and beyond) for later subsequent "​clean-up"​ and the like, having essentially made it through all the particularly critical and important portions of the main sequence of the upgrade proper. 
 +            * we also checked both web server and DNS server both on the host and up and running and operational as expected. 
 +          * **and done(!)** - these bits were completed later, but not later than 2012-03-17:​ 
 +            * went through and completed our various clean-up tasks: 
 +              * reviewed configurations - in the interests of time (and uptime), some of the configurations where we had local modifications,​ we just used our "​old"​ configurations,​ and didn't examine or merge/​integrate our earlier changes with the newer default configurations;​ completed going through those carefully and adjusting and cleaning up as appropriate 
 +              * init / rc stuff - we didn't earlier get the advantages of the newer faster start-up system - some legacy/​dependency bit prevented that from being activated; reviewed that and got that all squared away 
 +              * 4.7 (the part before 4.7.1) 
 +              * steps starting at and beyond step 4.8 
 +              * saved script output from most of the upgrade process - reviewed that for any potentially security sensitive bits, and as there weren'​t any significant security issues with public disclosure, made that data available (at least "for a while"​) at: [[http://​​upgrade-squeeze.tar.bz2]] 
 +              * high-level (human written (with some machine assist) and intended for human consumption) log file for the system is available at:\\ [[http://​​log.txt]] - for this system upgrade bit, look/search for the line that starts with:\\ 2012-03-14\\ (at the time of this writing that's near the end of that file) ... and got that file quite caught up. 
 +            * did full virtual power reboot test (did reboot tests 2012-03-14, but not with full virtual power down and back up again with guest), ensured that guest comes up fine with no need for any console input being provided 
 +      * and (documentation bits - didn't really need to refer to or use this particular document with/for our upgrade) also if/​as/​where/​relevant:​ [[http://​​releases/​stable/​i386/​|Debian GNU/Linux Installation Guide]] (installation instructions for the Debian GNU/Linux 6.0 system (codename “squeeze”),​ for the 32-bit PC (“i386”) architecture)
system/vicki_debian_lenny_to_squeeze.1329226857.txt.bz2 · Last modified: 2012-02-14T13:40:57+0000 by michael_paoli