by Steven J. Owens (unless otherwise attributed)
This document assumes you're using debian. This should be fairly useful for ubuntu users as well, but there may be some minor differences. I'll mention any that I know about, as I go along.
If I literally wanted to just copy the installation over to a new set of hardware:
a) rsync -DragtopvH --exclude=/proc/ --exclude=/tmp/target /tmp/target
b) install a master boot record
c) make some minor edits and tweaks
Note that this would certainly depend on the hardware configuration being almost identical.
However, what I want is to build a fresh system, with all the old user data and etc, moved over.
The following assumes that your box is freshly installed and basic network configuration, etc, is set up. There are too many variables to document that process here, and in any event, there are a zillion "how to install" tutorials out there, this one is about migrating your old stuff to your new box.
The particulars of this are up to your hosting arrangement, but note that security-conscious sysadmins quite routinely deny remote root login (e.g. you have to be at the console to login as root, or ssh in via some user account and then su to root). If you can't ssh in as root@yourdomain.com, a properly security-conscious sysadmin is a likely explanation.
First, make sure what release the new box is. Ubuntu has "lsb_release -a", but on debian there's no authoritative way, you just have to check various spots:
newdarksleep:~# cat /etc/debian_version 4.0 puff@newdarksleep:~$ uname -a Linux darksleep.com 2.6.18 #2 Mon Aug 24 17:07:06 UTC 2009 i686 GNU/Linux puff@newdarksleep:~$ cat /etc/apt/sources.list deb http://mirrors.kernel.org/debian etch main puff@newdarksleep:~$
Note: Debian now has an lsb-release package (note that's spelled with a dash, not an underscore).
Here's a high-level outline of the big steps in this process:
I'm going to include the backup step in each task below, but in general, before messing with any important config file, make a backup, just in case. In many cases, the default config file can be restored by reinstalling the package, but why go to all that trouble?
I prefer to use sudo to do most important things, just because it helps me avoid staying logged in as root and being lazy and sloppy. Plus, I can check /var/log/auth.log to see when and who installed something.
Logged in as root, do adduser:
newdarksleep:~# adduser mig<enter>...enter username, password, etc...
I used the username "mig" for migration. I have a regular user account on the old box, and I don't want to complicate things by editing the same account I'm using to do the editing. Probably I'm being overly paranoid, but just because you're paranoid doesn't mean they aren't out to get you!
Backup /etc/sudoers. You could probably get away with not backing it up, since we're going to edit /etc/sudoers with the visudo command, which syntax-checks your changes and will refuse to write a broken sudoers file to /etc/sudoers. But why not be paranoid?
newdarksleep:~# cp /etc/sudoers /etc/sudoers_backup_11_24_09
Used to be you could just run an editor on /etc/sudoers, but nowadays /etc/sudoers is chmodded to be not-writable, and sudo itself won't run if /etc/sudoers is writable. This doesn't matter as much when you're using the actual root account, but if you're using sudo to run the commad to edit /etc/sudoers, it creates a catch-22 and you need to jump through hoops to fix it.
Okay, so now edit sudoers and add a line to give privileges to your user.
newdarksleep:~# visudo
---------------------------------------------------------------------- ... mig ALL=(ALL) ALL ----------------------------------------------------------------------
Note: Ordinarily I'd use emacs to edit sudoers, not vi, but emacs isn't installed yet. More about that below.
Aptitude is now standard on debian, which is good for a variety of reasons that google will be happy to tell you about.
Remember, always update apt before installing.
mig@newdarksleep:~$ sudo aptitude update
You could lump the following all into a single, one-line aptitude install, and I recommend you do so. I'm going to leave them on separate lines so you can pick and choose more easily.
I'm going to assume you know why screen and rsync are a good thing, otherwise this whole topic is probably a bit too much for you to handle.
Obviously, I like emacs better than vi, for all the obvious reasons. If you like vi better, well, there's still hope for you :-), and remember, "vi vi vi == 666, QED vi is The Editor of the Beast" :-).
mig@newdarksleep:~$ sudo aptitude install screen
mig@newdarksleep:~$ sudo aptitude install emacs
mig@newdarksleep:~$ sudo aptitude install rsync
We probably don't need mutt right off the bat, but I use it, so it'll have to get installed sooner or later, and it might come in handy in the meantime:
mig@newdarksleep:~$ sudo aptitude install mutt
Now that we've installed the basic amenities, including a civilized text editor, we'll use emacs when editing /etc/sudoers.
If you're doing it as root you can specify the editor when you invoke visudo:
newdarksleep:~# EDITOR=emacs visudo
On some systems (ubuntu) you can specify the entire above command when invoking visudo via sudo:
puff@redbitter:~$ sudo EDITOR=emacs visudo
However, that doesn't work with the debian version of sudo that I'm working with. Here's one fast and dirty way to do it:
mig@newdarksleep:~$ sudo bash -c "EDITOR=emacs visudo"
Edit the following files (see below for how to edit them):
Remember, step one, back up the files:
mig@newdarksleep:~$ sudo cp /etc/passwd /etc/passwd_backup_11_25_09 mig@newdarksleep:~$ sudo cp /etc/shadow /etc/shadow_backup_11_25_09 mig@newdarksleep:~$ sudo cp /etc/group /etc/group_backup_11_25_09 mig@newdarksleep:~$ sudo cp /etc/gshadow /etc/gshadow_backup_11_25_09 mig@newdarksleep:~$ sudo cp /etc/aliases /etc/aliases_backup_11_25_09 mig@newdarksleep:~$ sudo cp /etc/sudoers /etc/sudoers_backup_11_25_09
Use the "last" command on your old box to review the list of recent user logins and see if you want to prune any who don't login. Bear in mind that the machine may be configured with services (like squirrelmail, or domain forwarding, etc) that never require the user to actually log into the shell account, even though they use the system every day.
Don't copy the whole files, rather copy and insert the entries from the old box into the new box.
Be careful to make sure all userids and groupids are > 1000, to avoid conflicts with users/groups created by the install process (for example, the bind user).
Use the following commands to edit the files:
Or in my case, since I'm using emacs:
mig@newdarksleep:~$ sudo EDITOR=emacs vipw mig@newdarksleep:~$ sudo EDITOR=emacs vipw -s mig@newdarksleep:~$ sudo EDITOR=emacs vigr mig@newdarksleep:~$ sudo EDITOR=emacs vigr -s
Remember, if sudo doesn't take your EDITOR=emacs, use:
mig@newdarksleep:~$ sudo bash -c "EDITOR=emacs vipw" mig@newdarksleep:~$ sudo bash -c "EDITOR=emacs vipw -s" mig@newdarksleep:~$ sudo bash -c "EDITOR=emacs vigr" mig@newdarksleep:~$ sudo bash -c "EDITOR=emacs vigr -s"
Edit /etc/aliases normally, with emacs, but make sure to "sudo newaliases" after saving the file:
mig@newdarksleep:~$ sudo emacs /etc/aliases ...edit the file... mig@newdarksleep:~$ sudo newaliases
Use visudo to add sudo permissions to any users you usually give special admin privileges to:
mig@newdarksleep:~$ sudo bash -c "EDITOR=emacs visudo"
puff@olddarksleep:~$ rsync -avz -e ssh /home/ mig@newdarksleep:/home/
One caveat here: I decided I really didn't want to do the whole migration process as root, so I created a user account first, before migrating /home. However, to make life easier, and avoid any possible issues in merging olddarksleep:/home/puff and newdarksleep:/home/puff, I create a disposable "mig" (for migration) account.
The /home directory isn't all the data; we're going to have to dig out more stuff, which I'll discuss further down.
Note: Okay, so as I'm actually going through and doing this, I noticed that, of course, I can't just rsync as mig, even though mig has full sudo privs, because the stuff in /home/ belongs to all sorts of different users. Unless I figure out some workaround to have rsync invoke mig's sudo privs, I'm going to have to enable remote login for root (mainly by just resetting the root password) and rsync /home/ as root.
And, of course, we're going to have to re-do the configuration changes so we should take a snapshot of /etc for future reference:
puff@olddarksleep:~$ rsync -avz -e ssh /etc/ mig@newdarksleep:/home/mig/olddarksleep_etc/
Okay, I had planned to be more precise and step by step about this, but the new box has more disk space, and I was running out of time at the old hosting center. So I just made a brute-force backup of /etc, /home, /var/lib/, and /var/www/htdocs to the new machine, and I will then merge them in from there:
puff@olddarksleep:~$ rsync -avz -e ssh /var/lib/ mig@newdarksleep:/home/mig/olddarksleep_var/lib/
puff@olddarksleep:~$ rsync -avz -e ssh /home/ mig@newdarksleep:/home/mig/olddarksleep_home/
puff@olddarksleep:~$ rsync -avz -e ssh /var/www/htdocs/ mig@newdarksleep:/home/mig/olddarksleep_htdocs/
Strictly speaking, I could have stuck htdocs under var on the box, but I like to be able to look in /home/mig and make sure I've snagged all the big ticket items, without having to grovel through the hierarchy.
The --set-selections flag doesn't actually download anything, just sets the package state in the dpkg database to selected, i.e. it marks them as selected.
Note: In this example, I have the dpkg files located in /old/var/lib/dpkg. This assumes that you've mounted the old machine's drive on the new machine, under the mount point "/old". If you're actually running the command on the old machine, just use "--admindir=/var/lib/dpkg/".
puff@olddarksleep:~$ sudo dpkg --get-selections --admindir=/old/var/lib/dpkg/ > oldpackages.txt
Review the package selections and see if there are any you want to skip in the new machine, then rsync it across to the new machine:
puff@olddarksleep:~$ rsync -avz -e ssh oldpackages.txt mig@newdarksleep:/home/mig/
Then, on the new machine, pump the packages into dpkg --set-selections. The --set-selections flag doesn't actually download anything, just sets the package state in the dpkg database to selected, i.e. it marks them as selected.
mig@newdarksleep:~$ cat oldpackages.txt | sudo dpkg --set-selections
To actually fetch and install them, I have to do something like:
However, I'd prefer to do it via aptitude, and I'd prefer to have aptitude download all the package files ahead of time.
I'm told that I should be able to just feed the packages list into aptitude instead of into dpkg --set-selections, but that turns out to be incorrect. I can, however, do:
mig@newdarksleep:~$ cat oldpackages.txt | sudo dpkg --set-selections mig@newdarksleep:~$ sudo aptitude --download-only install
And this does what I want, causing aptitude to go out and download (but not install yet) all of the packages that I set with dpkg --set-selections.
However, this brings up a whole new can of worms (you ever wonder about that? Who the heck sells worms by the can? And who buys them!?), which is that my package setup is broken and has all sorts of dependency conflicts. Hm...
This should show me the list of manually installed packages:
mig@newdarksleep:~$ sudo aptitude search ~i\!~M
However, in my case that just gives me a list of 638 packages, instead of 817. Probably because the existing system was built via dpkg --set-selection.
Some advice from some #debian folks:
<Foo> puff for instance...if you set selections...then you do apt-get install toilet...nothing will happen [23:24-12/06] <Bar> puff: each package has a 'desired status' in the dpkg database, /var/lib/dpkg/status <puff> Foo: Okay, I guess this means that --clear-selections is safe to use for me, then. <Foo> if you clear selections and do apt-get install moo ...nothing will happen <puff> Bar: Ah, cool. <Foo> puff first do get-selections though
I'm leaning, at this point, towards making a list of the packages I'm pretty sure I want, and installing them and configuring them, and adding anything I've missed later. So far my hand-composed package list is:
<themill> !aptitude clone
<dpkg> To clone a Debian machine using aptitude (or install your favourite
packages) use
aptitude search -F '%100p' '~i!~M' > package_list;
on the reference machine;
xargs aptitude --schedule-only install < package_list; aptitude install;
on the other machine.
This preserves information about "automatically installed" packages that other methods
do not. See also <reinstall>, <things to backup>. <debian clone>
<puff> !reinstall
<dpkg> methinks reinstall is
aptitude reinstall '~i' ;
or
COLUMNS=200 dpkg -l | awk '/^[hi]i/{print $2}' | xargs apt-get -y --reinstall install
or
dpkg --get-selections > my_packages.txt
then later,
dpkg --set-selections < my_packages.txt && apt-get install
See also <aptitude clone> <debian clone>
<puff> !things to backup
<dpkg> A list of some of the things you should back up on your box is:
/etc
/home
/root
/usr/local
/usr/src
/opt
/srv.
Tailor this list to your own purposes. If you think you don't need /var,
make sure you don't forget
/var/lib/dpkg
/var/lib/apt*
/var/lib/mysql
/var/spool/mail
/var/www
/var/cache/debconf ...<puff> !debian clone [16:27-12/07]
<dpkg> One method of cloning Debian installs is to take a current Debian
machine that is set up with the packages you want and run the command
"dpkg --get-selections > ~/selectionfile". Then, after the base
install on other machines use that file and do: "dpkg --set-selections
< ./selectionfile && apt-get dselect-upgrade". Also ask me about
<aptitude clone>, <reinstall>, <things to backup>.
This should be already handled when ryncing /home, but check, to make sure each user in /home has a /home/username/Maildir.
On my system, I have postfix configured to deliver messages into a /home/username/Maildir. This works fine, although it has two slight drawbacks. One is that I have to create a shell account for each web mail user, even if they never log in. The other is that I have to make sure the account has a Maildir created before they try to use the webmail interface, or Squirrelmail will spaz out when they try to log in and it can't find the Maildir. This is easy enough to do by just sending a test email to the new user, which will cause Postfix to create /home/username/Maildir.
The following are all just sketched in with "to be done". It's more to make sure I remember what needs to be done, than anything else, because each topic certainly has tons of other tutorials out there.
My bind server is also responsible for providing secondary DNS for a few other domains, so I have to remember to check the bind configs and insert those domains into the new machine's bind configs.
There will, of course, be cron jobs added by various packages. These should be added on the new machine by the normal package installation process.
But check for custom cron jobs and make sure you copy them over. I know I have a couple, but they may or may not be necessary on the new box.
I have postfix set up to deliver emails into /home/username/Maildir.
Earlier I installed apache2 by simply entering:
$ sudo aptitude install apache2
Now I need the rest, mysql and php and all that jazz.
puff@darksleep:~$ sudo aptitude install mysql-server mysql-client
[sudo] password for puff:
The following NEW packages will be installed:
heirloom-mailx{a} libdbd-mysql-perl{a} libdbi-perl{a}
libhtml-template-perl{a} libmysqlclient16{a} libnet-daemon-perl{a}
libplrpc-perl{a} mysql-client mysql-client-5.1{a} mysql-common{a}
mysql-server mysql-server-5.1{a} mysql-server-core-5.1{a} psmisc{a}
0 packages upgraded, 14 newly installed, 0 to remove and 0 not upgraded.
Need to get 23.8 MB of archives. After unpacking 56.5 MB will be used.
Do you want to continue? [Y/n/?] y
Get:1 http://debian.lcs.mit.edu/debian/ squeeze/main mysql-common all 5.1.49-3 [70.9 kB]
Get:2 http://debian.lcs.mit.edu/debian/ squeeze/main libnet-daemon-perl all 0.43-1 [46.6 kB]
Get:3 http://debian.lcs.mit.edu/debian/ squeeze/main libplrpc-perl all 0.2020-2 [36.0 kB]
Get:4 http://debian.lcs.mit.edu/debian/ squeeze/main libdbi-perl i386 1.612-1 [879 kB]
Get:5 http://debian.lcs.mit.edu/debian/ squeeze/main libmysqlclient16 i386 5.1.49-3 [1,935 kB]
Get:6 http://debian.lcs.mit.edu/debian/ squeeze/main libdbd-mysql-perl i386 4.016-1 [138 kB]
Get:7 http://debian.lcs.mit.edu/debian/ squeeze/main mysql-client-5.1 i386 5.1.49-3 [9,605 kB]
Get:8 http://debian.lcs.mit.edu/debian/ squeeze/main psmisc i386 22.11-1 [122 kB]
Get:9 http://debian.lcs.mit.edu/debian/ squeeze/main mysql-server-core-5.1 i386 5.1.49-3 [3,937 kB]
Get:10 http://debian.lcs.mit.edu/debian/ squeeze/main mysql-server-5.1 i386 5.1.49-3 [6,528 kB]
Get:11 http://debian.lcs.mit.edu/debian/ squeeze/main heirloom-mailx i386 12.4-2 [296 kB]
Get:12 http://debian.lcs.mit.edu/debian/ squeeze/main libhtml-template-perl all 2.9-2 [65.9 kB]
Get:13 http://debian.lcs.mit.edu/debian/ squeeze/main mysql-client all 5.1.49-3 [65.2 kB]
Get:14 http://debian.lcs.mit.edu/debian/ squeeze/main mysql-server all 5.1.49-3 [65.4 kB]
Fetched 23.8 MB in 7s (3,054 kB/s)
debconf: unable to initialize frontend: Dialog
debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
debconf: falling back to frontend: Readline
Preconfiguring packages ...
Configuring mysql-server-5.1
----------------------------While not mandatory, it is highly recommended that you set a password for the
MySQL administrative "root" user.
If this field is left blank, the password will not be changed.
New password for the MySQL "root" user:
Repeat password for the MySQL "root" user:
Selecting previously deselected package mysql-common.
(Reading database ... 32677 files and directories currently installed.)
Unpacking mysql-common (from .../mysql-common5.1.49-3all.deb) ...
Selecting previously deselected package libnet-daemon-perl.
Unpacking libnet-daemon-perl (from .../libnet-daemon-perl0.43-1all.deb) ...
Selecting previously deselected package libplrpc-perl.
Unpacking libplrpc-perl (from .../libplrpc-perl0.2020-2all.deb) ...
Selecting previously deselected package libdbi-perl.
Unpacking libdbi-perl (from .../libdbi-perl1.612-1i386.deb) ...
Selecting previously deselected package libmysqlclient16.
Unpacking libmysqlclient16 (from .../libmysqlclient165.1.49-3i386.deb) ...
Selecting previously deselected package libdbd-mysql-perl.
Unpacking libdbd-mysql-perl (from .../libdbd-mysql-perl4.016-1i386.deb) ...
Selecting previously deselected package mysql-client-5.1.
Unpacking mysql-client-5.1 (from .../mysql-client-5.15.1.49-3i386.deb) ...
Selecting previously deselected package psmisc.
Unpacking psmisc (from .../psmisc22.11-1i386.deb) ...
Selecting previously deselected package mysql-server-core-5.1.
Unpacking mysql-server-core-5.1 (from .../mysql-server-core-5.15.1.49-3i386.deb) ...
Processing triggers for man-db ...
debconf: unable to initialize frontend: Dialog
debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
debconf: falling back to frontend: Readline
Setting up mysql-common (5.1.49-3) ...
Selecting previously deselected package mysql-server-5.1.
(Reading database ... 33096 files and directories currently installed.)
Unpacking mysql-server-5.1 (from .../mysql-server-5.15.1.49-3i386.deb) ...
debconf: unable to initialize frontend: Dialog
debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
debconf: falling back to frontend: Readline
Selecting previously deselected package heirloom-mailx.
Unpacking heirloom-mailx (from .../heirloom-mailx12.4-2i386.deb) ...
Selecting previously deselected package libhtml-template-perl.
Unpacking libhtml-template-perl (from .../libhtml-template-perl2.9-2all.deb) ...
Selecting previously deselected package mysql-client.
Unpacking mysql-client (from .../mysql-client5.1.49-3all.deb) ...
Selecting previously deselected package mysql-server.
Unpacking mysql-server (from .../mysql-server5.1.49-3all.deb) ...
Processing triggers for man-db ...
debconf: unable to initialize frontend: Dialog
debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
debconf: falling back to frontend: Readline
Setting up libnet-daemon-perl (0.43-1) ...
Setting up libplrpc-perl (0.2020-2) ...
Setting up libdbi-perl (1.612-1) ...
Setting up libmysqlclient16 (5.1.49-3) ...
Setting up libdbd-mysql-perl (4.016-1) ...
Setting up mysql-client-5.1 (5.1.49-3) ...
Setting up psmisc (22.11-1) ...
Setting up mysql-server-core-5.1 (5.1.49-3) ...
Setting up mysql-server-5.1 (5.1.49-3) ...
debconf: unable to initialize frontend: Dialog
debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
debconf: falling back to frontend: Readline
Stopping MySQL database server: mysqld.
110515 6:54:36 [Note] Plugin 'FEDERATED' is disabled.
110515 6:54:36 InnoDB: Started; log sequence number 0 44233
110515 6:54:36 InnoDB: Starting shutdown...
110515 6:54:42 InnoDB: Shutdown completed; log sequence number 0 44233
insserv: warning: script 'moo.sh' missing LSB tags and overrides
Starting MySQL database server: mysqld.
Checking for corrupt, not cleanly closed and upgrade needing tables..
Setting up heirloom-mailx (12.4-2) ...
update-alternatives: using /usr/bin/heirloom-mailx to provide /usr/bin/mailx (mailx) in auto mode.
Setting up libhtml-template-perl (2.9-2) ...
Setting up mysql-client (5.1.49-3) ...
Setting up mysql-server (5.1.49-3) ...
puff@darksleep:~$ sudo /etc/init.d/apache2 restart
sudo /etc/init.d/apache2 restart
Restarting web server: apache2 ... waiting .
puff@darksleep:~$ sudo aptitude install php5 libapache-mod-php5
sudo aptitude install php5 libapache-mod-php5
No candidate version found for libapache-mod-php5
No candidate version found for libapache-mod-php5
The following NEW packages will be installed:
apache2-mpm-prefork{ab} libapache2-mod-php5{a} libonig2{a} libqdbm14{a}
php5 php5-cli{a} php5-common{a} php5-suhosin{a}
0 packages upgraded, 8 newly installed, 0 to remove and 0 not upgraded.
Need to get 6,654 kB of archives. After unpacking 17.3 MB will be used.
The following packages have unmet dependencies:
apache2-mpm-prefork: Conflicts: apache2-mpm which is a virtual package.
apache2-mpm-worker: Conflicts: apache2-mpm which is a virtual package.
The following actions will resolve these dependencies:
Remove the following packages:
1) apache2-mpm-worker
Accept this solution? [Y/n/q/?] y
The following NEW packages will be installed:
apache2-mpm-prefork{a} libapache2-mod-php5{a} libonig2{a} libqdbm14{a}
php5 php5-cli{a} php5-common{a} php5-suhosin{a}
The following packages will be REMOVED:
apache2-mpm-worker{a}
0 packages upgraded, 8 newly installed, 1 to remove and 0 not upgraded.
Need to get 6,654 kB of archives. After unpacking 17.3 MB will be used.
Do you want to continue? [Y/n/?] y>
Get:1 http://debian.lcs.mit.edu/debian/ squeeze/main apache2-mpm-prefork i386 2.2.16-6 [2,276 B]
Get:2 http://debian.lcs.mit.edu/debian/ squeeze/main libonig2 i386 5.9.1-1 [134 kB]
Get:3 http://debian.lcs.mit.edu/debian/ squeeze/main libqdbm14 i386 1.8.77-4 [147 kB]
Get:4 http://debian.lcs.mit.edu/debian/ squeeze/main php5-common i386 5.3.3-7 [543 kB]
Get:5 http://debian.lcs.mit.edu/debian/ squeeze/main libapache2-mod-php5 i386 5.3.3-7 [2,888 kB]
Get:6 http://debian.lcs.mit.edu/debian/ squeeze/main php5 all 5.3.3-7 [1,048 B]
Get:7 http://debian.lcs.mit.edu/debian/ squeeze/main php5-cli i386 5.3.3-7 [2,858 kB]
Get:8 http://debian.lcs.mit.edu/debian/ squeeze/main php5-suhosin i386 0.9.32.1-1 [80.8 kB]
Fetched 6,654 kB in 2s (2,919 kB/s)
debconf: unable to initialize frontend: Dialog
debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
debconf: falling back to frontend: Readline
dpkg: apache2-mpm-worker: dependency problems, but removing anyway as you requested:
apache2 depends on apache2-mpm-worker (= 2.2.16-6) | apache2-mpm-prefork (= 2.2.16-6) | apache2-mpm-event (= 2.2.16-6) | apache2-mpm-itk (= 2.2.16-6); however:
Package apache2-mpm-worker is to be removed.
Package apache2-mpm-prefork is not installed.
Package apache2-mpm-event is not installed.
Package apache2-mpm-itk is not installed.
(Reading database ... 33212 files and directories currently installed.)
Removing apache2-mpm-worker ...
Stopping web server: apache2 ... waiting .
Selecting previously deselected package apache2-mpm-prefork.
(Reading database ... 33208 files and directories currently installed.)
Unpacking apache2-mpm-prefork (from .../apache2-mpm-prefork2.2.16-6i386.deb) ...
Setting up apache2-mpm-prefork (2.2.16-6) ...
Starting web server: apache2.
Selecting previously deselected package libonig2.
(Reading database ... 33213 files and directories currently installed.)
Unpacking libonig2 (from .../libonig25.9.1-1i386.deb) ...
Selecting previously deselected package libqdbm14.
Unpacking libqdbm14 (from .../libqdbm141.8.77-4i386.deb) ...
Selecting previously deselected package php5-common.
Unpacking php5-common (from .../php5-common5.3.3-7i386.deb) ...
Selecting previously deselected package libapache2-mod-php5.
Unpacking libapache2-mod-php5 (from .../libapache2-mod-php55.3.3-7i386.deb) ...
Selecting previously deselected package php5.
Unpacking php5 (from .../archives/php55.3.3-7all.deb) ...
Selecting previously deselected package php5-cli.
Unpacking php5-cli (from .../php5-cli5.3.3-7i386.deb) ...
Selecting previously deselected package php5-suhosin.
Unpacking php5-suhosin (from .../php5-suhosin0.9.32.1-1i386.deb) ...
Processing triggers for man-db ...
debconf: unable to initialize frontend: Dialog
debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
debconf: falling back to frontend: Readline
Setting up libonig2 (5.9.1-1) ...
Setting up libqdbm14 (1.8.77-4) ...
Setting up php5-common (5.3.3-7) ...
Setting up libapache2-mod-php5 (5.3.3-7) ...
debconf: unable to initialize frontend: Dialog
debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
debconf: falling back to frontend: Readline
Creating config file /etc/php5/apache2/php.ini with new version
Reloading web server config: apache2.
Setting up php5 (5.3.3-7) ...
Setting up php5-cli (5.3.3-7) ...
debconf: unable to initialize frontend: Dialog
debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
debconf: falling back to frontend: Readline
Creating config file /etc/php5/cli/php.ini with new version
update-alternatives: using /usr/bin/php5 to provide /usr/bin/php (php) in auto mode.
Setting up php5-suhosin (0.9.32.1-1) ...
debconf: unable to initialize frontend: Dialog
debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.)
debconf: falling back to frontend: Readline
puff@darksleep:~$ sudo /etc/init.d/apache2 restart
sudo /etc/init.d/apache2 restart
Restarting web server: apache2 ... waiting .
puff@darksleep:~$ cd /var/www/
puff@darksleep:/var/www$
Make a little PHP file that contains:
<?php phpinfo() ; ?>
Save it as /var/www/test.php, and load it with your browser to make sure PHP is working correctly.
In debian this is handled via /etc/apache2/sites-available and /etc/apache2/sites-enabled. Sites-available holds the virtual domain config files, and sites-enabled holds links to the files in sites-available. The presence of the link turns on the domain.
Also, compare the apache config files and see if I made any tweaks to the default configs, like enabling .htaccess (if that's not enabled by default).
I already have the apache virtual domains configured in my old sites-available, which I have a copy of in my backup of the old box's /etc, on the new box in /home/darksleep/etc. All I have to do is copy the config files over. In fact, since the location of /etc/apache2/sites-enabled is the same on both boxes, I can just copy the links over, as well.
puff@darksleep:/var/www$ cd /etc/apache2/ puff@darksleep:/etc/apache2$ ls apache2.conf conf.d envvars httpd.conf magic mods-available mods-enabled ports.conf sites-available sites-enabled puff@darksleep:/etc/apache2$ sudo cp -i /home/darksleep/etc/apache2/sites-available/* /etc/apache2/sites-available/ cp: overwrite `/etc/apache2/sites-available/default'? n puff@darksleep:/etc/apache2$ sudo cp -i /home/darksleep/etc/apache2/sites-enabled/* /etc/apache2/sites-enabled/ cp: `/home/darksleep/etc/apache2/sites-enabled/000-default' and `/etc/apache2/sites-enabled/000-default' are the same file puff@darksleep:/etc/apache2$
Now I restart apache...
puff@darksleep:/etc/apache2$ sudo /etc/init.d/apache2 restart Syntax error on line 8 of /etc/apache2/sites-enabled/00.darksleep.com: Invalid command 'RewriteEngine', perhaps misspelled or defined by a module not included in the server configuration Action 'configtest' failed. The Apache error log may have more information. failed! puff@darksleep:/etc/apache2$ sudo ln -s mods-available/rewrite.load mods-enabled/rewrite.load
Whoops, need to enable mod_rewrite. This works pretty much the same as sites-available/sites-enabled, only you use mods-available/mods-enabled:
puff@darksleep:/etc/apache2$ sudo ln -s /etc/apache2/mods-available/rewrite.load /etc/apache2/mods-enabled/rewrite.load puff@darksleep:/etc/apache2$ sudo /etc/init.d/apache2 restart Syntax error on line 2 of /etc/apache2/sites-enabled/00.darksleep.com_ssl: Invalid command 'SSLEngine', perhaps misspelled or defined by a module not included in the server configuration Action 'configtest' failed. The Apache error log may have more information. failed! puff@darksleep:/etc/apache2$
Okay, another error, we're missing SSL.
Well first, we have to enable the module. This time I'll use a2enmod instead of creating the link myself. As far as I know this does exactly the same thing.
puff@darksleep:/etc/apache2$ sudo a2enmod ssl Enabling module ssl. See /usr/share/doc/apache2.2-common/README.Debian.gz on how to configure SSL and create self-signed certificates. Run '/etc/init.d/apache2 restart' to activate new configuration! puff@darksleep:/etc/apache2$
Now we have to generate an SSL cert. I'm not going to purchase a cert for a personal server, so we'll self-certify:
NOTE: These days you can get free SSL certs from a number of widely recognized CAs, and the more popular browsers are a lot more skeptical about self-signed certs, so I highly recommend doing that.
puff@darksleep:/etc/apache2$ sudo make-ssl-cert /usr/share/ssl-cert/ssleay.cnf /etc/ssl/private/darksleep.com.crt debconf: unable to initialize frontend: Dialog debconf: (Dialog frontend will not work on a dumb terminal, an emacs shell buffer, or without a controlling terminal.) debconf: falling back to frontend: Readline Configure an SSL Certificate. -----------------------------Please enter the host name to use in the SSL certificate.
It will become the 'commonName' field of the generated SSL certificate.
Host name: darksleep.com
puff@darksleep:/etc/apache2$
This next bit gets a bit tricky. It looks like some things changed between my old release and Squeeze. It appears ssl certs are commonly kept in /etc/ssl/private now, instead of in /etc/apache2/ssl. So I have to edit /etc/ssl/private/darksleep.com.crt and split the contents into two separate files.
The private key portion goes into a file named darksleep.com.key, and I chmod it to be readable only to root:
puff@darksleep:~$ sudo chmod go-r /etc/ssl/private/darksleep.com.key
The cert portion goes into a file named darksleep.com.pem.
Now I have to edit all of my sites-available entries and make them point to the new files in /etc/ssl/private, instead of the old files /etc/apache2/ssl, and then I can restart apache2 successfully.
As with apache virtual domains, an MTA virtual domain is where the MTA server is expected to handle incoming email for domains other than the primary domain.
Most databases install with ODBC limited only to local traffic. Make sure you have that opened if you need it. Consult the docs for your particular databases.
Most databases install with no root password. Debian has tightened up on this, so now dpkg prompts you for a root password when installing mysql, but still.... Make sure you set a database root password.
Import any databases from your old machine; make sure you issue the right GRANT statements.
It's possible to copy the entire set of binary database files over. This is not recommended however. It's theoretically possible if you shut down the old database cleanly, and if the database versions match closely enough, but really, why borrow the headaches?
Instead, just use your database's standard export and import features. For MySQL, for example, that's mysqldump.
Found several almost-comprehensive posts and docs on this.
http://mail.python.org/pipermail/mailman-users/2007-January/055211.html
http://mail.python.org/pipermail/mailman-users/2007-January/055208.html
http://www.mail-archive.com/mailman-users@python.org/msg46099.html
http://www.mail-archive.com/mailman-developers@python.org/msg03127.html
http://wiki.list.org/display/DOC/How+do+I+move+a+list+to+a+different+server-Mailman+installation.
http://www.debian-administration.org/article/Migratingmailmanlists
For the most part, it seems quite simple (copying the contents of a few directories over), but there seem to be some passing references to things that aren't clearly explained, like fix_urls. It's also not clear whether these are important, or not.
Note: msapiro on freenode's #mailman was kind enough to explain: "fixurl is necessary step if either the web host name or email host name or URL pattern changed. So you should run 'bin/withlist -l -a -r fixurl' after you put the new lists in."
Annoyingly enough, the debian-administration.org article (last link in the list above) doesn't match the debian mailman layout, but it appears that most of mailman is still under /var/lib/mailman.
Incoming mail: There's one gotcha, which is incoming messages getting stuck in the mailman queue during the transition. Most of my lists are low enough traffic that I don't have to worry about that anyway. But just in case, I temporarily shut down postfix on my old server for the move, which helps me avoid incoming messages during the move. Mail servers trying to send messages should just retry in a little while.
This message suggests setting your mail server to accept incoming mail from localhost only, which will stop incoming mail, but allow outgoing mail to trickle out until the queue is empty:
http://www.mail-archive.com/mailman-developers@python.org/msg03127.html
Step 1, of course, is to back up the old data, which I already did when I made a backup copy of /var. Also, since I made a backup copy of /etc, I have a backup copy of both /etc/mailman/mm_cfg.py and /etc/aliases.
Step 2, is to install mailman normally, using apt. Early on, in the larger migration process, I used apt-get --get-selections and apt-get --set-selections to do a bulk install of all of the packages that the old box was moving, which includes mailman.
Step 3, make a backup copy of your new mailman installation, just in case. This is particularly important because one of the changes below is going to overwrite the new mailman install's site-wide configs. You'll need to be able to undo this if something goes wrong.
puff@darksleep:~$ cp -a /var/lib/mailman /home/mig/newmailmanbackup
Step 4, now we get to the nitty gritty, copying the data around. Fortunately, since we're staying on the same distro, the files are pretty much in the same place. Also, according to this:
http://www.mail-archive.com/mailman-users@python.org/msg46099.html
It should not be necessary to install the "same" version and then upgrade. In this example, it should be OK to just install Mailman 2.1.9 directly on the new system.Mailman is aware enough to update a newly encountered, older version config.pck (or even config.db from 2.0.x) to the current format. A lot of what bin/update does when you update to a new release is stuff that Mailman will do on the fly when you drop an 'old' list into a working Mailman, or it is generic stuff having to do with file locations, queue entry formats and other things not directly relevant to a list.
Thus, it is normally just fine to drop a 2.1.4 config.pck into a working 2.1.9 installation.
One thing to bear in mind is that the last copy command in the list below will overwrite the site-wide configs. This includes the site-wide password, so make sure you have a backup, just in case.
Copy the lists directory:
puff@darksleep:~$ sudo cp -a /home/mig/var/lib/mailman/lists/* /var/lib/mailman/lists/*
Copy the public archives:
puff@darksleep:~$ sudo cp -a /home/mig/var/lib/mailman/archives/public/* /var/lib/mailman/archives/public/*
Copy the private archives:
puff@darksleep:~$ sudo cp -a /home/mig/var/lib/mailman/archives/private/* /var/lib/mailman/archives/private/*
Copy the site-wide configs:
puff@darksleep:~$ sudo cp -a /home/mig/var/lib/mailman/data/sitelist.cfg /var/lib/mailman/data/sitelist.cfg
Okay, now the even nittier-grittier:
Hand-merge in the mailman aliases, if necessary, from /home/mig/etc/aliases to /etc/aliases. I've heard that mailman can now handle this with a single over-arching alias and do all the list-specific alias handling inside mailman. However, I don't know the details, and I can't find anybody on #mailman to ask about them, but I do remember that leaving the old, detailed aliases in place should be safe, so that's what we'll do.
Or you can just run genaliases, which will generate a complete set of aliases for you, and append the output to /etc/aliases:
puff@darksleep:~$ sudo /var/lib/mailman/bin/genaliases >> /etc/aliases
Don't forget to run newaliases to regenerate the aliases DB:
puff@darksleep:~$ sudo newaliases
Now hand-merge any necessary changes from /home/mig/etc/mailman/mmcfg.py into /etc/mailman/mmcfg.py. In my case, I did a diff to see if there were any differences:
$ sudo diff /home/mig/etc/mailman/mm_cfg.py /etc/mailman/mm_cfg.py 59c59 < DEFAULT_URL_PATTERN = 'http://%s/cgi-bin/mailman' --- > DEFAULT_URL_PATTERN = 'http://%s/cgi-bin/mailman/' 95a96,101 > #------------------------------------------------------------- > # Uncomment if you want to filter mail with SpamAssassin. For > # more information please visit this website: > # http://www.daa.com.au/~james/articles/mailman-spamassassin/ > # GLOBAL_PIPELINE.insert(1, 'SpamAssassin') >
Hm, that doesn't seem like a hugely important difference, so I'm going to skip it and see how it goes.
Note: msapiro later informed me that DEFAULTURLPATTERN should always end with a trailing /, and that since the mailman install my old lists were under didn't, but the new install does, I should run fix_urls to update the lists URLs, even though the hostname didn't change.
Finally, restart mailman, then send a test message through and... no (sad horn music plays here). Not only did the message not get through, the bounce didn't get through: