netcat is the natural choice to send/receive raw data to a network. But netcat outputs just a byte count at EOD. Pipe View came to help. This handy tool can be put into any shell pipe and visualizes the amount of data going through the pipe. Practice:

On the “server” type into your shell (Please note that 1337 is the port number. You can use another port. If you do so, do it on both sides. :) ):

$ nc -l 1337 | pv

On the “client” run:

$ cat /dev/zero | nc ip.of.ser.ver 1337

This will give you the following output on the server:

5,27GB 0:03:22 [  24MB/s] [<=>

If you found this hint useful please leave me a comment.

Further information:

• Netcat: http://netcat.sourceforge.net/
• Pipe View: http://www.ivarch.com/programs/pv.shtml

This attack vector is not validated by me yet. I did not try it out but I am sure it would work. If not, please let me know in the comments.

The entry point to an encrypted system is the unencrypted /boot partition. Parts of the boot loader, the kernel image and often an initial ram disk image (initrd) are stored there.

To install a backdoor in the to be attacked system, initrd is the key. While one also could use the kernel itself, using initrd is much simpler. The initrd image contains a couple of scripts that are ran when the kernel boots up and AFTER the disk was decrypted.

Before the attacker can start, she needs physical access to the system or storage. That can either be done by removing e.g. the hard disk from the system and mount it with other hardware or, if possible, to boot the system with some live system.

Now the attacker must mount the partition and copy over initrd file. In Fedora Linux the image is an gzipped cpio archive. This might be different in other distributions.

The final steps are to modify some boot script to add an password to the LUKS system. Now if the attacked user boots up the system he will add an password known to the attacker to the system. Next time the hacker gets access to the computer, she can steal some data. The modified initrd must be put back in place of course.

There are pratically no ways to protect yourself from this since as long one has access to the /boot partition, she controls almost everything in the system. TPM based signatures of the kernel and initrd could be helpful, but I don’t know much about that stuff.

As stated before I did not try it out yet. I know one thing or two about Linux and so I am pretty sure this would work. Feel free and try it out. Please tell me about your results in the comments.

Mind like water, I thought. I went into myself and thought what could have happened. There came up some facts:

• Some web app generates charts using jfreechart
• jfreechart uses AWT to generate the graphics
• Java AWT is native code that uses the unterlying graphics subsystem, on Linux the nearest X server
• I was connected using ssh with ForwardX11 on

Like Jerlock Holmes I combined the facts and came up with a solution:

unset DISPLAY

I put this statement into the Tomcat start script (before the start of Java of course). This will break the chain between my local X-Server and Java.

The server is still running well.

Don´t get me wrong. I highly recommend to use OpenJDK if you don´t feel to need otherwise. But being a Java developer myself I find some things really annoying, so I still stick to Sun´s version of the great Java:

• Speed – For example Netbeans runs way faster on Sun´s Java than on OpenJDK. Being my workplace for most of the time, that really matters
• Font rendering – In Gnome and Fedora Netbeans looks not that well on OpenJDK, because anti-aliasing seems not to work too well. On Sun´s JDK in contrast…
• Memory management seems to be a bit better on Sun´s JDK at the moment
• Nimbus Look and Feel – I just like it

Sun (Oracle by now) offers two flavors of the JDK to download – one using RPM under the cover and one using cpio. Both come inside a big shell script aka self extracting archive that forces you to agree the licence before allowed to install.

Although there is a RPM version, it isn´t integrated into the operating system very tightly. On Fedora and derived distributions (I will only mention Fedora from here) there is that great “alternatives” system (actually it came from Debian, as far as I recall). This is a text file database, some scripts and a lot of symlinks. It lets several pieces of software share common names for programs and gives the user the choice which one to use.

On Fedora OpenJDK and GCJ share a lot of files, e.g. /usr/bin/java, /usr/bin/javac, etc. The Sun RPMs extract the files to some non-standard place, /usr/java/*. To make Sun´s JDK the system default, one has to change a lot of files and links, or use the alternatives system.

Since there are still plenty of files to provide alternatives for, I wrote a little script that will do the work. I tested it with Sun´s JDK 1.6.0_18 (RPM) on Fedora Core 12, but it should work with all versions from the Java 1.6.0 stem on Fedora and derived distributions.

One word of warning: The script is provided AS IS. I do not take any responsibility for whatever happens to you, your computer, etc. due using this script You use the script on YOUR OWN RISK.

The script is released to the public domain. Do what you want but don´t blame me. :-) If you like, please leave a comment if you liked it or have issues/patches.

So where is this §$%& script you are talking about? Click here to download the script.

Stay calm, I finally figured it out. While fiddling around I found two very nice things in the internet that I want to share with you.

First, I learned how to debug the SSL problems on Sun´s JSSE documentation. You will find a very comprehensible example of a Java SSL debug session.

portecle screenshot, stolen from http://portecle.sourceforge.net

But the real gem I found was Portecle, a small, overlooked utterly useful tool I wouldn´t ever remove from my tool chain. Portecle is a Swing-based frontend to JSSE keystores (it is a replacement for keytool). It actually works (In contrast to many other outdated or useless tools that fill the same gap) and beside many other things, it enables you to import chained certificate files. Check it out.

Fortunally I found out about ”ALTER table ENGINE=…;”.

Since I am very lazy I created myself a little helper, that generates per MyISAM table a matching ALTER TABLE statement. So all I had to do was to copy the result of the query back to mysql and lean back.

The statement I talk about is:

select concat('ALTER TABLE `', table_schema, '`.`', table_name, '` ENGINE InnoDB;')
from information_schema.tables where table_schema <> 'mysql'
AND table_schema <> 'information_schema'
AND engine = 'MyISAM';

It queries the information_schema for MyISAM tables outside mysql (the database) and outputs a corresponding statement.

If you found that useful, please leave me a comment.

Since 6.0 java defaults to allowing connects to running JVM-processes on the same computer. Fortunally this doesn´t apply to remote connects. Unfortunally jconsole just quits with a “Connection failed”.

To get jconsole to work from a remote computer, some properties must be set on start of the JVM:

java -Dcom.sun.management.jmxremote.local.only=false -Dcom.sun.management.jmxremote JMX Auth Settings...

There are different documents for Java 5.0 and Java 6.0. Please consult them for the authentication settings.

I found the final clue at Oli’s Blog.

Below are some screenshots of this awesome application.

Bitte was? Liebe Nerds, die mit dem Gedanken spielen, Informatik zu studieren, bitte sucht euch eine andere Uni. In Passau weiß man offensichtlich echtes Interesse am Fach und totalen Einsatz, für das was man liebt, nicht zu schätzen. Dafür steht Nerdtum nämlich oder hat das den Schlaubergern aus dem Hochschulmarketing keiner gesagt? Oder will man lieber ein paar lernschwache Halbaffen als Studenten? Nur zur Info für die Provinzler der deutschen akademischen Szene: Die Aushängeschilder der US-Uni, denen man sonst mit glühenden Bäckchen nacheifert, sind meist Nerds, nämlich Leute, die sich voll auf ihr Forschungsgebiet konzentrieren und nicht darauf, möglichst cool daherzukommen und den Jahrgangsrekord der facebook-Kontakte zu knacken.

The no-nerd-campaign of Passau’s IT department totally ruined my lunch break today. Are they trying to keep the nerds away from their department or what? Does that mean, they don’t need any IT students or that they rather want mentally retarded jocks instead of people who devote most of their time and energy to study a subject they love? Let’s just hope that IT students in Passau react appropriately.

If you are interested in questions of religion, or atheism and fan of Richard Dawkins for that matter, please watch this excellent debate between Hitchens and Rabbi Boteach:

Freeing Hardware from the dictat of the vendor at last became mainstream since the iPhone. In this article I guide through the process of jailbreaking a LaCie Big Ethernet Disk NAS applicance. Beside the fun you get a nice little Linux server you can make something great with.

Das Gerät enthält zwei oder mehr Festplatten und einen Linux 2.6 basierten Server, der auf einer ARM-CPU läuft. Dazu kommen noch 124MB RAM.

The device contains two or more hard disks and a Linux 2.6 based server running on an ARM cpu. Further it contains 124MB RAM.

Der Zugriff auf das Gerät erfolgt wahlweise über ein Web-basiertes Admin-Interface, SMB, AFP, HTTP oder FTP. Die Adminoberfläche enthält auch einen einfachen Dateibrowser. Benutzer und Gruppen können wahlweise lokal angelegt werden oder über eine Active Directory Integration angebunden werden. Die AD-Integration, welche hier leider nicht wirklich funktioniert hat, war der Hauptgrund, warum ich mich überhaupt mit dem Gerät angelegt habe.

The device can be accessed using either a web based admin interface or SMB, AFP, HTTP or FTP. The admin interface also contains a simple file browser. Users and groups can be created locally or received from an Active Directory. Unfortunally the Active Directory integration didn´t worked for me. So I scratched my fingers and started hacking a bit.

Der folgende Prozess hat mich dazu in die Lage versetzt, per ssh als root auf das Gerät zuzugreifen und damit das volle Potential des “kleine” Linux-Servers zu nutzen. Ich distanziere mich hier ausdrücklich von allen Schäden, die durch das Befolgen dieser Anleitung entstehen. Bei mir hat´s funktioniert. Auf jeden Fall sollte ein Backup aller Daten erstellt werden.

The following process lead me to an ssh login which is usable as as root which enables me to use the full potential of this “little” Linux server. I am not responsible for any data loss, malfunction or any other inconvinience you encounter by following this guide. It worked for me, it might not work for you as well. Be sure to have a backup of all your data.

Bevor es losgeht, muss man sich mit der Adminoberfläche vertraut machen. Die wesentlichen Hauptmenüpunkte für den Hack sind Benutzer (Users), Freigaben (Shares) und Wartung (System).

Before we start, lets have a quick look on the admin interface. For our purposes we need the main menu entries Users, Shares and System.

Im ersten Schritt verschafft man sich Zugriff zum Root-Verzeichnis des installierten Linux. Dies wird möglich durch eine Schwachstelle in der Eingabevalidierung und der Tatsache dass der Webserver und die CGI-Skripte mit den Rechten von root läuft. Und so geht´s:

1. Ein neues Share mit dem Namen “Hack” anlegen (Freigaben -> Hinzufügen -> Name: “Hack”, alle Checkboxen aus)
2. Konfiguration als XML-Datei herunterladen (System -> Wartung -> Konfiguration speichern)
3. Bearbeiten der heruntergeladenen Datei, siehe Screenshot
4. Geänderte Konfigurationsdatei hochladen (System -> Wartung -> Konfiguration laden)
5. Dateien im Webrowser anzeigen und dort das Share “Hack” auswählen (Letzer Hauptmenüpunkt, in der deutschen Übersetzung mit Anzeigefehler)
6. Staunen

The first step is to gain access to the root directory of the running Linux. This is possible through a flaw in input validation and the fact, that the webserver is running as user root. That´s how it´s done:

1. Create a new Share named “Hack” (Shares -> Add -> Name: “Hack”, all checkboxes unchecked)
2. Download configuration as XML (System -> Maintenance -> Save configuration)
3. Edit the file as shown on the screenshot
4. Upload the modified configuration file (System -> Maintenance -> Load configuration)
5. Open the integrated file browser and choose  the share “Hack” (Browse)
6. Be amazed

Im nächsten Schritt wird das System so konfiguriert, dass nach einem Neustart der SSH-Daemon gestartet und ein öffentlicher ssh-Key für den Benutzer root hinterlegt wird. Ein solcher Schlüssel muss natürlich vorher generiert werden. Alle folgenden Tätigkeiten müssen im Dateibrowser im Webbrowser (Letzer Menüpunkt) durchgeführt werden:

1. Das Verzeichnis “/etc/initng” auswählen
2. Die Datei “sshd.i” auf dem Rechner speichern und in einem Editor öffnen
3. Die Datei wie im Beispiel weiter unten gezeigt anpassen – PUBLIC SSH KEY dabei durch den eigene Key ohne Umbrüche ersetzen
4. Die Checkbox neben der Datei “sshd.i” im Webbrowser auswählen und den Button “Dateien löschen” klicken
5. Auf den Button “Datei hochladen” klicken und dann die eben geänderte Datei “sshd.i” vom Rechner hochladen
6. Das Verzeichnis “/etc/initng/runlevel” auswählen
7. Die Datei “default.runlevel” auf dem Rechner speichern und in einem Editor öffnen
8. In die letzte Zeile “sshd” (Ohne Hochkommata) hinzufügen
9. Die Checkbox neben der Datei “default.runlevel” auswählen und den Button “Dateien löschen” klicken
10. Auf den Button “Datei hochladen” klicken und dann die eben geänderte Datei “default.runlevel” vom Rechner nochladen

#!/sbin/itype
# This is a i file, used by initng parsed by install_service

service sshd/generate_keys {
 need = udev;
 env KEYGEN=/usr/bin/ssh-keygen;
 env RSA1_KEY=/etc/ssh/ssh_host_key;
 env RSA_KEY=/etc/ssh/ssh_host_rsa_key;
 env DSA_KEY=/etc/ssh/ssh_host_dsa_key;
 script start = {
 [ ! -s ${RSA1_KEY} ] && \
 ${KEYGEN} -q -t rsa1 -f ${RSA1_KEY} -C '' -N '' 2>&1 >/dev/null
 if [ ! -s ${RSA_KEY} ]
 then
 ${KEYGEN} -q -t rsa -f ${RSA_KEY} -C '' -N '' 2>&1 >/dev/null
 chmod 600 ${RSA_KEY}
 chmod 644 ${RSA_KEY}.pub
 fi
 if [ ! -s ${DSA_KEY} ]
 then
 ${KEYGEN} -q -t dsa -f ${DSA_KEY} -C '' -N '' 2>&1 >/dev/null
 chmod 600 ${DSA_KEY}
 chmod 644 ${DSA_KEY}.pub
 fi
 }
}

service sshd/addkey {
 script start = {
 mkdir -p /root/.ssh
 echo "PUBLIC SSH KEY" > /root/.ssh/authorized_keys
 chmod 600 /root/.ssh/authorized_keys
 }
}

daemon sshd {
 require_network;
 need = sshd/generate_keys sshd/addkey;
 exec daemon = /usr/sbin/sshd;
 pid_file = /var/run/sshd.pid;
 forks;
 daemon_stops_badly;
}

In the next step the system is configured to start the SSH daemon on boot time and add your public ssh key to the user root. This key must be generated first of course. Everything outlined next must be done in the filebrowser in the web browser (Last main menu entry):

1. Select the directory “/etc/initng”
2. Save the file “sshd.i” to your computer
3. Modify the downloaded file as in the example above – Replace the phrase PUBLIC SSH KEY with your actual public ssh key without line breaks
   
4. Select the checkbox next to the file “sshd.i” in the web browser and click the button “Delete Files”
5. Click button “Upload file” to upload the modified “sshd.i” from your computer to the remote disk
6. Select the directory “/etc/initng/runlevel”
7. Save the file “default.runlevel” to your computer and open it in an editor
8. Add the text “sshd” (Without the apostrophes) in the last line of the file
9. Select the checkbox next to the file “default.runlevel” and click the button “Delete Files”
10. Click the button “Upload file” to upload the modified “default.runlevel” from you computer to the remote disk

Im letzten Schritt muss das Gerät neu gestartet werden. Hierzu einfach System -> Status -> Neustart ausführen. Nach wenigen Minuten ist das Gerät wieder online. Mittels ssh kann man sich nun als “root” einloggen und das System weiter erkunden.

Finally the device must be rebooted. Click System -> Status -> Reboot. After a few minutes the box is back. Using ssh you can connect to it as “root” and investigate further.

Viel Spaß mit dem neuen Linux-Server.

Have fun with your new Linux server.

This week I tried to set up a fully automated installation for Centos 5.3 and ran into some problems.

Connect from Ubuntu 9.04 to Centos 5.3 via ssh

Connecting to a Centos 5.3 host using the installation of ssh provided by Ubuntu 9.04 should be straightforward. Unfortunally it doesn´t work out of the box because the connection seems to hang forever and times out in the end. The reason is that the sshd configuration option GSSAPIAuthentication is set to “on” by default on Centos hosts. This leads sshd to do DNS reverse lookups to Kerberos servers. If you don´t understand anything of this you can safely change the option to “no” on the Centos host and restart sshd. The problem should have went away by then. That´s it.

Mistake in Centos/Redhat Configuration

One more no-brainer: There is a mistake inside the documenation on howto set up PXE/BOOTP on RedHat/Centos. While the documentation states

allow booting; allow bootp; class
"pxeclients" { match if substring(option vendor-class-identifier, 0, 9)
= "PXEClient"; next-server <server-ip>; filename "linux-install/pxelinux.0"; }

the bold part must be /linux-install/pxelinux.0 (Mind the leading "/"). That´s it.

X11Forwarding on Centos

My Centos installation was based on an out of the box non-GUI profile. As I needed to run one of the X11-based system-config-* tools I used X11Forwarding. Unfortunally that didn´t work. All I got was

Can´t open display:

I found this bug report which asks one to install the package xorg-x11-xauth. That´s it.

Now it´s weekend. (Not really)

This week one Windows 2003 Domain Controller crashed badly and left behind a struggling network. So I made up some optimizations.

Install additional domain controllers as VMware Virtual Machine

The affected location had only one dedicated domain controller. So I chose to set up another one. Since virtualization is cool I did it using VMware Server. Windows installed flawlessly and everything went right until i started DCPROMO.EXE to get the beast into the domain. All I got was:

Active Directory Installation Wizard

The wizard cannot gain access to the list of domains in the forest.

This
condition may be caused by a DNS lookup problem. For information about
troubleshooting common DNS lookup problems, please see the following
Microsoft Web site: http://go.microsoft.com/fwlink/?LinkId=5171

The error is:

The RPC server is unavailable.

I found the solution (not the reason) here (german). I had to reinstall the VMware Tools and deselect the Shared Folders component. That´s it. It can be reinstalled after the server became a DC. Alternativly installation of VMware Tools can be delayed until the DC setup is finished.

Use more than one LDAP server with Apache httpd mod_authnz_ldap

Since I had two domain controllers now, I wanted to use them. Due historically reasons I do authentication on web services by accessing the domain controller via LDAP instead NTLM. Until the crash there was only one LDAP server in Apache config. I tried to add the new server to by entering the host names without domain (e.g. dc1 dc2) into AuthLDAPUrl directive.

Unfortunally this didn´t worked. Apache cried about format validation of LDAP URL. So I went back and entered the FQDN of the domain controllers as in:

AuthLDAPUrl ldap://dc1.domain.local dc2.domain.local/dc=domain,dc=local?sAMAccountName?sub" NONE

That´s it. BTW. Here is my full config if you mind:

AuthType Basic
AuthName "Enter your Windows credentials"
AuthBasicProvider ldap
AuthLDAPURL "ldap://dc1.domain.local dc2.domain.local/dc=domain,dc=local?sAMAccountName?sub" NONE
AuthLDAPBindDN apache@domain.local
AuthLDAPBindPassword secret
AuthLDAPGroupAttribute member
AuthLDAPGroupAttributeIsDN On
AuthzLDAPAuthoritative On

That´s it.

Scheduled Task runs as non-administrative domain user batch script

Not really related to the crash, but noteworthy. I tried to talk Windows 2003 Server into running a scheduled task as domain user with no administrative privileges that is a .bat file.

Sounds simple, doesn´t it? Windows showed itself very unruly. All I got, when I ran the script, was “Not started”. So what? Here is the solution:

First c:\Windows\System32\cmd.exe, which is interpreting batch scripts, is allowed to be run by either administrators and an ominous group named INTERACTIVE. This group is applied to any user, that is logged into the system either locally or with terminal services, but not when running as scheduled task.

So what I did was to add a group BATCHSERVICES to my domain, added the user in question to that group and changed the security settings of cmd.exe to allow members of BATCHSERVICES to read and execute cmd.exe.

Unfortually that´s not enough. To allow non-admin users to run scheduled tasks, they need to have the authorization to login into the system as batch process. So add the user to the local security policy “Log on as batch job”. That´s it.

Now it´s weekend.

There are different strategies to get around this. On Debian I use apt-proxy for this task. It is neatly integrated into the apt package management tool and works like charm most of the time.

For CentOS I wasn´t able to find such a specialized proxy. So I decided to try to set up a private mirror. It wasn´t that hard at all and this post outlines all necessary steps. I used this howto as a starting point, but found it faulty and outdated. So here is how I did it.

BE WARNED: I am not responsible for any data loss, malfunction or any other inconvinience you encounter by following this howto. It worked for me, it might not work for you as well. Be sure to have a backup of all your data.

I assume you set up the mirror on a CentOS host itself. Other host operating systems work as well, but you will have to adjust some paths. Per distribution release version and architecture you will need about 10GB of disk space, but be sure to keep it extensible (by using LVM for example). All these steps apply to CentOS 5.x, other versions of CentOS are slightly different.

Start by installing the required software (if not done yet). You will need rsync and a http server (I assume you use Apache web server here). To install everything, run

su -c 'yum install httpd rsync'

I assume here that you don´t want to use virtual hosts, so following steps apply to the standard installation of Apache. Be also sure that your mirror stays private if you want it to. See the Apache HTTP server documentation for more on securing the web server and virtual hosting.

First create the directory, that will hold all the RPM files:

su -c 'mkdir -p /var/www/html/centos/5/{os,updates}/x86_64'

Please note that you have to keep the 5 (printed bold) even you use a point release like 5.3. Adjust x86_64 to the used architecture.

Now mount the CentOS installation DVD and copy the relevant files (If you use installation media on CD-ROM, repeat these steps per disk):

su -c 'mount /dev/cdrom /mnt'
su -c 'cp -rv /mnt/CentOS /mnt/repodata /var/www/html/centos/5/os/x86_64/'
su -c 'umount /mnt'

At this point you have set up a private mirror for the distribution´s installation media. To verify it´s working, open a web browser and open http://ip.of.you.server/centos/5/os/x86_64 (Replace ip.of.you.server by the real ip). You should get either a “Forbidden” message or a file list containing two directories, repodata and CentOS. There must be no “File not found message”.

Now on the updates. Unlike the installation media they change often. So we need a way to keep our mirror in sync with the distributions update servers. rsync is the tool for the job. It scans a directory tree for changes and applies these changes to a local directory. It transfers only the delta which makes it very bandwith saving.

We will set up rsync to run once per day. To do so open up the file /etc/cron.daily/yum-repos-rsync-update as root and put the following content into it:

#!/bin/sh
rsync -avrt rsync://your.rsync.mirror.server/centos/5.3/updates/x86_64 \
--exclude=debug/ /var/www/html/centos/5/updates/ > /dev/null

Make the file executable:

su -c 'chmod 755  /etc/cron.daily/yum-repos-rsync-update'

Now run the update once manually once:

/etc/cron.daily/yum-repos-rsync-update

It will take some time and afterwards there should be some files in /var/www/html/centos/5/os/x86_64.

That was all on the server part. On all clients open the file /etc/yum.repos.d/CentOS-Base.repo and make it look like (Comment out the bold lines starting with “mirrorlist” and add the bold line starting with baseurl. Adjust ip.of.your.server to the right value):

...
[base]
name=CentOS-$releasever - Base
baseurl=http://ip.of.your.server/centos/$releasever/os/$basearch/
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=os
#baseurl=http://mirror.centos.org/centos/$releasever/os/$basearch/

gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5

#released updates
[updates]
name=CentOS-$releasever - Updates
baseurl=http://ip.of.your.server/centos/$releasever/updates/$basearch/
#mirrorlist=http://mirrorlist.centos.org/?release=$releasever&arch=$basearch&repo=updates
#baseurl=http://mirror.centos.org/centos/$releasever/updates/$basearch/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-5
...

Finally run

su -c 'yum clean all'
su -c 'yum update'

If all runs well, you have properly set up a private CentOS mirror. Congratulations!

I have the following things in sync at the moment:

• Bookmarks between computers and browsers (Firefox, Safari, Mobile Safari)
• Passwords
• Contacts
• Calendars
• Preferences
• Mail Accounts
• iTunes library
• Documents folder
• Java development tools (Eclipse, NetBeans, maven, tomcat, …)
• GPG keys
• Zotero
• OmniFocus

For all this sync magic I use

• A MobileMe subscription for bookmarks on the iPhone, passwords, contacts, calendars, preferences and mail account settings
• Foxmarks for bookmarks between Macs and Browsers (Safari and Firefox)
• Unison for all file synchronization tasks (iTunes, Documents, Java development stuff, GPG keys, Zotero)

In the following howto I outline the steps necessary to get synchronization to work for you.

Most of these steps need you to have access to an administrative account (e.g. the first you created on your Mac) to work.

BE WARNED: I am not responsible for any data loss, malfunction or any other inconvinience you encounter by following this howto. It worked for me, it might not work for you as well. Be sure to have a backup of all your data.

MobileMe

MobileMe is tightly integrated with Mac OS X and all one has to do is to get a subscription. Unfortunally MobileMe is an all or nothing offer so you cannot only buy the sync services. As soon as you added your Macs to MobileMe cloud, everything should work fine. I have activated all sync modules on one of my Macs (Your mileage may vary) and everything except Bookmark sync on the other (could be others, but I only have two macs in my lifecycle right now). If I didn´t deactivate Bookmark sync on all but one Mac, I had to accept recreation of ALL my bookmarks after every sync with Foxmarks. If I deactivate it on both (all) Macs, I wouldn´t get the iPhone in sync.

Step by step:

1. Get MobileMe subscription
2. Put your Macs and iPhone into the cloud
3. Activate all the sync services you need
4. Deactivate MobileMe bookmark sync on all but one Mac

Foxmarks

Foxmarks is a bookmark and password sync service. It is ran by a company founded by an ex-Mozilla board member. Using the Foxmarks bookmark sync service is free as in beer.

The Safari plugin is a little system extension, that adds itself to the menu bar and preference pane. Pratically it also contains an uninstaller.

For Firefox there is an extension (what else?).

Step by step:

1. Open Safari and go to http://download.foxmarks.com/download
2. Download the dmg and open it from your downloads directory
3. Run the installer
4. Choose to create an account when the wizard comes up
5. Fire up Firefox and go to https://addons.mozilla.org/de/firefox/addon/2410.
6. Click to install the extension
7. After Firefox restart a wizzard comes up. Choose to not create a new account and enter the credentials of your Foxmarks account.
8. Repeat this process on each of your Macs (and/or PC)

Now you should have bookmark sync running between all your Macs (and PCs if you mind) and your iPhone – and as a side effect the same bookmarks in Safari and Firefox.

Unison

Unison is a command line tool (Runs in Terminal) that is able to do two-way file synchronization. I use it to hold various directories in sync on my Macs. A less complex solution was to use the iDisk of MobileMe, but this would imply you hold the relevant data on that iDisk which might not be possible at all.

Although it is supposed to work, I never used it to connect more than two Macs or PCs. I also didn´t ever try to connect different operating systems (e.g. Mac OS X and Linux). But I never had problems doing sync between two hosts of the same operating systems.

MacPorts

Unison is contained in MacPorts, the Mac OS X ports collection hosted by Apple Inc. You need to install it first.

Download the disk image (.dmg) and install MacPorts following the usual steps. If you don´t like to use the Terminal to install things, download and install Porticus. It will provide you with a great GUI for MacPorts.

Now update your ports list by running

sudo port selfupdate

or in Porticus select “Ports -> MacPorts selfupdate” from the main menu.

…and back to Unison

To install Unison open Terminal or Porticus.

For Terminal type

sudo port install unison

For Porticus select “All ports” from the left pane and type “unison” into the search box. Select “unison” from the search result and click “Install” in the toolbar.

There will pop up a window showing you the available variants. Select nothing and click “install”.

Repeat these steps on your other Mac.

Configuring Unison

Final step. As template process lets create a new directory inside your personal folder on one Mac called UnisonTestDir. Copy some random files into it.

Open your user´s directory in Finder and go to Library/Application Support. There should be a directory called Unison, if not, create it.

Open any text editor (Not TextEdit) you like. Copy the following template into the editor:

# Unison preferences file
root = /Users/user/UnisonTestDir
root = ssh://user@ip.of.other.mac//Users/user/UnisonTestDir
ignore = Name .DS_Store

Replace user with your short username and ip.of.other.mac with the IP-Adress or hostname of the other computer.

Save this file as UnisonTestDir.prf inside Library/Application Support/Unison .

If you type the following into the Terminal:

unison -servercmd "/opt/local/bin/unison" UnisonTestDir

Now you should get:

Contacting server...
Warning: No xauth data; using fake authentication data for X11 forwarding.
Connected [//Localmac.local//Users/user/UnisonTestDir -> //RemoteMac.local//Users/user/UnisonTestDir]
Looking for changes
Warning: No archive files were found for these roots, whose canonical names are:
	/Users/user/UnisonTestDir
	//RemoteMac.local//Users/user/UnisonTestDir
This can happen either
because this is the first time you have synchronized these roots,
or because you have upgraded Unison to a new version with a different
archive format.

Update detection may take a while on this run if the replicas are
large.

Unison will assume that the 'last synchronized state' of both replicas
was completely empty.  This means that any files that are different
will be reported as conflicts, and any files that exist only on one
replica will be judged as new and propagated to the other replica.
If the two replicas are identical, then no changes will be reported.

If you see this message repeatedly, it may be because one of your machines
is getting its address from DHCP, which is causing its host name to change
between synchronizations.  See the documentation for the UNISONLOCALHOSTNAME
environment variable for advice on how to correct this.

Donations to the Unison project are gratefully accepted:

http://www.cis.upenn.edu/~bcpierce/unison

Press return to continue.[<spc>]

Hit the space bar and Unison will start to copy things over:

Reconciling changes

local          RemoteMac...      
dir      ---->            /  [f] ?

Hit f and y to start the sync process:

Proceed with propagating updates? [] y
Propagating updates

UNISON 2.27.57 started propagating changes at 22:43:19 on 10 Mar 2009
[BGN] Copying  from /Users/user/UnisonTestDir to //RemoteMac.local//Users/user/UnisonTestDir
[END] Copying
UNISON 2.27.57 finished propagating changes at 22:43:19 on 10 Mar 2009

Saving synchronizer state
Synchronization complete  (1 item transferred, 0 skipped, 0 failures)

Now run

unison -servercmd "/opt/local/bin/unison" UnisonTestDir

again. There should be:

Contacting server...
Warning: No xauth data; using fake authentication data for X11 forwarding.
Connected [//LocalMac.local//Users/user/UnisonTestDir -> //RemoteMac.local//Users/user/UnisonTestDir]
Looking for changes
  Waiting for changes from server
Reconciling changes
Nothing to do: replicas have not changed since last sync.

You can repeat these steps for any directory you want. Simply create a profile per directory and adjust the values. I use Unison for a whole bunch of directories including iTunes and Documents without bigger problems for years now.

You have to consider, that Unison can become very complex when it comes to synchronization conflicts. Especially you have to care about iTunes and other database-backed software, so that only one instance of it (iTunes) might be run on one computer between two synchronization processes.

Unison convenience

Because always calling

unison -servercmd "/opt/local/bin/unison" UnisonTestDir

all the time is very annoying, I built a little Bash-Script that will run unison for every existing profile:

#! /bin/sh
servercmd="/opt/local/bin/unison"
for profile in /Users/ninan/Library/Application Support/Unison/*.prf
do
	baseprofile=`basename "$profile" .prf`
	echo "Synchronizing $baseprofile"

	/opt/local/bin/unison $baseprofile -auto -servercmd "$servercmd" -ui text
done

Save this file to /Users/user/bin/syncws and make it executable. Now all you should have todo is to run syncws in Terminal every time you want to synchronize your directories.

If you found these Unison steps hard to follow because all of the shell work, it isn´t probably for you.

Conclusion

Apart from MobileMe everything is free and apart from Unison everything happens automatically (most of the time).

You should consider, that either MobileMe and Foxmarks put your data onto servers outside your reach. I consider Apple trustworthy and aware of their responsibilty (They state to encrypt your data with your MobileMe password) and my bookmarks are not that critical in terms of privacy that I wouldn´t give Foxmarks a shot.

I would like to replace the MobileMe sync services with something free and open, but this decision has to be made within a year from now since I just renewed my subscription (BTW Today I´ve found out about fruux as they started to follow me on Twitter). I will look for alternatives in the near future.

Unison, if used right and with a bit of care, is hell of a tool. I use it for years now on either Linux and Mac OS X without big pain. But since with great power comes great responsibility, you should think twice about implementing the Unison solution. Unfortunally I wasn´t yet able to find working and stable GUIs for Unison apart from the command line interface.

OmniFocus, btw. supports several synchronization strategies out of the box. I use idisk based synchronization.

3.14159265358979323846264338327950288419716939937510582097
4944592307816406286208998628034825342117067982148086513282
3066470938446095505822317253594081284811174502841027019385
2110555964462294895493038196442881097566593344612847564823
3786783165271201909145648566923460348610454326648213393607
2602491412737245870066063155881748815209209628292540917153
6436789259036001133053054882046652138414695194151160943305
7270365759591953092186117381932611793105118548074462379962
7495673518857527248912279381830119491298336733624406566430
8602139494639522473719070217986094370277053921717629317675
2384674818467669405132000568127145263560827785771342757789
6091736371787214684409012249534301465495853710507922796892
5892354201995611212902196086403441815981362977477130996051
8707211349999998372978049951059731732816096318595024459455
3469083026425223082533446850352619311881710100031378387528
8658753320838142061717766914730359825349042875546873115956
2863882353787593751957781857780532171226806613001927876611
1959092164201989380952572010654858632788659361533818279682
3030195203530185296899577362259941389124972177528347913151
5574857242454150695950829533116861727855889075098381754637
4649393192550604009277016711390098488240128583616035637076
6010471018194295559619894676783744944825537977472684710404
7534646208046684259069491293313677028989152104752162056966
0240580381501935112533824300355876402474964732639141992726
0426992279678235478163600934172164121992458631503028618297
4555706749838505494588586926995690927210797509302955321165
3449872027559602364806654991198818347977535663698074265425
2786255181841757467289097777279380008164706001614524919217
3217214772350141441973568548161361157352552133475741849468
4385233239073941433345477624168625189835694855620992192221
8427255025425688767179049460165346680498862723279178608578
4383827967976681454100953883786360950680064225125205117392
9848960841284886269456042419652850222106611863067442786220
3919494504712371378696095636437191728746776465757396241389
0865832645995813390478027590099465764078951269468398352595
7098258226205224894077267194782684826014769909026401363944
3745530506820349625245174939965143142980919065925093722169
6461515709858387410597885959772975498930161753928468138268
6838689427741559918559252459539594310499725246808459872736
4469584865383673622262609912460805124388439045124413654976
2780797715691435997700129616089441694868555848406353422072
2258284886481584560285060168427394522674676788952521385225
4995466672782398645659611635488623057745649803559363456817
4324112515076069479451096596094025228879710893145669136867
2287489405601015033086179286809208747609178249385890097149
0967598526136554978189312978482168299894872265880485756401
4270477555132379641451523746234364542858444795265867821051
1413547357395231134271661021359695362314429524849371871101
4576540359027993440374200731057853906219838744780847848968
3321445713868751943506430218453191048481005370614680674919
2781911979399520614196634287544406437451237181921799983910
1591956181467514269123974894090718649423196156794520809514
6550225231603881930142093762137855956638937787083039069792
0773467221825625996615014215030680384477345492026054146659
25201497440

– echo “scale=3141; 4*a(1)” | bc -l

This leads to the conclusion, that our existence and awareness needs these ticks of time we define as period of elementary particle processes. What is the minimal time period? Probably the time it takes to move one elementary particle the minimal distance. What´s the minimal distance? I don´t know. Perhaps every particle is everywhere at every time and we only change our view? There are plenty of theories around this and I don´t want to add one more here.

What I really do think about is, what happens between two of these minimal time ticks? What if between two of them there is a very long break which happens in another time. In this OTHER time everything gets arranged for the next tick. Sounds mad, doesn´t it?

So now imagine a computer at todays levels of CPU power and incredible huge storage to hold the third dimension coordinates of all elementary particles including there features as spin, type etc. that makes up a universe equal sized to ours. Now this computer might simulate the universe by moving each elementary particle one by one inside the rules of quantum physics (ok, lets assume multicore processing and some neat algorithms). From outside the computer it will take a very long time to get things (the next tick of time) done. But inside the computer and inside the simulated universe time will run as smoothly as OURS. So does it still sound so mad to talk about time that run between ours?

A galaxy seems red

When it recedes from you.

And when it approaches

It will look nicely blue. 

– ninan

This little poem refers to the famous Doppler effect (Doppler shift) which is named after the austrian scientist Christian Doppler. In fact this is the relativistic version of the Doppler effect, which considers the effects of the special theory of relativity by Albert Einstein.