Firefox 5, Cherokee and the (Full) Firefox Sync Server

I lately wrote about setting up your own Firefox Sync / Mozilla Weave server under the Cherokee webserver.
When I recently updated Firefox on my laptop, the first thing I saw was an error from the sync function – the server could not be found. Mozilla must have changed some essential things in the sync functionality, so I almost came to terms with going back to Firefox 4 in order to maintain synced bookmarks. Luckily I did some googling and came across a nice how-to (in german) for updating the minimal FF sync server to make it work with Firefox 5 again.

The article links to the blog of Toby Elliot, who is responsible for the Minimal version of the sync server. However, he states that there will assumingly be no more new versions for the minimal server, as he suggests to use the full version, which is completely rewritten in Python (instead of PHP) and – so he claims – much easier to install. So instead of updating the minimal server I took a chance and tried to install the full version. Here’s how to do it with Cherokee:

First of all: In contrast to the installation of the minimal server, this is not primarily about juggling with regular expressions and handler rules in Cherokee, just to get the right content to the server script. That part is fairly easy. But there are some other pitfalls to run into while setting this thing up.

Prerequesites

The official Mozilla page for the Sync server describes what packages we need for the installation:

  • python-dev
  • make
  • mercurial
  • sqlite3

So under Debian we could install these via

server:/var/ffsync# sudo apt-get install python-dev mercurial sqlite3 python-virtualenv

Downloading and installing

The easiest way to get the sources (and to keep them up to date) is to have Mercurial installed on your system. The repository is at http://hg.mozilla.org/services/server-full/, so doing

server:/var/ffsync# hg clone http://hg.mozilla.org/services/server-full/

would give you the whole server at /var/ffsync/full-server/. Alternatively, you could download the sources as a zipped or tar-ed archive (e.g., as .tar.bz2) and extract that to the directory of your choice. Refer to the repository page for all the links.

The extracted source files have to be built to generate the neccessary binaries. Do so by navigating to the created directory and running

server:/var/ffsync/server-full# make build

This will (amongst others) generate a bin-directory which holds the executables for the server.

Configuring the server

The server comes with a central configuration file named development.ini. We don’t have much to configure here, but one option is important here: Look for the line

args = ('/tmp/sync-error.log',)

and change it to something like

args = ('/var/log/ffsync.error',)

That way errors get written to that file. Unfortunately Cherokee was not able to create that file itself in the /var/log-directory, so we rather create it beforehand:

server:/var/ffsync/server-full# cd /var/log
server:/var/log# touch ffsync.error
server:/var/log# chown www-data:www-data ffsync.error

where www-data should be the user/group your Cherokee runs under. Also reduce the size of the server pool running simultaneously. If you are just running this server for your own purpose, changing

[server:main]
...
threadpool_workers = 60

into

[server:main]
...
threadpool_workers = 5

should suffice.

The development.ini file also loads another configuration file located in the etc-directory, the default is sync.conf. So we go and edit etc/sync.conf and adjust the following values:

[storage]
backend = syncstorage.storage.sql.SQLStorage
sqluri = sqlite:////var/ffsync/server-full/db/storage.db
...
use_quota = false
...
 
[auth]
backend = services.auth.sql.SQLAuth
sqluri = sqlite:////var/ffsync/server-full/db/users.db
...
fallback_node = https://your.domain.here/
 
[smtp]
host = localhost
...
sender = mailaddress@from.your.domain.com

The replacements for the backend values are due to a bug that seems to be known but not yet fixed (as of now). With sqluri, we specify the sqlite3-database files we want to use for storing the payload data (such as bookmarks and browsing history) and the user credentials, respectively. The directory and the database-files don’t exist, so we again create them now in beforehand:

server:/var/ffsync/server-full# mkdir db
server:/var/ffsync/server-full# touch db/storage.db
server:/var/ffsync/server-full# touch db/users.db
server:/var/ffsync/server-full# chown -R www-data:www-data db

Setting up Cherokee

Now everything should be prepared – and the rest should be a piece of cake. Check into your Cherokee-admin panel and create a new virtual server. Choose “Manual” as the type. The “nickname” should be the subdomain you want your Sync server to be accessible at, while the “document root” is quite arbitrary – let’s choose the root of our Sync server here, so /var/ffsync/full-server in this example case.

Click on the “Behavior” tab for the new vserver and on “Rule Management“. Delete the two top handlers that have been created automatically (“Directory /cherokee_themes” and “Directory /icons”) – we won’t need them here. Click the remaining “Default”-handler, switch to the “Handler“-tab and select “None” as the handler.

Now add a new behavior by clicking the “+” button and choose “Directory” as “Rule Type” and simply / as the “Web Directory”. Select “HTTP Reverse Proxy” as the “Handler” and scroll to the very bottom of the page, where you select “Round Robin” as the “Balancer“. Problem is: We don’t have set up a data source for the HTTP reverse proxy yet, so we will do that now.

Click on “Sources” at the top of the screen, and then on the “+” sign to the left to create a new information source. The nick is arbitrary, just choose something you will recognize in a moment, like “FFSyncServer”. For the “Connection” we enter “127.0.0.1:5000” as this is where the server will be running. After you hit “Add“, change the type of the source from “Remote host” to “Local interpreter“. Finally, we add a value for the “Interpreter” field:

/var/ffsync/full-server/bin/paster serve /var/ffsync/full-sever/development.ini

where of course again you will have to adjust the path to your sync server installation. Now head back to the vserver you just created, go to the HTTP reverse proxy handler and choose the newly created information source from the dropdown menu at the very bottom on the page.

That should be it. Restart your Cherokee by clicking on “Save” in the top right corner.

Configuring Firefox

Now only Firefox has to be connected. The easiest way is when you go to your Firefox preferences and on to the “Sync” tab and choose “Deactivate this device” to disconnect from the old Sync server. The good thing is: The full server even has functionality for registering new users via the Firefox interface. :) So you can click “Set up Sync” in the Firefox menu, choose “Create new account” and enter your credentials. As the server of course select “Own/custom server” and enter your URL. If everything went well you should be presented a success message and your encryption key. And your account is set up. :)

Some drawbacks

There are some minor problems with the above set up I would not like to conceal:

  • The official documentation says:

    The built-in server should not be used in production, as it does not really support a lot of load.

    I did read that, and I really tried to get some of the other solutions going. I would have especially favored running the server as a uWSGI application, as Cherokee promotes that functionality and brings along support out of the box, but … it simply didn’t work. There is a wsgi file included especially for that purpose, but it threw numerous errors when I tried to start it with uWSGI. However, if you are like me and you are running the Sync server for your own private use, maybe even only yourself using it, I think the “paster” server might not be a problem.

  • If for some reason Firefox won’t find the server, try running the server from a command line first by making yourself the webserver-user and executing the server:
    server:/var/ffsync/server-full# su www-data
    server:/var/ffsync/server-full# /var/ffsync/full-server/bin/paster serve /var/ffsync/full-sever/development.ini

    Now try connecting again from Firefox or surf yourself to the server URL to see the responses in the console. That way I came to knowing that Cherokee had to permission to write the log file and changed the location in the config, as described above.

  • Changing the password for a user (i.e., you) didn’t work directly from Firefox for me. When I tried that, FF would report some error and could not connect to the server anymore. But I was then offered to reset my password, and when I chose it, I received an e-mail to my user-name-address with a reset link, which worked like a charm. So the full server even offers password reset functionality via e-mail and a built-in web interface. :)

Finally, if anything goes wrong or doesn’t work at all, don’t hesitate to post a comment. Personally, I found this full version was much less a hassle to install than the minimal version was, but that was probably due to personal incompetence. ;) Have fun with your fully functional webserver.


3 thoughts on “Firefox 5, Cherokee and the (Full) Firefox Sync Server

  1. This is an excellent tutorial. My only problem is that I am running Apache :( Can you point me in the direction of how to configure Apache as a back end? I’ve been looking around but most of the pages are for the older version of sync.

  2. Ok, I’ve been working on this project for a little while. I’m running Python 2.6.2 which seems to be a problem. I can’t easily upgrade this server at the moment but I will in the near future. That may fix my issues with WSGI on Apache. I did use those links and I did get the sync server working with WSGI at one point but things weren’t 100%.

    I installed Python 2.7.2 with ‘make altinstall’ to a different location. Unfortunately, I can’t get my mod_wsgi module to function properly. I’ll keep looking at it.

    For now, I have the sync server behind an Apache proxy, similar to your configuration. The following page helped me out this afternoon for Apache. I am not resolving the images on the weave-password-reset form but everything else works just fine. Maybe someone else can figure out what is missing.

    http://www.apachetutor.org/admin/reverseproxies

    I got a proxy configuation set up in about 15 minutes. I’m going to leave it alone for the moment :)

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>