I lately wrote about setting up your own Firefox Sync / Mozilla Weave server under the Cherokee webserver.
When I recently updated Firefox on my laptop, the first thing I saw was an error from the sync function – the server could not be found. Mozilla must have changed some essential things in the sync functionality, so I almost came to terms with going back to Firefox 4 in order to maintain synced bookmarks. Luckily I did some googling and came across a nice how-to (in german) for updating the minimal FF sync server to make it work with Firefox 5 again.
The article links to the blog of Toby Elliot, who is responsible for the Minimal version of the sync server. However, he states that there will assumingly be no more new versions for the minimal server, as he suggests to use the full version, which is completely rewritten in Python (instead of PHP) and – so he claims – much easier to install. So instead of updating the minimal server I took a chance and tried to install the full version. Here’s how to do it with Cherokee:
First of all: In contrast to the installation of the minimal server, this is not primarily about juggling with regular expressions and handler rules in Cherokee, just to get the right content to the server script. That part is fairly easy. But there are some other pitfalls to run into while setting this thing up.
The official Mozilla page for the Sync server describes what packages we need for the installation:
So under Debian we could install these via
server:/var/ffsync# sudo apt-get install python-dev mercurial sqlite3 python-virtualenv
Downloading and installing
server:/var/ffsync# hg clone http://hg.mozilla.org/services/server-full/
would give you the whole server at
/var/ffsync/full-server/. Alternatively, you could download the sources as a zipped or tar-ed archive (e.g., as .tar.bz2) and extract that to the directory of your choice. Refer to the repository page for all the links.
The extracted source files have to be built to generate the neccessary binaries. Do so by navigating to the created directory and running
server:/var/ffsync/server-full# make build
This will (amongst others) generate a
bin-directory which holds the executables for the server.
Configuring the server
The server comes with a central configuration file named
development.ini. We don’t have much to configure here, but one option is important here: Look for the line
args = ('/tmp/sync-error.log',)
and change it to something like
args = ('/var/log/ffsync.error',)
That way errors get written to that file. Unfortunately Cherokee was not able to create that file itself in the
/var/log-directory, so we rather create it beforehand:
server:/var/ffsync/server-full# cd /var/log server:/var/log# touch ffsync.error server:/var/log# chown www-data:www-data ffsync.error
www-data should be the user/group your Cherokee runs under. Also reduce the size of the server pool running simultaneously. If you are just running this server for your own purpose, changing
[server:main] ... threadpool_workers = 60
[server:main] ... threadpool_workers = 5
development.ini file also loads another configuration file located in the
etc-directory, the default is
sync.conf. So we go and edit
etc/sync.conf and adjust the following values:
[storage] backend = syncstorage.storage.sql.SQLStorage sqluri = sqlite:////var/ffsync/server-full/db/storage.db ... use_quota = false ... [auth] backend = services.auth.sql.SQLAuth sqluri = sqlite:////var/ffsync/server-full/db/users.db ... fallback_node = https://your.domain.here/ [smtp] host = localhost ... sender = firstname.lastname@example.org
The replacements for the
backend values are due to a bug that seems to be known but not yet fixed (as of now). With
sqluri, we specify the sqlite3-database files we want to use for storing the payload data (such as bookmarks and browsing history) and the user credentials, respectively. The directory and the database-files don’t exist, so we again create them now in beforehand:
server:/var/ffsync/server-full# mkdir db server:/var/ffsync/server-full# touch db/storage.db server:/var/ffsync/server-full# touch db/users.db server:/var/ffsync/server-full# chown -R www-data:www-data db
Setting up Cherokee
Now everything should be prepared – and the rest should be a piece of cake. Check into your Cherokee-admin panel and create a new virtual server. Choose “Manual” as the type. The “nickname” should be the subdomain you want your Sync server to be accessible at, while the “document root” is quite arbitrary – let’s choose the root of our Sync server here, so
/var/ffsync/full-server in this example case.
Click on the “Behavior” tab for the new vserver and on “Rule Management“. Delete the two top handlers that have been created automatically (“Directory /cherokee_themes” and “Directory /icons”) – we won’t need them here. Click the remaining “Default”-handler, switch to the “Handler“-tab and select “None” as the handler.
Now add a new behavior by clicking the “+” button and choose “Directory” as “Rule Type” and simply
/ as the “Web Directory”. Select “HTTP Reverse Proxy” as the “Handler” and scroll to the very bottom of the page, where you select “Round Robin” as the “Balancer“. Problem is: We don’t have set up a data source for the HTTP reverse proxy yet, so we will do that now.
Click on “Sources” at the top of the screen, and then on the “+” sign to the left to create a new information source. The nick is arbitrary, just choose something you will recognize in a moment, like “FFSyncServer”. For the “Connection” we enter “127.0.0.1:5000” as this is where the server will be running. After you hit “Add“, change the type of the source from “Remote host” to “Local interpreter“. Finally, we add a value for the “Interpreter” field:
/var/ffsync/full-server/bin/paster serve /var/ffsync/full-sever/development.ini
where of course again you will have to adjust the path to your sync server installation. Now head back to the vserver you just created, go to the HTTP reverse proxy handler and choose the newly created information source from the dropdown menu at the very bottom on the page.
That should be it. Restart your Cherokee by clicking on “Save” in the top right corner.
Now only Firefox has to be connected. The easiest way is when you go to your Firefox preferences and on to the “Sync” tab and choose “Deactivate this device” to disconnect from the old Sync server. The good thing is: The full server even has functionality for registering new users via the Firefox interface. So you can click “Set up Sync” in the Firefox menu, choose “Create new account” and enter your credentials. As the server of course select “Own/custom server” and enter your URL. If everything went well you should be presented a success message and your encryption key. And your account is set up.
There are some minor problems with the above set up I would not like to conceal:
- The official documentation says:
The built-in server should not be used in production, as it does not really support a lot of load.
I did read that, and I really tried to get some of the other solutions going. I would have especially favored running the server as a uWSGI application, as Cherokee promotes that functionality and brings along support out of the box, but … it simply didn’t work. There is a wsgi file included especially for that purpose, but it threw numerous errors when I tried to start it with uWSGI. However, if you are like me and you are running the Sync server for your own private use, maybe even only yourself using it, I think the “paster” server might not be a problem.
- If for some reason Firefox won’t find the server, try running the server from a command line first by making yourself the webserver-user and executing the server:
server:/var/ffsync/server-full# su www-data server:/var/ffsync/server-full# /var/ffsync/full-server/bin/paster serve /var/ffsync/full-sever/development.ini
Now try connecting again from Firefox or surf yourself to the server URL to see the responses in the console. That way I came to knowing that Cherokee had to permission to write the log file and changed the location in the config, as described above.
- Changing the password for a user (i.e., you) didn’t work directly from Firefox for me. When I tried that, FF would report some error and could not connect to the server anymore. But I was then offered to reset my password, and when I chose it, I received an e-mail to my user-name-address with a reset link, which worked like a charm. So the full server even offers password reset functionality via e-mail and a built-in web interface.
Finally, if anything goes wrong or doesn’t work at all, don’t hesitate to post a comment. Personally, I found this full version was much less a hassle to install than the minimal version was, but that was probably due to personal incompetence. 😉 Have fun with your fully functional webserver.