↓Skip to main content

Self-host file syncing for Zotero using WebDAV

·
Homelab

Zotero has two types of file syncing: Zotero and WebDAV. Syncing by Zotero is free up to certain capacity. For more capacity, one can pay a subscription for Zotero Storage. On the other hand, syncing by WebDAV allows one to self-host this function. If you have free space on your storage, then it is free.

Here are my requirements:

  • The software stack is free.
  • I am the only user.
  • Minimize Internet-facing footprint of the server.
  • Both the Zotero Desktop and iOS apps have access.

Here is the system diagram:

flowchart TD subgraph Laptop ZotLaptop[Zotero Desktop app] <--> TailLaptop(Tailscale) end subgraph Phone / Tablet ZotTablet[Zotero iOS app] <--> TailTablet(Tailscale) end subgraph Server subgraph Docker TailDocker(Tailscale) <--> CaddyDocker[Caddy WebDAV] end end TailLaptop <--> Int((Internet)) TailTablet <--> Int Int <--> TailDocker

Here is the repo with the docker-compose files: https://github.com/rlan/zotero-webdav-server

We need the following prerequisite:

  • A Tailscale account.
    • A service name for the WebDAV server, e.g., zstore.
    • MagicDNS name for your Tailscale cloud, e.g, magicdns.
    • An auth key.
  • A username and password pair.
  • A hashed password by Caddy.
  • A Zotero account.
  • Official Zotero desktop app.
  • Beta version of Zotero iOS app.

To generate an auth key from Tailscale:

  • Go to Admin console.
  • Settings tab.
  • Generate auth key…
  • Turn on reusable and ephemeral.
  • Click on Generate key button.

To generate the hashed password from password:

docker run -it --rm caddy:2.10.0-alpine caddy hash-password --plaintext password

Then we populate the following parameters in .env:

  • TS_AUTHKEY with the Tailscale OAuth key.
  • zstore
  • magicdns
  • username
  • hashed.password

Start the docker containers:

docker compose up -d

Verify that the WebDAV server is up:

curl -vk --netrc -X PROPFIND https://zstore.magicdns.ts.net/zotero/

Adrian Orenstein reported that WebDAV returns 404 until /srv/zotero exists. If you executed docker compose up -d already, then they shared that the following fixes the issue:

docker compose down
docker compose exec caddy mkdir -p /srv/zotero
docker compose up -d

In Zotero Desktop app, Settings, Sync tab:

  • Sign into Zotero account.
    • Choose Libraries.
    • Check “Sync automatically”.
    • Check “Sync full-text content”.
  • Check “Sync attachment files in My Library using” and choose “WebDAV”.
  • Choose “https” and enter “zstore.magicdns.ts.net” for the rest of the URL
  • Enter username for Username.
  • Enter password for Password. Note: not the Caddy-hashed password.

After entering the settings, click on the sync icon. As of writing, that is the double arrows icon on the top right window of the Zotero Desktop app. Hover the mouse over this icon to see the progress of the sync. If there are any errors, there will be a red exclamation mark next to the sync icon.

Notes:

  • Tested with:
    • Zotero Desktop 7.0.21.
    • Zotero iOS 2.0.0 Build 369.
    • Tailscale v1.86.5 (docker).
  • The maximum lifetime of a Tailscale auth key is 90 days. So we need to regenerate, edit .env and restart the docker containers.
    • Update: One can disable key expiration via the Tailscale Admin Console. On the Machines tab, find the zstore device, click on triple dot, then select Disable Key Expiry. To undo this, select Enable Key Expiry.
  • What doesn’t work with this setup: Syncing Group libraries. Zotero does not support syncing to a WebDAV server for group libraries.
  • I have not tested this. It is possible to run the docker containers on your laptop, without the need of a server. Like the server setup, the laptop needs access to the Internet when the iOS app tries to fetch a document.