Real-time Sync

Real-time Sync Guide

BitMarks supports real-time synchronization via WebSocket, allowing instant updates across all your devices.

Overview

Real-time sync uses Cloudflare Durable Objects to coordinate WebSocket connections. Each user has a dedicated Durable Object instance that:

1. Maintains WebSocket connections from all devices
2. Broadcasts changes to connected devices
3. Persists changes to the database

┌──────────────┐     ┌──────────────────────────────┐
│   Device A   │────▶│                              │
│  (Browser)   │◀────│                              │
└──────────────┘     │     SyncCoordinator          │
                     │     (Durable Object)          │
┌──────────────┐     │                              │
│   Device B   │────▶│  - WebSocket management     │
│  (Extension) │◀────│  - Change broadcasting      │
└──────────────┘     │  - D1 persistence           │
                     │                              │
┌──────────────┐     │                              │
│   Device C   │────▶│                              │
│   (Mobile)   │◀────│                              │
└──────────────┘     └──────────────────────────────┘

---

Connecting

WebSocket Endpoint

GET /api/v1/sync/realtime

Required Headers

Header	Description
Upgrade	Must be websocket
X-User-ID	Your user ID
X-Device-ID	Unique device identifier
Cookie	Session cookie

Connection Example

JavaScript

const connect = async () => {
  const url = 'wss://app.bitmarks.sh/api/v1/sync/realtime';

  const ws = new WebSocket(url, [], {
    headers: {
      'X-User-ID': userId,
      'X-Device-ID': deviceId
    }
  });

  ws.onopen = () => {
    console.log('Connected to sync server');
  };

  ws.onmessage = (event) => {
    const message = JSON.parse(event.data);
    handleMessage(message);
  };

  ws.onerror = (error) => {
    console.error('WebSocket error:', error);
  };

  ws.onclose = (event) => {
    console.log('Disconnected:', event.code, event.reason);
    // Implement reconnection logic
  };

  return ws;
};

React Hook

import { useEffect, useRef, useCallback } from 'react';

function useSyncWebSocket(userId: string, deviceId: string) {
  const ws = useRef<WebSocket | null>(null);
  const reconnectTimeout = useRef<number>();

  const connect = useCallback(() => {
    const url = 'wss://app.bitmarks.sh/api/v1/sync/realtime';

    ws.current = new WebSocket(url);

    ws.current.onopen = () => {
      console.log('Connected');
    };

    ws.current.onclose = () => {
      // Reconnect after 5 seconds
      reconnectTimeout.current = setTimeout(connect, 5000);
    };

    ws.current.onmessage = (event) => {
      const message = JSON.parse(event.data);
      // Handle message...
    };
  }, [userId, deviceId]);

  useEffect(() => {
    connect();
    return () => {
      ws.current?.close();
      clearTimeout(reconnectTimeout.current);
    };
  }, [connect]);

  return ws;
}

---

Message Protocol

Inbound Messages (Client → Server)

Sync Event

Push a change to the server:

{
  type: 'sync',
  event: {
    id: 'sync_abc123',
    user_id: 'user_01ABC',
    entity_type: 'bookmark',
    entity_id: 'bm_xyz',
    operation: 'create',
    timestamp: 1735430400000,
    device_id: 'device_456',
    encrypted_data: 'base64...',
    data_hash: 'sha256...',
    conflict_resolved: false
  }
}

Ping

Keep-alive ping:

{
  type: 'ping'
}

Outbound Messages (Server → Client)

Connected

Sent immediately after connection:

{
  type: 'connected',
  deviceId: 'device_456',
  timestamp: 1735430400000
}

Pong

Response to ping:

{
  type: 'pong'
}

Sync Broadcast

Change from another device:

{
  type: 'sync',
  event: {
    id: 'sync_def789',
    user_id: 'user_01ABC',
    entity_type: 'bookmark',
    entity_id: 'bm_new',
    operation: 'create',
    timestamp: 1735430500000,
    device_id: 'device_789',  // Different device
    encrypted_data: 'base64...',
    data_hash: 'sha256...',
    conflict_resolved: false
  }
}

Error

Server error:

{
  type: 'error',
  error: 'Invalid message format'
}

---

Implementation Guide

1. Generate Device ID

   Each device needs a unique, persistent identifier:

   import { generateDeviceId } from '@bitmarks.sh/core';
   
   const getDeviceId = () => {
     let deviceId = localStorage.getItem('bitmarks_device_id');
     if (!deviceId) {
       deviceId = generateDeviceId();
       localStorage.setItem('bitmarks_device_id', deviceId);
     }
     return deviceId;
   };

2. Connect with Reconnection

   class SyncConnection {
     private ws: WebSocket | null = null;
     private reconnectDelay = 1000;
     private maxDelay = 30000;
   
     connect() {
       this.ws = new WebSocket('wss://app.bitmarks.sh/api/v1/sync/realtime');
   
       this.ws.onopen = () => {
         this.reconnectDelay = 1000; // Reset delay
       };
   
       this.ws.onclose = () => {
         // Exponential backoff
         setTimeout(() => this.connect(), this.reconnectDelay);
         this.reconnectDelay = Math.min(
           this.reconnectDelay * 2,
           this.maxDelay
         );
       };
     }
   }

3. Handle Messages

   const handleMessage = async (message: WebSocketOutbound) => {
     switch (message.type) {
       case 'connected':
         console.log('Connected as device:', message.deviceId);
         // Maybe trigger initial sync
         break;
   
       case 'pong':
         // Keep-alive confirmed
         break;
   
       case 'sync':
         // New change from another device
         await applyRemoteChange(message.event);
         break;
   
       case 'error':
         console.error('Server error:', message.error);
         break;
     }
   };

4. Send Changes

   const sendChange = (event: SyncEvent) => {
     if (ws?.readyState === WebSocket.OPEN) {
       ws.send(JSON.stringify({
         type: 'sync',
         event
       }));
     } else {
       // Queue for later
       pendingChanges.push(event);
     }
   };

5. Keep-Alive

   // Send ping every 30 seconds
   setInterval(() => {
     if (ws?.readyState === WebSocket.OPEN) {
       ws.send(JSON.stringify({ type: 'ping' }));
     }
   }, 30000);

---

Complete Example

import { SyncEvent, WebSocketOutbound } from '@bitmarks.sh/core';

class RealtimeSyncManager {
  private ws: WebSocket | null = null;
  private userId: string;
  private deviceId: string;
  private encryptionKey: Uint8Array;
  private pendingChanges: SyncEvent[] = [];
  private reconnectDelay = 1000;
  private pingInterval: number | null = null;

  constructor(userId: string, deviceId: string, key: Uint8Array) {
    this.userId = userId;
    this.deviceId = deviceId;
    this.encryptionKey = key;
  }

  connect() {
    const url = 'wss://app.bitmarks.sh/api/v1/sync/realtime';
    this.ws = new WebSocket(url);

    this.ws.onopen = () => {
      console.log('WebSocket connected');
      this.reconnectDelay = 1000;
      this.startPing();
      this.flushPendingChanges();
    };

    this.ws.onmessage = (event) => {
      const message: WebSocketOutbound = JSON.parse(event.data);
      this.handleMessage(message);
    };

    this.ws.onclose = (event) => {
      console.log('WebSocket closed:', event.code);
      this.stopPing();
      this.scheduleReconnect();
    };

    this.ws.onerror = (error) => {
      console.error('WebSocket error:', error);
    };
  }

  disconnect() {
    this.stopPing();
    this.ws?.close();
    this.ws = null;
  }

  private handleMessage(message: WebSocketOutbound) {
    switch (message.type) {
      case 'connected':
        console.log('Confirmed connection:', message.deviceId);
        break;

      case 'sync':
        this.handleRemoteSync(message.event);
        break;

      case 'pong':
        // Connection healthy
        break;

      case 'error':
        console.error('Server error:', message.error);
        break;
    }
  }

  private async handleRemoteSync(event: SyncEvent) {
    // Skip our own changes
    if (event.device_id === this.deviceId) return;

    // Decrypt and apply
    const decrypted = await decryptObject(
      event.encrypted_data,
      this.encryptionKey
    );

    // Emit event for UI to handle
    this.emit('change', {
      type: event.operation,
      entity: decrypted
    });
  }

  async pushChange(
    entityType: 'bookmark' | 'history' | 'group',
    entityId: string,
    operation: 'create' | 'update' | 'delete',
    data?: any
  ) {
    const encrypted = data
      ? await encryptObject(data, this.encryptionKey)
      : undefined;

    const event: SyncEvent = {
      id: `sync_${crypto.randomUUID()}`,
      user_id: this.userId,
      entity_type: entityType,
      entity_id: entityId,
      operation,
      timestamp: Date.now(),
      device_id: this.deviceId,
      encrypted_data: encrypted,
      data_hash: await hashData(JSON.stringify(data)),
      conflict_resolved: false
    };

    if (this.ws?.readyState === WebSocket.OPEN) {
      this.ws.send(JSON.stringify({ type: 'sync', event }));
    } else {
      this.pendingChanges.push(event);
    }
  }

  private flushPendingChanges() {
    while (this.pendingChanges.length > 0) {
      const event = this.pendingChanges.shift()!;
      this.ws?.send(JSON.stringify({ type: 'sync', event }));
    }
  }

  private startPing() {
    this.pingInterval = setInterval(() => {
      this.ws?.send(JSON.stringify({ type: 'ping' }));
    }, 30000);
  }

  private stopPing() {
    if (this.pingInterval) {
      clearInterval(this.pingInterval);
      this.pingInterval = null;
    }
  }

  private scheduleReconnect() {
    setTimeout(() => {
      this.connect();
      this.reconnectDelay = Math.min(this.reconnectDelay * 2, 30000);
    }, this.reconnectDelay);
  }

  // Simple event emitter
  private listeners = new Map<string, Function[]>();

  on(event: string, callback: Function) {
    const list = this.listeners.get(event) || [];
    list.push(callback);
    this.listeners.set(event, list);
  }

  private emit(event: string, data: any) {
    this.listeners.get(event)?.forEach(cb => cb(data));
  }
}

---

Best Practices

Tip

Always implement offline support. Real-time sync enhances but shouldn’t replace traditional pull/push.

Offline Handling

// Queue changes when offline
window.addEventListener('online', () => {
  syncManager.connect();
});

window.addEventListener('offline', () => {
  // Changes are automatically queued
});

Conflict Resolution

Real-time sync uses the same Last-Write-Wins strategy as regular sync:

// Server compares timestamps
if (localEvent.timestamp > serverEvent.timestamp) {
  // Local wins
} else if (localEvent.timestamp < serverEvent.timestamp) {
  // Server wins
} else {
  // Same timestamp: compare device IDs lexicographically
}

Performance

• Batch rapid changes (debounce)
• Don’t send redundant updates
• Use message compression for large payloads

// Debounce rapid changes
const debouncedSync = debounce((event) => {
  syncManager.pushChange(event);
}, 500);

---

Troubleshooting

Connection Refused (426)

The server requires WebSocket upgrade:

// Ensure you're using WebSocket, not HTTP
const ws = new WebSocket('wss://...'); // NOT fetch()

Frequent Disconnections

1. Check network stability
2. Ensure ping/pong is working
3. Look for proxy/firewall issues

Messages Not Received

1. Verify device ID is consistent
2. Check user ID matches
3. Ensure session is valid