Automated Firestore backups in 5 easy steps

Setting up automated Firestore backups in 5 easy steps

The real-time schema-less database Firestore from Firebase does not come with built-in automated backups. In this blog post, we will guide you through setting up automated Firestore backups inspired by how we do it at Kvalifik.

☝️ We assume that you are an owner of the Firebase project and that you have enabled billing. Read more here.

The Firestore backups are initiated by a cloud function and are stored in a dedicated storage bucket on Google Cloud Platform.

‍

1) Apply appropriate service account role

Open Google Cloud Platform and select the Google Cloud project associated with your Firebase project.

Go to IAM & Admin → IAM.

Edit the Member "[project-id][@appspot.gserviceaccount.com]".

Add another role: "Cloud Datastore Import Export Admin".

Click save.

‍

2) Create storage bucket for Firestore backups

Go to the Storage section in Google Cloud Platform.

Create bucket:

Name your bucket, ex:
Name: [project-id]-backup
Choose where to store your data, we suggest:
Location type: Multi-region
Location: eu (multiple regions in European Union)
Choose a default storage class for your data, we suggest:
Default storage class: Nearline
Choose how to control access to objects, we suggest:
Access control: Uniform
Advanced settings, we suggest:
Retention policy: Retain objects for 7 days

✅ This ensures that backups from the last seven days can not accidentally be deleted.

Click Create.

‍

3) Configure bucket lifecycle settings

Feel free to adjust these lifecycle settings to fit your needs.

Go to Lifecycle settings and add the following rules:

Select an action: Set storage class to Coldline
Select object conditions: Age: 7 days
Select an action: Set storage class to Archive
Select object conditions: Age: 30 days
Select an action: Delete object
Select object conditions: Age: 365 days

4) Setup cloud function to initiate backups

We recommend using environment variables to be able to share the same cloud function across multiple Firebase projects. To do this, run the following command on your local machine (with the Firebase CLI installed):

⚠️ If you have several environments, then be sure to switch to the correct project using the command "firebase use [environment-name]".

‍

$ firebase functions:config:set backup.bucket_name="*[project-id]*-backup"

‍

Add the following Typescript cloud function firestoreBackupDaily. Note that this cloud function is adapted from the Javascript version found here.

‍

import * as functions from 'firebase-functions'
import * as admin from 'firebase-admin'

const client = new admin.firestore.v1.FirestoreAdminClient()
const env = functions.config()

export const firestoreBackupDaily = functions
  .region('europe-west1') // Change to appropriate region.
  .pubsub.schedule('0 0 * * *') // Every 24 hours (at midnight).
  .onRun(async () => {
    const projectId = process.env.GCP_PROJECT || process.env.GCLOUD_PROJECT
    const databaseName = client.databasePath(projectId, '(default)')
    const timestamp = new Date().toISOString()
    const dest = `gs://${env.backup.bucket_name}/firestore/${timestamp}`

    try {
      const responses = await client.exportDocuments({
        name: databaseName,
        outputUriPrefix: dest,
        collectionIds: [] // Leave empty to export all collections.
      })
      const response = responses[0]
      console.log(`Operation Name: ${response['name']}`)
    } catch (error) {
      console.error(error)
      throw new Error('Export operation failed.')
    }
  })

‍

Finally, deploy your cloud function using the following command:

‍

$ firebase deploy --only functions:firestoreBackupDaily

‍

5) Test your cloud function

Go to the Cloud Scheduler page and click "Run now" in the row corresponding to your newly created cloud function. Shortly after, you should see a Firestore backup within the folder "firestore" in the storage bucket created earlier.

‍

Next steps

To finish off this setup we recommend setting up monitoring and a plan for disaster recovery. In addition, please note that the solution presented here only performs backups of Firestore. You might also be interested in backing up Firebase Storage and Firebase Auth. The former can be achieved by configuring a daily transfer job on Google Cloud Platform, whereas the latter is more tricky (remember to also store a copy of the password hash parameters!). You can use the listUsers function presented here in a scheduled cloud function similar to the one presented earlier. We might cover some of these topics in a future blog post.

‍