How to Update From Simple SDK Methods to Using a Workbook and Listener

Flatfile offers some simple SDK methods with each of its importers to simplify the embedding process and ensure you can get up and running as quickly as possible. These methods are easy to use, but they are also not very customizable. Using sheet to provide your sheet config, for example, will always result in a Workbook with the name "Embedded Workbook". Using onSubmit will always create a submit action with the name "Submit", and so on.

If you do want to customize your Portals, you'll need to change the properties you're using to launch your Portal to use workbook and a listener, instead. Here's how you can do that!

Sheet to Workbook Conversion

If you have a sheet that looks like this:

const blueprint = {
  "name": "Contacts",
  "slug": "contacts",
  "fields": [
    {
      "key": "firstName",
      "type": "string",
      "label": "First Name"
    },
    {
      "key": "lastName",
      "type": "string",
      "label": "Last Name"
    },
    {
      "key": "email",
      "type": "string",
      "label": "Email"
    }
  ]
}

This will need to be updated to look like the following example. Note that we're adding in a Submit action as well; we'll cover this later!

export const workbook = {
  name: "All Data",
  labels: ["pinned"],
  sheets: [
    {
      name: "Contacts",
      slug: "contacts",
      fields: [
        {
          key: "firstName",
          type: "string",
          label: "First Name",
        },
        {
          key: "lastName",
          type: "string",
          label: "Last Name",
        },
        {
          key: "email",
          type: "string",
          label: "Email",
        },
      ],
    },
  ],
  actions: [
    {
      operation: "submitAction",
      mode: "foreground",
      label: "Submit foreground",
      description: "Submit data to webhook.site",
      primary: true,
    },
  ],
};

All sheets in a Flatfile space/Portal live inside workbooks. When you use the sheet parameter to provide your sheet config to the importer, Flatfile assigns the workbook name and actions to the workbook for you and create the workbook under the hood. Configuring a workbook yourself gives you the control of these items so they can be customized correctly.

Don't forget to import your new workbook instead of your sheet, and provide it to your Flatfile options!

const flatfileOptions = {
    name: 'Embedded Space',
    publishableKey,
    workbook,
    // Additional props...
  };

onRecordHook to recordHook Conversion

We provide onRecordHook for handling data transformations with our simple SDK methods. The validations you add here don't need to be modified at all, just moved from inside onRecordHook to a listener. Once you move other features into a listener, you'll want to move your validations there as well to ensure that the validations work correctly with the workbook and other listener code.

Original onRecordHook validation:

    onRecordHook: (record) => {
      const firstName = record.get("firstName");
      console.log({ firstName });
      record.set("lastName", "Rock");
      return record;
    },

In a new file, you'll want to create a file named listener.ts or something similar. This will contain your Flatfile event listener. This will listen for any Flatfile events and then execute the code you provide. onRecordHook would listen for the commit:created event, and then make changes or set errors based on the validations you provided. For the listener, we've released a plugin that will listen for the same event and perform those same validations.

In your new listener.ts file, you'll want to install the plugin with npm install @flatfile/plugin-record-hook and then set up your listener like so:

import { FlatfileListener } from "@flatfile/listener";
import { recordHook } from "@flatfile/plugin-record-hook";

export const listener = FlatfileListener.create((listener) => {
  listener.use(
    recordHook("contacts", (record) => {
      const firstName = record.get("firstName");
      console.log({ firstName });
      record.set("lastName", "Rock");
      return record;
    })
  );
});

Notice that the validation syntax is exactly the same, it's just wrapped in the recordHook plugin instead of onRecordHook. Another thing to note is that in the recordHook, you'll want to define the slug of the sheet on which these validations should run. As the sheet we've been working with in these examples is "contacts", that is set as the slug in the `recordHook`.

Update your Flatfile Options to import your listener:

const flatfileOptions = {
    name: 'Embedded Space',
    publishableKey,
    workbook,
    listener,
    // Additional props...
  };

onSubmit to Submit Action

The last piece of the puzzle is moving your onSubmit code to your listener.

The first step here is to define your action on your workbook, which we did in the first step! When you use onSubmit, the action is automatically added to the workbook for you with our pre-defined name and settings. When you define the action on your workbook, you get to set the name/label and other custom information.

export const workbook = {
  name: "All Data",
  labels: ["pinned"],
  sheets: [
    //sheets here
  ],
  actions: [
    {
      operation: "submitAction",
      mode: "foreground",
      label: "Submit foreground",
      description: "Submit data to webhook.site",
      primary: true,
    },
  ],
};

Next, you'll want to handle the data from your sheet in your listener code. In the listener, you'll be listening for the "job:ready" event of the operation you're defining in your action above. So we set the operation to "submitAction", and you'll listen for "workbook:submitAction".

import api from "@flatfile/api";
import { FlatfileListener } from "@flatfile/listener";
import { recordHook } from "@flatfile/plugin-record-hook";

export const listener = FlatfileListener.create((listener) => {
  listener.use(
    recordHook("contacts", (record) => {
      const firstName = record.get("firstName");
      console.log({ firstName });
      record.set("lastName", "Rock");
      return record;
    })
  );

  listener.on(
    "job:ready",
    { job: "workbook:submitAction" },
    async ({ context: { jobId, workbookId } }) => {
      const { data: workbook } = await api.workbooks.get(workbookId);
      const { data: workbookSheets } = await api.sheets.list({ workbookId });

      const sheets = [];
      for (const [_, element] of workbookSheets.entries()) {
        const { data: records } = await api.records.get(element.id);
        sheets.push({
          ...element,
          ...records,
        });
      }

      try {
        await api.jobs.ack(jobId, {
          info: "Getting started.",
          // "progress" value must be a whole integer
          progress: 10,
        });

        //example using webhook.site as your backend
        const webhookReceiver = "https://webhook.site/...";

        const response = await fetch(webhookReceiver, {
          method: "POST",
          headers: {
            "Content-Type": "application/json",
          },
          body: JSON.stringify({
            workbook: {
              ...workbook,
              sheets,
            },
          }),
        });

        await api.jobs.complete(jobId, {
          outcome: {
            acknowledge: true,
            message: "This is now complete.",
            next: {
              type: "wait",
            },
          },
        });
      } catch (error) {
        console.error("Error:", error.stack);

        await api.jobs.fail(jobId, {
          outcome: {
            message: "This job encountered an error.",
          },
        });
      }
    }
  );
});

When listening to a custom job like the submit action, you need to acknowledge the job when it's in a ready state to start executing the job, as demonstrated above. You can easily do this with the @flatfile/api package used in the example! Once you're done handling your data, you'll also want to make sure to complete the job so your user knows all the data has been submitted successfully.

You'll then fetch the data from the sheet and can submit that to your backend however you'd like! The example above shows how to grab data if there are multiple sheets in a workbook, but this could easily be simplified if you are only planning to use one sheet.

Please note: One benefit of using sheet.allData in onSubmit is it provided a very simplified set of records rather than providing all of your sheet information like this process will. If the simplified dataset is all you're looking for, you'll want to pare down the information you're sending to your back end rather than providing the whole workbook + sheets of data. We have an example of a way you can simplified a record returned from our API to better mimic the data you'd get from sheet.allData here, if you're interested!

Done!

That's it! It seems like a lot, but the changes are mostly just copying/pasting your current configuration into another location. Moving to a workbook and listener opens up a variety of options for expansion and customization down the road!