This article is for Flatfile's Portal 3.0 and Workspaces. While the basic principle behind the importer remains the same in our latest version, we've made some big improvements since publishing this! If you'd like to check out the latest version of the Flatfile Data Exchange Platform, click here.

Flatfile’s importer simplifies the process of transforming data between two systems. This is accomplished by walking the user (let's say, you) through steps to align the input data with the required output format (defined via Flatfile template). Some steps are automated when Flatfile is able to predict the correct choices at that step, while others are presented in a simplified way to minimize work for you. Our importer learns from each interaction, further simplifying the required actions with each additional import.

Flatfile’s ability to convert incoming data of varied formats and quality levels to the requested output removes any need for data preparation pre-import, saving time both on data management and related communications. Imports can happen asynchronously and with files as they exist.

The importer has 4 stages:

The first step in the Data Experience is adding the input data. Over the course of an import this data will be transformed into the requested output format, meaning that data added at this step does not need to be prepared or cleaned prior to import. Bring your data as it exists!

Data can be added via several methods:

If the added file has multiple sheets, you will be asked to select the relevant sheet to import. Currently, Portal is only able to process one sheet per import.

After a file has been uploaded, the next step is to identify the data to be imported and its headers. This prepares us to match the original data headers to template headers and ensures we do not include anything outside of the dataset (like report metadata) in the imported data.

If the imported file does not contain a header row, columns can be mapped using the column identifier (A, B, C etc).

The Mapping step is central to the Flatfile import flow, confirming the alignment of columns from the input data with columns in the output template. Any data in columns mapped at this step will be migrated and transformed into the requested output format.

Mapping is also where Flatfile brings the most magic. Using mapping choices recorded from 1.8 billion (and counting!) rows processed in Flatfile, we’ve trained a machine learning model that works alongside memory of your and your colleagues’ past selections to accurately predict over 90% of matching actions. When available, these mappings will be applied at the mapping step, reducing the number of mapping actions we ask you to make.

When we need input, the dropdown in the template column will remain blank. Use the dropdown to select the relevant field from the output template.

Scroll down the page to review any fields determined to be not relevant and excluded from matching. If an excluded field should be included, use the arrow next to the field name to include it. Then match it to the relevant template field.

In the final step of Flatfile’s import process, you see your data translated into the requested output format. This step allows you to review and validate the migration as well as correct or provide additional values that are required before the transformation is complete.

You may notice some things have changed:

These changes are all made according to rules set up on the output template and help reduce the amount of work required by you to transition your data into the requested output format.

There may still be outstanding errors that require your attention at this stage - empty values in a required column or email addresses found to be invalid for example. Use the filters at the top of the page to filter for only rows containing errors to understand what actions are required of you. Then make your edits directly in the data table. Any changes will be validated immediately against the template validations. Instant gratification!

You will be warned and guided to correct any outstanding errors before submitting. When submission begins, a loading screen will count down the records to be transferred until submission is complete. As records are transferred, they can be additionally validated against the Flatfile customer's system to ensure completeness. Data rejected at this stage can be returned to Flatfile for additional review.

If all goes well, we celebrate the significant victory of simplifying another data transfer (why not?) before sending you off to the happy task of using your data in its new home. Onwards!