TEST Data Importer

Data loading process in the TEST module

Written by Eduardo Abeliuk
Updated over a week ago

Each assay that is recorded in the TEST application can be loaded with experimental data tables associated with a particular experiment. In general these data tables share a similar structure since they come from the same experiment assay. Their structure is understood as the number and type of columns in the tables. If you're starting a never before recorded data structure, it will require a loading process consisting of two fundamental steps.
Through these steps the user is able to restructure and give meaning to the loaded data by assigning each column a particular type. The available types are defined by the application (metadata structure) and must be chosen and named by the user.
This two-step process can be saved as a parser (you can get familiar with the technical vocabulary in this glossary). Parsers save everything done by the user during a data loading process, so that the user does not need to do this process again, as long as the data table maintains the structure used to generate the parser.
To illustrate all of this let's look at an example. We will load the table below from a CSV file to the app following the steps appointed: 

Example of experimental data generated

First, the user has to select a previously registered experiment or creating a new one and then naming the assay.  This is where the user must upload the CSV file by clicking on the upload area.

As you can see in the image below, there is an additional field that allows you to select a parser. The parsers allow you to avoid performing the following steps several times, provided they have been done previously.

Step 1: this step of the loading process is useful to restructure the loaded data. The TEST module supports a series of actions that modify the format of this data. The image below shows a list with the reformat actions supported by the module. Among them are: change the name of a column, ignore a column, add a column with default values (specially useful to add unit columns). In case you do not need to restructure the loaded data this step can be omitted.

Step 2: this step consists in defining the metadata of the loaded data table. The types for each column should be registered. As you can see in the image below and according to the example we have uploaded, an "assay subject" must be defined ("Strain ID" in the example). It is usual that there are at least a couple of columns of measurement type ("Measurement") and unit type ("Unit"). Commonly the assays and their subjects will be characterized by descriptors. Two of these are shown here, corresponding to Enzyme A and Enzyme B. More details on this step on another article.

Then, once every column has been defined the user clicks on the "Validate" button on the bottom right corner, this is is a mere revision of the column definitions.  

If the column definitions is set correctly, the user can go ahead and click on "Save".

Finally, once the data has been saved, a record of it can be seen by selecting the "Data" option in the top menu, from that view you can open the individual record and visualize data loaded in the platform.

Did this answer your question?