This feature is an automated version of "Load Mode". It allows the bulk importing of data from one or more text files or Excel® spreadsheets. The data loading is controlled using a script file which contains a sequence of commands. These commands can create or update instances in any table and can also load measurement property data.
The LoadInstancesFromFile command loads one or more instances from the specified file. The data is extracted from columns which are identified using the column specification syntax in exactly the same fashion as in "Load Mode".
The LoadMeasurementPropertiesFromFile command adds a new property (i.e. a column of data) to an existing Measurement.
The general form of this command is:
<LoadInstancesFromFile table="TableName"
file="FileName"
preset="PresetName"
matchmode="MatchMode"
encoding="EncodingName">
<ParserSetting name="SettingName"
value="SettingValue" />
<ColSpec name="FieldOrAttributeName"
value="ColumnSpecification" />
</LoadInstancesFromFile>
Values must be provided for table and file, but preset, matchmode and encoding are optional.
The ParserSetting element can occur zero or more times, and the ColSpec element can occur one or more times.
Certain chacters are 'special' in XML documents and must be encoded using particular representations to avoid them being interpeted as part of the XML formatting, see below for some examples.
The matchmode parameter (see below) determines what should be done when an instance being loaded from a file has the same name as an instance already in the database. Possible actions include updating the values of the existing instance, or creating a new instance with a duplicate name.
If you have already defined a Preset setting for the table you are loading into, then it is possible access it within the LoadInstancesFromFile command. This means that the ColSpec and ParserSetting elements will not be required as the values will be retrieved from the Preset. It is still possible to provide one or more of them, and their values will override the values defined by the Preset.
The LoadInstancesFromFile command can have an optional matchmode parameter which determines what will happen when the name of instance being loaded matches the name of an instance that is already in the database.
UpdateFirst updates the values in first instance (in the order in which they were created) with a matching name.
UpdateLast updates the values in last instance (in the order in which they were created) with a matching name.
CreateNew ignores the exiting instance(s) and creates a new instance.
Note that this paramter only detemines what happens when matching names are found for the instance begin loaded, not for any instances that it is being linked to. In the case of linking, when more than one candidate instance is found (because they have the same name) then the most recently created one is always chosen.
The general form of this command is:
<LoadMeasurementPropertiesFromFile file="FileName"
measurement="MeasurementName"
preset="PresetName">
<ParserSetting name="SettingName"
value="SettingValue" />
<ColSpec name="FieldOrAttributeName"
value="ColumnSpecification" />
<Property name="PropertyName"
colspec="ColumnSpecification"
labelledextract="LabelledExtractName"
quantitationtype="QuantitationTypeValue"
unit="UnitValue"
scale="ScaleValue"
origin="OriginValue"
datatype="DataTypeValue" />
</LoadMeasurementPropertiesFromFile>
Values must be provided for file and measurement, but preset is optional.
The ParserSetting element can occur zero or more times, the ColSpec element should occur exactly once (for the 'Feature' field) and there can be any number of Property elements.
As in the LoadInstancesFromFile command, it is possible to access any Preset settings that you have defined. This means that the ColSpec, ParserSetting and Property elements will not be required as the values will be retrieved from the Preset. It is still possible to provide one or more of them, and their values will override the values defined by the Preset.
The Property element is exactly analogous to the collection of fields which must be completed when defining a measurement property manually.
| Name | Presence | Meaning | Possible Value(s) | |
| name | required | the name of the property | any | |
| colspec | required | which column(s) contain the data | a valid column specification | |
| labelledextract | optional | which LabelledExtract to link to | a valid LabelledExtract name | |
| quantitationtype | optional | MeasuredSignal,Ratio,PresentAbsent,Failed,etc. | ||
| target | optional | which other Property this one is related to (only for 'quantitationtype' Error or PValue) | ||
| unit | optional | any | ||
| scale | required | LINEAR,LN,LOG2,LOG10,FOLD_CHANGE,OTHER | ||
| origin | required | Feature,Background |
Any number of ParserSetting elements can be provided. These elements are used to control the file parsing in exactly the same way as the type-in boxes in "Load Mode".
The following table shows the possible entries in the "name" field of ParserSetting elements and the corresponding data type that should be provided in the "value" field.
| Name | Value | |
| Ignore first | integer | |
| Ignore last | integer | |
| Ignore until seen | regular expression | |
| Ignore once seen | regular expression | |
| Ignore beginning | character(s) | |
| Column Delimiters | character(s) | |
| Ignore blank | true or false |
Examples:
<ParserSetting name="Ignore first" value="1" /> <ParserSetting name="Ignore blanks" value="true" /> <ParserSetting name="Column Delimiters" value="\t" />
The ColSpec element correspnds to the fields used to specify how to extract data from columns in a "Load Mode" form. One ColSpec must be provided for the 'Name' of the instance and one for each of the 'Link' and 'Attribute' fields that are to be used.
The "name" part of the ColSpec element should identify which field the ColSpec applies to , and the "value" part should be written using the column specification syntax.
To determine the correct name for an attribute (and to have easy access to a textual representation which you can cut & paste into your script) click on the help button next to the attribute in any form in which it is displayed.
Examples:
<ColSpec name="Name" value="$1:$2">
<ColSpec name="TreatedSample.Material Type" value="total_RNA">
<ColSpec name="ImageAnalysisAction.Performed By" value="${UserName}">
Because the batch script is an XML document, certain special characters are not allowed to appear, most notably ", < and >. These characters must be represented using special strings (called XML entities) as follows:
| Character | Entity | |
| " | " | |
| < | < | |
| > | > | |
| & | & |
(note the semicolon at the end of the entity name, it is required)
So, the column specification
$5{"missing"->""} would be written as
To avoid having to convert complicated column specifications into thier XML representation, it can be easier to create a Preset setting in "Load Mode" and use this preset in the LoadInstancesFromFile command.
<maxdLoadBatchScript version="1.0">
<LoadInstancesFromFile table="Extract" file="example-extracts.dat" preset="My Settings" />
<LoadInstancesFromFile table="Feature" file="example-features.xls" >
<ParserSetting name="Ignore beginning" value="#" />
<ColSpec name="Name" value="$1" />
<ColSpec name="Reporter" value="$3" />
<ColSpec name="ArrayType" value="Custom Array 17" />
<ColSpec name="Feature.Location.Row" value="$5" />
<ColSpec name="Feature.Location.Column" value="$6" />
<ColSpec name="Feature.Shape" value="$8" />
</LoadInstancesFromFile>
<LoadInstancesFromFile table="ArrayType" file="example-array-types.dat">
<ParserSetting name="Ignore first" value="1" />
<ParserSetting name="Ignore blanks" value="true" />
<ColSpec name="Name" value="$1" />
<ColSpec name="ArrayType.Surface Type" value="polylysine" />
<ColSpec name="ArrayType.Substrate Type" value="glass" />
<ColSpec name="ArrayType.Technology Type" value="spotted_colony_features" />
<ColSpec name="ArrayType.Number Of Features" value="$2" />
<ColSpec name="ArrayType.Number Of Plates" value="$3" />
<ColSpec name="ArrayType.Number Of Rows Per Plate" value="$4" />
<ColSpec name="ArrayType.Number Of Columns Per Plate" value="$5" />
</LoadInstancesFromFile>
<LoadMeasurementPropertiesFromFile
file="example-property-data.dat"
measurement="MyExample-1" >
<ParserSetting name="Ignore first" value="1" />
<ColSpec name="Feature" value="$1" />
<Property name="IntRed"
labelledextract="MyLabelledExtract"
colspec="$2{""-amp;gt;"NaN"}"
unit="no unit"
datatype="INTEGER"
scale="LINEAR"
origin="Feature" />
</LoadMeasurementPropertiesFromFile>
</maxdLoadBatchScript>