Data Mappings

Data Mappings are a part of our importing and data integration technologies. Instead of just importing a csv file as is, data mappings allow you to specify the nature of the file and how to process importing it. You can explicitly state that certain imports will be of a particular text encoding, whether or not the file has headings, the delimiter to use and many more options including formulas to run on the data itself.

This provides an extremely powerful mechanism to tailor and automate you data integration processes (along with our other technologies such as Import Definitions). Most importantly you don't need help from a technical development team in order to get data into your system in the manner you require.

Data Mapping Features

It's too often the case that legacy systems aren't easily capable of integrating with other products. They often have no API, don't easily provide the data in formats you need, or are rigid about the data structure coming out in a certain format.

LemonEdge has standard template formats (in excel, csv and for certain hierarchical data even xml), but not all systems are capable of providing data in the format you require to import. Rather than introduce manual process of data transformation, which is resource and time intensive (and is a major factor in the length of project implementations), we have Data Mappings to take the output file from one system and map that into a format we can use for importing into our system.

Data Mappings can be specific to importing a certain data type, or general to all such as describing a general file format of a Pipe delimited file with Unicode Encoding.

File Format

Data Mappings allow can describe the following file formats:

  • Text Encoding: ASCII, Big Endian Unicode, Unicode, UTF32, UTF8 UTF7
  • Whether or not the file has named column headers
  • The delimiter to use for the data separators (i.e. ',' or '|')
  • How to treat each column (either named column or from order if has no headers)
    • What property (if any, as could be used as source for calculation on other properties) it should map to in the system
    • Whether to use it as part of a data lookup key
    • Methods for transforming the data including using formulas (see our formula engine)

Context

Whenever you are importing a file, the system will present a list of appropriate data mappings. The system goes through the following process when importing a file:

  1. Scans the file to determine if it knows what specific data type this file is importing for.
    It does this by looking for a column named IMPORT_DATA_TYPE which holds the type of data being imported.
    1. If it can't determine the data type
      1. Displays a prompt asking for the data type being imported
      2. Checks to see if there are any general data mappers for all data import types, and if there are any specifically for the selected data type to be imported
      3. Allows a selection of any appropriate data mapping to use for that selected data type
    2. If it could determine the data type
      1. Checks to see if there are any general data mappers for all data import types, and if there are any specifically for the data type being imported (determined it in step 1)
      2. If there are appropriate data mappings then it will display a prompt asking if you wish to use any
  2. Starts Importing the file - see our import process for more information.

Data Mappings are also used in our other data integration technologies such as Import Definitions [more info]. Here for each Import Definition Step you can specify a data mapping to use.

Views

Data Mappings can be edited using the following views: