timelink

Timelink, formerly known as MHK (Micro History with Kleio) is an information system designed for processing person related information collected from historical sources.

View the Project on GitHub joaquimrcarvalho/timelink

Exporting and Importing real entities (identifications)

Kleio notation for real entities

From version 2019 A3 Timelink/MHK has the ability to export current linking information in kleio format. This allows backup and exchange of record linking information between users and databases.

Although the export files are in Kleio format they are not “sources” and keep the clear distinction in Timelink between the information gathered from historical sources and inferences and “interpretations” produced by the historians.

This is done by introducing the concept of authority-register at the same level as historical source

Gacto definitions:

part 	name=kleio;
    also=structure,translator,autorels,obs,prefix,translations;
    position=structure,translator,obs;
    part=historical-source,fonte,authority-register, identifications

note ===========================================================
note doc group authority-register a register of authority-records
note    authority records are used to identify people, objects
note    and to normalize vocabularies and other authority information.
note    authority-registers must have an unique id, but a given community
note    can have as many authority-registers as needed
note

part name=authority-register;
    position=id,name;
    guaranteed=id,name,date,user,dbase;
    also=date,user,dbase;
    part=authority-record

note ===========================================================
note identifications: a authority-register for identifications
note doc group identifications are authority-registers for record linking.
note     They contain real-entity records (people or objects) that aggregate
note     occurrences in the source pertaining to the same person or entity.
note     To keep imported records separate from local ones use a different user.
part name=identifications;
    source=authority-register;
    position=id,name;
    guaranteed=id,name,date,user,dbase;
    also=date,user,dbase,replace_mode,ignore_date,obs;
    part=rentity,rperson,robject

Example of an identification file generated internally.


kleio$gacto2.str/translator=authorities/obs=Generated from database testes by user testes, file date: +2020-07-02 Timestamp:2020-07-02T11:23:01.581Z

   identifications$real-entities/Real Entities, including real persons and real objects
      /dbase=testes
      /user=testes
      /date=20200702
      /mode=backup

      rperson$rp-59/afonso goncalvez/m/
         occ$c1568.03-per3
         occ$c1569.07-per2

Identification files

Identification files store information about real entities, normally persons, but can also be other types of entities, like objects.

These files pose a problem, because MHK provides interactive ways to identify entities directly on the database. So, unlike source files, it is possible to add information to real entities, namely new occurrences directly on the database. So a file representing real entities may easily get out of date regarding the information in the database.

On the other hand it is good to have a proper record of real entities outside the database, for three types of situations:

  1. backup purposes, so that in case of a crash of the database the real entity information can be restored. Note that source files are normally stored on Git so this would also provide version control on real entity decisions.
  2. sharing of identifications between users and databases. For instance data collectors working on new sources on a copy of the main database can get updated information on real persons from the project coordinator.
  3. Synchronising a public server with a local database: git can be used to keep a server synchronised with a local database, if not only source information but also identifications are shared in git commits.

Identification files and permissions

Use cases and permissions

Generating identification files

Identification files are generated by MHK in the “Identification” section of the interface, in export.

Rules on importing Identification files

The mode element of the identifications group determines how the file is processed.

   identifications$real-entities/Real Entities, including real persons and real objects
      /dbase=testes
      /user=testes
      /date=20200702
      /mode=backup

Rule 1: files with mode=backup are processed but the content is ignored except if mhk.import.authority.backup=yes

The content of a file with mode=backup on the identifications group is skipped on import, since it is considered to be a backup of existing real entities. Note that the file is “processed” and a report is produced To import the content of the file either change the mode to “restore” or add the following property to an existing user with level 3 permissions:

mhk.import.authority.backup=yes

Related properties

mhk.import.auto=yes (auto import)

Having both mhk.import.authority.backup=yes and mhk.import.auto=yes makes MHK to automatically update the database and current identifications, and is a way to keep a server synchronised with a reference database.

Rule 2: Non admin Users can only import files where the user element correspond to the current (“importing”) user.

   identifications$real-entities/Real Entities, including real persons and real objects
      /dbase=testes
      /user=testes
      /date=20200702
      /mode=backup

A file can only be imported by a user with the same id as the user stated in the user element of the identifications group. However, the user that corresponds to the admin of the database can import identification files from other users. The real entities imported by an admin on behalf of other users will be stored in the space associated with the file user.

Any user can import identifications from another user by changing the value of the “user” element in the file so that it matches its own id.

Rule 3: Except for the current admin user, users cannot overwrite on import real entities that are associated with another user

Real entities Ids are generated sequentially across the database, so it is very much possible that when exchanging identification files there is an overlap of incoming ids with existing ids.

To avoid replacing ids of a user with those of another upon import, the system generates an error when importing a real entity which already exists in the database associated with a different user.

Current if the importing user is admin and the imported real entity belongs to an user in the database different from the onde of the importing file, then the imported real entity replaces the existing one. Example:

In the future it would be better to implement a mhk.import.authority.rentities.id.conflict=mode with mode being:

Non covered use-case

Public replica data set with several users that do their own identifications. They do not export. Main admin regularly updates identifications of reference database and pushes to git. On update user identification are replaced by new ones coming from admin. Solutions: