Calico Pie's GEDCOM Coverage Test
How the Test Works
Calico Pie's GEDCOM Coverage Test is designed to discover the extent to which different applications support the GEDCOM standard - that is, GEDCOM 5.5 which has been the current version of GEDCOM since 1996.
The original idea for the GEDCOM Coverage Test was simple: create a GEDCOM test file that contains one record for each possible GEDCOM record type, and one field of every possible field type within each record. Each field would contain some simple test data. Then to test an application's coverage of GEDCOM, you only need do the following:
(1) load the test file into the program (into an empty database if 'importing' it).
(2) Use the program to save all of the data ('export' the data) in the form of a GEDCOM file.
(3) Count the number of fields in the original test file and the number of fields in the output file, and express the latter as a percentage of the former, thereby giving you a measure of the program's coverage of GEDCOM.
In fact, it wasn't possible to stick too precisely to the original idea. It was necessary to create a few extra records, for example, because you couldn't create fields of each possible type if you only had one record of each possible type (for example, each family record has a child field, a husband field and a wife field - so you need at least 3 individual records for these to link to). A small number of compromises were needed, but in each case, it was easy to allow for the compromises without affecting the validity of the resulting score. The compromises and all other details are fully described in the test documentation - see How Can the Test Be Checked for Fairness and Accuracy? below.
What the Results Mean and What They Don't Mean
Family Tree Maker 2006 got a GEDCOM Coverage of 4.5%. This does not mean that if you import a GEDCOM file into a Family Tree Maker database, only 4.5% of the data will be imported. Nor does it mean that that if you export a GEDCOM file from a Family Tree Maker database, only 4.5% of the data will be exported. The test is only measuring what percentage of all possible different types of GEDCOM data, a genealogy application can load and then save. Some types of data are likely to be much more commonly used than others. If it just so happened that a given GEDCOM file contained nothing but data of the types that Family Tree Maker supports, it might be able to load all of it. Equally, it is quite possible that all of the data in a given Family Tree Maker database could be successfully exported to a GEDCOM file.
It is also worth bearing in mind that the test is a combined test of loading and saving. It would be nice to test separately how much data any given program can load from a GEDCOM file, and how much it can save to a GEDCOM file. But there is no standard way of measuring how much data has been loaded into a program, so it is not practical to do this. It is theoretically possible (although unlikely) that some applications might be able to load all possible data in a GEDCOM file, while only being able to export a small part of it.
Why Does Family Historian Do So Much Better Than Other Programs?
The reason for this is simple: Family Historian was designed from the ground up to be 100% GEDCOM compatible and complete. It uses GEDCOM as its native file format. Other applications were not designed to do this.
Applications have always been able to support as much or as little of the GEDCOM standard as they wished to do or were capable of doing.
Is the Test Unfairly Biased in Favour of Family Historian?
The test is not biased in favour of Family Historian. If anything, it is unduly kind to other applications for these reasons:
The GEDCOM test file only contains one instance of each possible field type. But many, perhaps even most, GEDCOM fields can have multiple instances. In a great many instances there is no limit to the number of instances that they can have. Family Historian supports all possible instances. It is unlikely that other applications could, unless like Family Historian, they were designed from the start to fully support GEDCOM. But the GEDCOM Coverage Test makes no attempt to test whether an application can support more than a single instance of any given field.
Many GEDCOM fields have a complex range of possible values. Family Historian was designed to support all possible values of every field. Again, it is unlikely that an application which was not designed from the start to fully support GEDCOM could do this. But the test data for each field is only a simple sample valid value. The GEDCOM Coverage Test makes no attempt to test whether an application can support all possible values for any given field.
Family Historian has a deserved reputation for being extremely accurate in its support for GEDCOM. Notoriously some other applications frequently make mistakes. The GEDCOM Coverage Test makes no attempt to test whether an application has made any errors when exporting to GEDCOM.
Finally, the GEDCOM Coverage Test also does not test whether applications can support all possible amounts of data for any given field, however small or large.
It would be difficult indeed to devise a test that took into account all these additional considerations. But the fact that the GEDCOM Coverage Test does not, means that it is if anything unduly kind to applications other than Family Historian, as it is unlikely that they could have done as well as Family Historian had these other factors been included.
How Can the Test Be Checked for Fairness and Accuracy?
The answer to that is simple - you, or anyone, can repeat the test yourself, and confirm for yourself that the results are fair. To do this you need the following:
The GEDCOM Standard is specified in a document called "The GEDCOM Standard", prepared by the Church of Jesus Christ of Latter Day Saints, who created and designed GEDCOM. To find a copy of "The GEDCOM Standard", go to the LDS Church website at http://www.familysearch.org and search for 'GEDCOM'. Or just search for "The GEDCOM Standard" on Google, or other search engines. Detailed understanding of the GEDCOM spec is not really necessary, but at least some understanding of how GEDCOM works is needed.
Click here for a copy of the document"GEDCOM Coverage Test". This documentation describes how the test works in general, and how in particular it was applied to Family Historian 3, Roots Magic 3.0.3, and Family Tree Maker 2006.
Click here for a copy of the GEDCOM test file, used in the test.
Home | Links | About Us | Contact
Copyright © 2008 Calico Pie Limited