MAKING THE CASE
Investigating Large Scale Human Rights Violations Using Information Systems and Data Analysis

Chapter 2

The Haitian National Commission for Truth and Justice:
Collecting Information, Data Processing, Database Representation, and Generating Analytical Reports

Patrick Ball and Herbert F. Spirer

Introduction

In early May 1995, Francoise Bouchard, President of the Haitian National Commission for Truth and Justice (La commission nationale pour la vérité et justice, CNVJ), invited the American Association for the Advancement of Science (AAAS) to advise the commission on how to develop a large-scale interviewing project to take the testimonies of a several thousand witnesses of human rights violations. The AAAS team1 met with the CNVJ commissioners and planned a project to include 40 interviewer teams, ten data processors, and five data entry specialists. The interviewing was to be done in July and August 1995, and the report produced by mid-December 1995. The commission’s final report was given to President Aristide in February 1996, but because of policy disagreements in the Haitian government, it was not published until September 1996, and then in a printing of only 75 copies. A second edition was published in February 1997.2

The CNVJ team took 5,453 interviews. In all, they identified 8,667 victims who suffered 18,629 violations. The CNVJ interviewing was quite good by scientific standards. A data processing group composed of eleven of the interviewers applied standard definitions to the raw interview data and produced detailed regional analyses, incorporating qualitative material from the interviews, as well as historical, economic and demographic analysis. Unfortunately, in the last stages of the process, the commissioners discarded almost all the work the field investigators did and substituted a chronology of the de facto regime. The commissioners never informed the AAAS of their reasons for not using the regional data; although the statistical analyses were presented, the tables omitted most of the content and the translations into French were inadequate. Thus, observers should not judge the quality of the field research by the AAAS team for the CNVJ on the basis of the published official report.

A statement of the prevailing atmosphere appeared in Le Monde Diplomatique:3

To the great disappointment of all, this report [the final CNVJ report submitted to President Aristide on 4 February, 1996], was for some strange reason was hidden in the files of the minister of justice, M. P- Pierre-Max Antoine, for many months. After many protests, only small parts of this report were published. The public and the many victims still wait for its publication in Creole. The majority of the final recommendations were never enacted. Former perpetrators occupied positions in the new national police or as prison guards: one of them was even in the security guard of the national palace even though his name appeared in Appendix 4 of the final CNVJ report (Page 1-b, code P 0402). However, a truth commission without the ability to punish can only be effective if it can fully inform society at large, and transform the judicial system and the functioning of administrative bodies. Nothing has been done; paralysis, inertia, and inactivity reign.

The failure to publicize the report was the responsibility of the administration of President René Préval, not of the commissioners.

Interviews, Data Processing, and the Database

Interviews

The sequence of data collection in the CNVJ interviews was as follows. A person, the dénunciateur4 comes to the interview team to give information about an abuse. The violent events being reported may have happened to the dénunciateur, and no one else. Or, the dénunciateur may be reporting abuses that happened to other people. Thus, the dénunciateur may or may not be a victim. Furthermore, there may be other victims. Thus a single interview may yield information about one, two, three, or any number of victims.

Each victim may have suffered one or many violations. The violations may have happened at one or many points in time. That is, a victim may have been detained and tortured on one date in one place, but raped and murdered on a subsequent date in a different place.

Furthermore, one or many identifiable perpetrators may have committed each violation. That is, "Antoine" and "Pierre" were responsible for the hypothetical detention and torture in the previous paragraph, but "Pierre" and "Michel" committed the rape and murder. Complex relationships among these various entities existed, and were captured in the interviews.

The teams conducted more than 7,000 interviews. Interview teams consisted of one Haitian and one international team member. This pairing was largely to satisfy an explicit mandate of the CNVJ that interview teams comprise both Haitians and internationals. Few internationals – even Francophones – speak Haitian Créole so that it was an absolute necessity that each interview team includes a Haitian.

Data Processing

A serious handicap to the work of the CNVJ was the late start of data processing, which did not begin until the teams had completed interviewing at the end of August 1995. This was in large part due to a significant leadership vacuum that took some months to fill. This lapse occurred because of serious political differences and the lack of staff experience with research of this kind. As a consequence, the directors of the interviewing teams resigned in late August 1995.

Analysts were chosen from among the interview teams, and they applied the codes following the methods that most projects use. Again, like most projects, the definitions of key concepts changed and new ideas were added to the analysis after most of the interviews had been coded. The analysts re-coded the entire set of interviews at least three times, although during the second and third reads on each interview they were re-coding only for specific themes.

All data entry was done by means of FoxPro "browse windows," with field and record level validation. For example, all the codes (for types of violation, geography, victim or perpetrator references) were checked as the users typed them to at least assure that the codes were valid. The six workstations were all freestanding. There was no sharing or serving of files across the network (although the machines did share a printer).

All entered data were aggregated into a single database on a central machine. We achieved this result without duplication or loss of entered data by assigning a unique block of key values to each workstation. Before aggregation, the relational integrity of the data was checked by tracing each foreign key to the primary table from which it originated.5 This was necessary because the database software used (FoxPro for Macintosh) did not do internal integrity checking. The data were merged into the common database on which the analysis was run. Our major programming task was to carry out the preceding functions essential to creating a common database with assured data integrity.

As part of the aggregation process we standardized several codes that were not originally controlled in the data entry, such as political affiliation of the victims. We achieved this result by creating a unique list of all the phrases and abbreviations found in the free text field. An analyst reviewed these selected phrases and abbreviations and assigned a code to each of the text fields. We then merged the code and text combination back into the original data, thereby assigning a code to each of the previously uncontrolled fields.

But even with the recoding, many analytic categories remained poorly reported. For example, many deponents were reluctant to report their political affiliation, and so the only analysis that could be done on this field used only a small fraction of the data and was therefore unstable.

Database

We designed the CNVJ's database to the standards established in (Ball, et. al., 1994). In accordance with those standards, we followed two fundamental rules:

1. The database must not introduce additional ambiguity into the data. That is, to the extent that the original sources permit, the database must be absolutely precise regarding who committed which violations against whom.

2. To represent a wide range of abuses, interventions, people, organizations, and to unambiguously represent the complex relations among these entities the database must be as extensive and amenable to change as is consistent with available resources.

As we have mentioned, any of the entities in a human rights violation may have complex relationships with none, some, or all the other entities; it is important that the data model enables an appropriate representation in the database. The diagram below shows in schematic form the data representation model that we used for the CNVJ.

The database designed to represent the processed data in accordance with this model permitted the CNVJ database to represent the complex stories the dénunciateur reported to the CNVJ interview teams.

Analyses

The teams conducted 5,453 interviews, in which we heard about 18,629 violations that were committed against 8,667 victims. During the course of the project, we carried out many statistical analyses using the relational structure described above as a basis for analyzing counts of violations by type, time, and geographic location.

A discussion of some of the statistical analyses follows. Analysis at the CNVJ met with a set of problems similar to those dealt with at the TRC and CEH.6 Continuous efforts to establish and maintain data quality at all stages kept the database itself in a state of change until hours before the results had to be reproduced for distribution. The challenge for the system designers – and to some extent, those that implemented their design – was to define the entire analytic process in ways that supported dynamic updating.

As we have discussed at greater length elsewhere, our experience has convinced us that every human rights project is unique and has different attributes from the others.7 Some analyses are general and likely to be common to most human rights projects (such as victim, perpetrator, witness attributes, and violations by time and place), but many of the analytical issues are particular to a given project.

In the following sections, we first describe the nature of many of the analyses performed that are of the general category, and then discuss several of the analyses that were particular to this project.

Victims and Violations

We analyzed victims by age category (infant, child, adult, elderly), profession, sex, and affiliation.8 As we mentioned earlier, the affiliation data are not reliable because they were not recorded consistently. Each of these analyses is repeated in a monthly time series for each of the CNVJ’s categories (Life, Integrity and Liberty, and Property). Table 1 shows these categories and the types of violations within the categories.

A subset of analyses reported on the violation of rape. These analyses included a histogram of victims’ ages, affiliation, the number of total female victims by department and month and corresponding proportion raped, and the proportion of rape victims assaulted in a named place such as a barracks or a military post.9 (Note that the affiliation data are weak for the reasons already stated.)

To understand the nature of the problems with affiliation and to extract such limited information as we could from the incomplete and inconsistent data, we determined the proportion of victims with some affiliation. For this purpose, we coded a victim as having affiliation if there was any text in the relevant descriptive field. In most of the analyses for which there is sufficient data for reliable estimates, the percent of victims with an affiliation was from 50-70%. Other analyses include victims driven into internal refugee status,10 finding their proportions by department, age and sex of the victim.

Perpetrators

We analyzed perpetrators with monthly and annual time series by CNVJ human rights violation category (Life, Integrity and Liberty, and Property) and by the affiliation of the perpetrator. Table 2 shows the five categories of perpetrator that we tracked (FadH, FRAPH, Police rural, Attaché, and Other). Violations without perpetrators identified by category were not included even if there were perpetrators identified for other violations against the same victim at the same time and place. We also did these analyses by department.

Table 2. CNVJ Human Rights Violation Perpetrator Categories and Types

There are special considerations in analyzing data about victims and perpetrators in combination. One or many violations could have happened to each victim. Thus, sums of the numbers of violations are usually significantly different from sums of numbers of victims. This disparity is logical, since in a given interview, a violation may have been committed on the same victim more than once at several dates. Similarly, none, one, two, or many identified perpetrators may have committed each violation. Consequently, no count of perpetrators from one or more given organizations can be summed with counts for other perpetrating organizations unless the perpetrators are combined in categories, as we describe below.

Our final analysis in this section looked at combinations of perpetrators. Since any violation may have been committed by one or more perpetrators who were not identified, or one, two, or many identified perpetrators, we had to combine categories of perpetrators to analyze how actual violations were committed. For example, it is clear that the Haitian army alone committed the bulk of violations in which a perpetrator is identified. Also, it is much easier to identify "two soldiers" than to identify random civilians. Substantial numbers of violations were committed by the Attachés working in conjunction with the FRAPH, by the militia working with the Haitian army, and by the militia. However, the single largest category is "no identified perpetrator."

National and Departmental Trends

To determine national trends, we analyzed the number of CNVJ human rights violation categories by month and by department. We also analyzed the individual and joint occurrence of detention and extortion by month and by department, and the relative proportion of all the possible combinations of torture, detention, and extortion. We found that extortion happened much more frequently in the presence of detention or the combination of detention and torture than it does alone or in conjunction with torture.

However, our principal finding was that detention, detention and torture in combination and torture are the most frequent combinations of abuse. The number of violations of these three types was much greater than the number of killings.

We also analyzed the counts of the CNVJ human rights violation categories and summarized the counts of different violation types by commune for each department. In this analysis and in the subsequent analyses in which departmental breakdowns were used, Port-au-Prince was separated from the Ouest and Raboteau is separated from l’Artibonite.

Validity of the Survey with Respect to Time

Findings concerning the distribution of violations in time were the next section of the report. For example, we noted that for the national and departmental breakdowns, October and November 1991, and October 1993, were exceptionally violent periods. They showed significant, large and increased numbers of all categories of human rights violations. To be sure of the validity of these findings, we wanted to assure that the non-probabilistic sampling and the nature of the interviewing process had not led to any non-representative selection of interviewees. One way to confirm or disconfirm this hypothesis was to compare the CNVJ team’s time distribution of violations in time with other human rights violation data for the period September 1991 to October 1994.

Fortunately we were given access to a dataset based on cadaver data collected at the University Hospital by Mercedes Dorretti of the Argentine Anthropological Forensic Team (EAAF; see Dorretti and Cano, n.d.).11 From this EAAF dataset, we determined the number of cadavers found each month, and plotted these counts against the monthly counts of summary executions from our survey data.

The two data series measured different variables and covered different geographical regions. Thus it is extremely unlikely that they will be alike in their values or time series unless they have a common underlying source. If the process of violations generated both series, we expect them to increase and decrease at the same time. When the level of human rights violations increases, there should be more cadavers in the morgue and more dénunciateurs reporting killing of members of their family or friends. The goal was to measure how much correlation there was between the monthly movements (rises and falls) of the EAAF and CNVJ killings data.

We measured this effect by calculating the increases or decreases of killings from each month to the subsequent month, which are statistically called first differences. The first data point was calculated as the difference in the number of killings in September 1991, from October 1991 in the CNVJ data. The next point is the difference between October and November in the same data, the third between November and December, and so on. We repeated these calculations for the months up to October 1994.

To determine if these two series represent different views of the same phenomenon (and thus confirm that the CNVJ data are representative), we used the first differences of both series to determine the extent to which the two monthly series move together despite the large difference in their absolute values.

We determined correlated movement in two ways:

1. Graphical, by visual examination of the scatterplot of the CNVJ and the EAAF first differences.

2. Analytically, by computing the correlation coefficient to numerically express the extent of the correlation.

From the scatterplot, a viewer can readily ascertain that there is a strong tendency for a rise in the EAAF series to be coincident with a rise in the CNVJ series, and similarly, for coincident behavior of the declines. In addition, there is an obvious tendency for the magnitude of the differences to correlate: the greater the rise in one series, the greater the rise in the other.

For the analytical measurement, we calculated the Pearson Product Moment Correlation Coefficient (r) these series to be r=.865 (r²=.748), which is quite high. If these data were from random samples from two series in which there was no correlation among first differences, the probability of such a high correlation occurring by chance is vanishingly small (z = 10.4, p < .000000).12

This was strong evidence of a similarity in the two series, confirming the representative validity of the CNVJ team data’s representation of the trends of violations.

Validity of the CNVJ Survey with Respect to Location

The sample of dénunciateurs was not distributed in exact proportion to the number of people living in each location. For example, approximately 33% of Haitian people live in the Ouest department, where the team performed only 18% of its interviews. We felt it necessary to deal with the possibility that our findings regarding inter-departmental differences in absolute numbers of violations are due in part to our area sampling bias.13

Our technique for estimating the amount of over- and under-sampling by department is to compute the ratio of the proportion of all Haitians living in a given department to the proportion of interviews collected in that department. This ratio is denoted sampling weight. The departmental sampling weights ranged from .273 to 2.009 (mean value 1.11, standard deviation .63).

Using these sampling weights, we recalculated the numbers of different types of violation by department at the national level, including Raboteau and Port-au-Prince. The principal result of applying these sampling weights is that the Ouest department becomes even more extreme in its number of violations, having more violations than other departments. Even in the raw data, the Ouest (usually with Port-au-Prince separated) had been one of the departments with the most numerous violations. This effect is consistent across different violation types, but is most pronounced for execution and disappearance. In violations of the rights to life and property, Nord and l’Artibonite seem also to have more violations than the other departments in the weighted data, but violations are more evenly distributed across departments in violations of the rights to integrity and liberty.

We concluded that controlling for area sampling bias by this method did not affect our findings.

Correlation Analysis of the Violation Time Series

Summary

The analysis in this section deals with the tendency of the numbers of violations of different types to increase or decrease at the same times and the tendency of numbers of a particular violation type to rise and fall together in time across different departments. As we discussed above, we argue that finding high correlation among a number of types of violations and different departments is strongly suggestive that the perpetrators of different types of violations, and the perpetrators of the same type of violation in different departments, are responding to similar influences that lead them to act at the same times. In both analyses, we found the hypothesized high correlations in these data and conclude that the human rights violations were consistent in time across different kinds of violation, and that the same kinds of violation were consistent in different departments.

Similarity of trends among different kinds of violations

How similar are the trends in time of the different types of violation? By similarity, we mean the tendency of the trends to rise and fall together. If different types of violence tend to occur at nearly the same times, and to rise and fall together, then we have a strong suggestion that people who perpetrate violence are responding to the same influences. If, however, some kinds of violence follow very different patterns in time, then we would have to conclude that influences to commit violence are different for different kinds of violence or that the data that we have do not support the contention that they shared a common influence.

As we did previously (in Validity of the Survey with Respect to Time), we compared the EAAF University Hospital cadaver data and the CNVJ summary execution data. We obtained the correlation of the first differences of the monthly counts of violations for each of ten types of violation. Table 3 shows the violation codes and Table 4 shows the correlation coefficients for these violations.

Table 3: Violation codes.

Table 4. Correlation matrix of monthly first differences between different types of human rights violations at the national level in Haiti, September 1991 to October 1994.

We performed hypothesis tests on each of the coefficients to determine which of the coefficients were significantly different from zero at the a = .01 level. This is a more stringent requirement than the usual a = .05 level. Of the 45 correlation coefficients, 45 were statistically significant at the a = .01 level.

Because we made multiple hypothesis tests, we could not assume that the significance level is truly a = .01. To determine the possibility that this many correlation coefficients could have been found significant by chance, we determined the probability that out of the 45 "trials," all 45 of these hypothesis tests were truly significant.

Since the probability of a hypothesis test turning out to be significant by chance is .01 (by our choice of a = .01), we modeled this process with the binomial distribution for 45 trials and 45 "successes" each with a probability of success of p=.01. The result is p<.000000; it is essentially impossible to have this many significant correlation coefficients by chance. The violation counts are measuring an underlying, common phenomenon. Additional support for this contention comes from the magnitude of the correlation coefficients. The coefficients range from .61 to .98. The mean is .85, and the median, .87. These coefficients are not only statistically significant (i.e., not zero), they are practically significant; these are strong correlations. Additional support from this statement is found by looking at the r² values in Table 5.

Table 5: Matrix of square of the correlation coefficients (r²) for the monthly first differences between different types of human rights violations at the national level in Haiti, September 1991 to October 1994.

Similarity of trends between departments

For any given type of violation, how similar are the trends in time for different departments? That is, does the number of arbitrary detentions in the Nord-Ouest increase and decrease in the same periods as the number of arbitrary detentions in l’Artibonite?

We sought an answer to this question in the same way that we did for the similarity of trends between different types of violations. Table 6 shows the number of significant coefficients by violation type. The second column of Table 6 presents the probability of obtaining that many significant coefficients by chance. Note that in all but the case of arbitrary executions, it is essentially impossible that we would find this many significant coefficients by chance.

Table 6. Number of significant correlation coefficients for the monthly first differences between different types of human rights violations by department in Haiti, September 1991 to October 1994.

For all of these kinds of violations, it would be extremely unlikely to find so many significant correlation coefficients by chance (e.g., for rape, seven significant correlation coefficients in 45 pairs would occur by chance on average 16 times in ten billion trials). We find that the departments are much more consistent for the violations for which there are many more instances of this kind (torture, arbitrary detention, attacks on goods). More than half of all possible pairs of departments has significant, nonzero correlation coefficients between their monthly first differences of instances of torture and arbitrary detention.

Although the findings are weaker, it is also true that instances of rape and sexual abuse, extortion, and arbitrary execution are consistent across departments. Across all the kinds of violations examined here, the number of significant correlation coefficients found is sufficient to find that for these kinds of violations, the violence was committed consistently in time across different departments.

A concern of the project was the extent to which data could support the hypothesis that there was national influence on local perpetrators.14 We took the approach of seeing if the time series for the several departmental series for relevant variables were in fact, more correlated in their trends than would be likely by chance. Similarity of movements of a variable in all departments would be taken as evidence of a national influence. For example, if in most departments rape rose during the same several months and fell during others, this would be an indication of some kind of national control.15

Lessons Learned

References

Ball, Patrick. 1996. Who Did What to Whom? Planning and implementing a Large Scale Human Rights Data Project. Washington, DC: American Association for the Advancement of Science.

Ball, Patrick, Ricardo Cifuentes, Judith Dueck, Romilly Gregory, Daniel Salcedo, and Carlos Saldarriaga. 1994. A Definition of Database Design Standards for Human Rights Agencies. Washington, DC: American Association for the Advancement of Science and Human Rights Information and Documentation Systems International.

Dorretti, Mercedes, and Ignacio Cano, n.d. "Violations of the Right to Life in Haiti: 1985 to 1995." Paper available from authors.

1 Drs. Patrick Ball and Daniel Salcedo comprised the AAAS team.

2 A version of the report that does not include the appendices describing the work of the AAAS team is available at www.haiti.org/truth/table.htm

3 Roussiere, D. and Danroc, G., "Soif de justice en Haïti," Le Monde Diplomatique, May 1998, pp. 22-3. The original text follows:

Ce rapport, à la grande déception de tous, est étrangement resté, durant de très longs mois, caché dans les tiroirs du ministre de la justice, M. Pierre-Max Antoine. Après de nombreuses protestations, celui-ci ne l'a publié qu'au compte-gouttes. La population et les nombreuses victimes attendent toujours sa diffu-sion en créole. La majorité des recommandations finales n'ont pas été mises en oeuvre. D'anciens bour-reaux ont occupé des fonctions dans la nouvelle police nationale ou encore comme gardiens de prison : l'un d'eux était même dans le corps de sécurité du palais national alors que, pourtant, son nom figurait dans l'annexe 4 du rapport final CNVJ (page 1-b code P 0402). Pourtant une Commission vérité, sans compétence pénale, ne peut être efficace qu'en informant largement la société civile et en transformant véritablement le système judiciaire ainsi que le fonctionnement des administrations. Rien n'y a fait, la paralysie, l'inertie et le laxisme demeurent.

4 The standard terminology for a person complaining about a human rights violation in the Central American and Caribbean regions is denunciador (Spanish) and dénunciateur (French and Créole). The word is more closely related to the English "complain or "report" than to "denounce."

5 When a primary key from a table is incorporated into another table to form a relationship between the ta-bles, it is called a "foreign" key.

6 At this point, we recommend that the concerned reader read or reread relevant sections of Chapters 3, 4, and

7 See the introductory chapter in this volume.

8 In French, denoted appartenance.

9 In French, caserne and avant-poste, respectively;

10 For convenience, we refer to these internal refugees by the French term, marronage, which literally means "runaway."

11 Dr. Mercedes Doretti and her international team of forensic experts were a second component of AAAS assistance to the CNVJ.

12 In this case, n=38. For n>30, when the population correlation coefficient is zero, the standard error is sqrt((1-r2)/(n-2)), and the sampling distribution is normal. In this case, (1-r2) = .252, (n-2)=36, and the stan-dard error is .084; z=.865/.084=10.4.

13 Our analyses of trends in time are independent of differences in the absolute magnitudes of the series being considered. Accordingly, issues of regional coverage in the sampling do not affect the trend analyses.

14 It was beyond the scope of the AAAS scientific component of the CNVJ activities to attempt to determine that nature of that influence.

15 While national motivation or stimulation of actions is a likely cause, it is also possible for apparently disorganized local perpetrators to communicate about their activities or be the recipients of information coming from other departments.

Science and Human Rights Program

Copyright © 2000