👥Consensus calculation

To provide a quick explanation of how the voting system works, this section will present a comprehensive example. We will explain the general algorithm step-by-step, which allows for the validation of data and the remuneration of the different actors involved.

Step 0: the context

A production task concerning the voice recording of an adult male with background noise was sent to a user. Once the data is produced by the user, it is checked by 5 checkers. Each of them has to verify three criteria:

1. The gender

2. The age group

3. The presence or the absence of background noise

Step 1: is the data valid?

This step aims to determine whether the data produced is valid. This means knowing if the data has a consensus among the checkers and if this consensus is in agreement with the instructions sent to the producer (adult male with background noise).

Below is the voting matrix of the 5 producers:

	Gender	Age	Background noise
Checker #0	M	Young	Yes
Checker #1	M	Adult	Yes
Checker #2	M	Old	Yes
Checker #3	F	Adult	Yes
Checker #4	F	Adult	Yes
Consensus	M	Adult	Yes
Confidence	3/5 = 0.6	3/5 = 0.6	5/5 = 1

In this table, each row corresponds to a vote. For example, checker #0 thinks that the data is produced by a young man and that there is background noise. Thanks to all the votes, it is possible to extract a consensus for each criterion as well as a confidence index. The higher the latter, the stronger the consensus for this criterion. For instance, when considering the "Gender" criterion, 3 checkers indicated "Male" and 2 indicated "Female". The majority answer is "Male", which represents 3 out of 5 votes (i.e., 60%). In regards to the background noise, all the voters agreed on the same answer, which represents 100% agreement.

In this example, the general consensus indicates that the data produced comes from an adult male with background noise. As the general consensus corresponds in all points to the production instructions, the data is validated and the producer is paid.

Step 2: checkers remuneration

The compensation and slashing of checkers is a function of the distance between their vote and the consensus. The closer the checker's vote is to the consensus, the higher their compensation. Proximity is determined by the following formula: $P = {\text{number of common criteria with the consensus} \above{0.5pt} \text{total number of criteria}}$.

This formula is used to calculate the value of the “Proximity” column below:

	Gender	Age	Background noise	Proximity
Checker #0	M	Young	Yes	0.66
Checker #1	M	Adult	Yes	1
Checker #2	M	Old	No	0.33
Checker #3	F	Adult	Yes	0.66
Checker #4	F	Adult	Yes	0.66
Consensus	M	Adult	Yes
Confidence	3/5 = 0.6	3/5 = 0.6	4/5 = 0.8

Depending on the proximity, it is possible to determine if a user is slashed or not. For example, in the table above, checker #2 could be slashed while the others would be not (checker #0 has a proximity of 100% with the consensus, others have 66%).