Quality Assurance

Ensuring Quality Work

Quality work adheres to specific criteria established by the company. These criteria include accuracy, completeness, and adherence to guidelines.

First Process: Community Checking

Beyond meeting these criteria, the work must also be approved by a sufficient subset of the community. This means that after a task is completed, it is reviewed by several community members who evaluate whether it meets the established standards. For instance, if a comment is posted, community members will check its relevance and appropriateness. If the majority agree that the task meets the criteria, it is considered valid.

In our system, we use the concept of the Schelling point to ensure consensus within the community. This means that the validation relies on the collective agreement of the community, ensuring that tasks are performed to a high standard and approved by a representative group. For example, if all checkers listening to an audio recording agree without consulting each other that there is background noise, we simply consider that there is background noise in the recording. If the company wanted such noise, the data is considered valid (it matches the company's expectations); otherwise, it is invalid.

Now that we have clarified the concepts of quality and validity, let's see how users are incentivized to produce good quality data. Each time a user performs an action (production or vote), they lock in a deposit (a small amount of tokens).

When a producer's data is validated by the checkers, TADA tokens are added to the prize pool, and the producer earns XP, allowing them to climb the ranking. The users with the most XP earn the most tokens. Conversely, if the data is not valid, they earn nothing and lose their deposit. The deposit is a structural incentive system for producers and checkers, ensuring that people will do a good job, even if it is insignificant compared to the gains. This setup has deep roots in DeFi and blockchain protocols in general and is proven to work.
For the checkers, the mechanism is slightly more complex and relies on a consensus calculation among the voters. Basically, the checkers who voted against the majority are slashed. For example, if four checkers indicate hearing an elderly person in an audio recording and only one checker indicates hearing a child's voice, the four checkers are paid while the last one is slashed.

Second Process: Automatic Checking

While the community-based verification method is highly effective for ensuring the quality of AI-related datasets, such as audio recordings, we have also implemented automated systems to verify other types of tasks. These automated verification systems are designed to efficiently and accurately validate user actions, such as social media interactions.

For example, in verifying that users have retweeted a specific post, the automated system performs the following steps:

Task Assignment: Users are assigned the task of retweeting a specific post according to the project's requirements.
Action Tracking: The system automatically tracks the completion of this task by monitoring users' social media activities.
Verification: The system verifies that the retweet action has been completed by checking the user's social media account for the specified retweet, interfacing with the social media platform's API.
Validation: Upon confirmation of the retweet, the system marks the task as “in review.” Based on the project’s parameters, an automatic verification process will initiate, taking anywhere from a few minutes to a few days. If successful, the task is validated and the user is rewarded accordingly.
Incentives and Penalties: Similar to the community-based verification, users receive rewards in the form of points upon successful completion of tasks. If the task is not completed as required, the user may lose their deposit, discouraging fraudulent activities (e.g., retweet, check, unretweet).

By leveraging automated verification systems, we ensure that tasks such as social media engagement are validated quickly and accurately, reducing the manual effort required and increasing overall efficiency. This dual approach, combining both community-based and automated verification, enables Ta-da to handle a diverse range of tasks with the appropriate level of scrutiny and accuracy needed for each use case.

PreviousNext step: Ta-da chain NextEconomy

First Process: Community Checking

When a producer's data is validated by the checkers, TADA tokens are added to the prize pool, and the producer earns XP, allowing them to climb the ranking. The users with the most XP earn the most tokens. Conversely, if the data is not valid, they earn nothing and lose their deposit. The deposit is a structural incentive system for producers and checkers, ensuring that people will do a good job, even if it is insignificant compared to the gains. This setup has deep roots in DeFi and blockchain protocols in general and is proven to work.

For the checkers, the mechanism is slightly more complex and relies on a consensus calculation among the voters. Basically, the checkers who voted against the majority are slashed. For example, if four checkers indicate hearing an elderly person in an audio recording and only one checker indicates hearing a child's voice, the four checkers are paid while the last one is slashed.

Second Process: Automatic Checking

For example, in verifying that users have retweeted a specific post, the automated system performs the following steps:

Task Assignment: Users are assigned the task of retweeting a specific post according to the project's requirements.

Action Tracking: The system automatically tracks the completion of this task by monitoring users' social media activities.

Verification: The system verifies that the retweet action has been completed by checking the user's social media account for the specified retweet, interfacing with the social media platform's API.

Validation: Upon confirmation of the retweet, the system marks the task as “in review.” Based on the project’s parameters, an automatic verification process will initiate, taking anywhere from a few minutes to a few days. If successful, the task is validated and the user is rewarded accordingly.

Incentives and Penalties: Similar to the community-based verification, users receive rewards in the form of points upon successful completion of tasks. If the task is not completed as required, the user may lose their deposit, discouraging fraudulent activities (e.g., retweet, check, unretweet).