Evaluating
How to define methodologies and broadcast trust signals into the Fide Context Graph.
Evaluating is the Trust-Signal Authoring Path of the Fide Context Protocol. This page covers how applications define methods and issue evaluation statements. For read-time aggregation and weighting, see Query & Consensus.
An evaluation is simply a Statement where:
- Subject: The entity being evaluated
- Predicate: An EvaluationMethod ID
- Object: A score, verdict, or level
1. Defining a Methodology
Before you can evaluate, you must define HOW you evaluate. This is done by creating an EvaluationMethod entity.
Looking for EvaluationMethod ID calculation?
Method Types
Your method type depends on the subject and intent:
| Type | Purpose | Examples |
|---|---|---|
| Reputation | Trust scoring for any entity | Community Trust Score, PageRank |
| Fact-Check | Verifying statements | FactCheck-bot, Statement Verification |
| Quality | Assessing product/content quality | Code Audit, NSFW Detection |
| Compliance | Regulatory checks | KYC Verification, SOC2 Review |
Output Styles
Every method must declare its output scale so indexers know how to interpret the value:
| Output Type | Values | Use For |
|---|---|---|
| Verdict | -1, 0, 1 | Discrete judgments (False/Unverified/True) |
| Score | 0 to 100 | Continuous ratings (0=Bad, 100=Perfect) |
| Level | 1 to 5 | Ranked tiers (1=Low Priority, 5=Critical) |
2. Issuing Evaluations
Once your methodology is defined, you can start issuing evaluations.
Structure
An evaluation is a signed statement where the predicate is your method ID.
const evaluationStatement = {
subjectFideId: "did:fide:0x10...", // Alice (Person)
predicateFideId: "did:fide:0xe0...", // Your EvaluationMethod (see Entities)
objectFideId: await calculateFideId("CreativeWork", "CreativeWork", "85") // The Value "85" (did:fide:0x66...)
};Updates & Corrections
To update a score, use the Correction Pattern:
- Issue a new statement with the new score.
- Link it to the old statement via
prov:wasRevisionOf.
3. Persistence Strategy
Not all evaluations need to be stored as signed statements.
Deciding when to store (create immutable signed statements) vs. compute (calculate on demand) is a critical architectural choice.
| Strategy | Pattern | Best For | Trade-offs |
|---|---|---|---|
| Signed Statements | Store every verdict | Audits, Binary Decisions (True/False), Policy Violations | ✅ Full History ❌ High Write Load |
| Snapshots | Store periodic updates | Reputation Scores (0-100), Dynamic Metrics | ✅ Good balance ✅ "Current" queryable |
| Computed | Never store, API only | Real-time counts, High-frequency signals | ✅ Zero storage ❌ No history/audit |
Standard Evaluation Methods (Reference Patterns)
These are reference method patterns you can implement as EvaluationMethod entities. They are intended as practical defaults, not mandatory protocol predicates.
Default methods are represented as EvaluationMethod entities with stable names (for example Fide-StatementAccuracy-v1) and protocol-native IDs (for example did:fide:0xe0...).
Semantic Alignment Methods
- Statement Predicates — Evaluate semantic relationships between predicates (
owl:equivalentProperty,owl:inverseOf) - Entity Identity — Determine whether two identifiers refer to the same real-world entity (
owl:sameAs)
General Evaluation Methods
- Statement Evaluation — Assess factual accuracy and consensus of statements (StatementAccuracy, BridgingConsensus)
- Content Evaluation — Evaluate satisfaction and readability of creative works (TaskSatisfaction, NonTechnicalReadability)
Where is Reputation?
Trust is subjective. We deliberately do not define a standard "Reputation Score" or "Reliability Metric" in v1. Valid strategies range from simple whitelists to complex PageRank algorithms.
Developers should build their own reputation models by aggregating the atomic signals above (Facts, Tasks, Content) according to their own risk tolerance.
Anti-Patterns: What is NOT an Evaluation Method?
- Binary Relationships: If the output is just a link (e.g., "I accept this PR"), use a Relationship Predicate such as
schema:revieworschema:creator, typically not an evaluation. - Future Predictions: Evaluations measure past or present quality. Predictions about the future are Statements, not Evaluations.
- Undefined Subjectivity: If the method cannot be explicitly described as an algorithm or recipe, it is typically an Attribute (e.g., "favorite color"), not an Evaluation Method.
Next Steps
Now that you've created evaluations, how do applications use them?
- Query & Consensus — How to read, aggregate, and weight these signals to form ground truth.