Research & evaluations
Evaluation methodology
Definitions, calculation rules and evidence requirements for yute research.
What is being evaluated
| Object | Measure | Limit |
|---|---|---|
| Documented product capabilities | Criterion-level editorial assessment | Not a substitute for hands-on testing |
| Recommendation behavior | Mentions and recommendations in recorded responses | Specific to the sampled queries and collection surface |
| Category demand | Provider estimates or first-party observations | Units and populations must be identified |
| Commercial outcomes | Client referral, inquiry and CRM data | Separate from model mentions |
Evaluation design
Define the category, vendor inclusion rules, country, language and buyer requirements before collecting results. Keep questions that name vendors separate from discovery questions. Record the reason for the selected sample and repeat observations where needed to assess variability.
Collection and provenance
Record the query, time, collection surface, model identifier when available, search settings, response, cited URLs and failure status. An API response is not interchangeable with a consumer application response. Version extraction rules and retain the original response before interpretation.
Scoring and uncertainty
Specify the rubric and weights for each documentary score. Show missing values rather than silently replacing them with zero. Recalculate totals from their components and disclose rounding. For response metrics, show numerator, denominator and failed calls; account for related questions and repeated runs when estimating uncertainty.
Interpretation
A citation identifies an observed reference. It does not prove why a vendor was recommended. Changes in results may reflect query wording, system updates, content changes or sampling variation. State the supported conclusion and distinguish hypotheses from measurements.
Release requirements
A release includes scope, protocol version, observation dates, results, evidence inventory, limitations, publisher and review status. Supporting materials may be public or subject to access restrictions, which must be explicit.
Review and corrections
Record actual authors, reviewers and relevant conflicts. Preserve earlier releases when making corrections. Independent review is a status supported by a review record, not a label assigned at publication.
Publication policy