Join the Eval Army.
Shape the Standard.
Alt.QA is establishing a global human evaluation corps, and announcing eval.qa, our upcoming benchmark and open standard for AI trust.
Evaluation is infrastructure. You are the signal.
eval.qa
We're building a modular, human-plus-machine evaluation standard to define, score, and improve AI behavior, across outputs, agents, and workflows.
eval.qa is not an internal tool. It is a public benchmark and open specification designed to help the entire ecosystem measure trust.
eval.qa is stewarded by Alt.QA with public input through an open RFC process.
Derived from live evaluation pipelines used within Alt.QA.
The Eval Army
The Eval Army is a distributed corps of evaluators defining what "good" AI behavior actually looks like.
Output Evaluation
Assess AI-generated content across real tasks. Score accuracy, coherence, helpfulness, and safety using standardized eval.qa templates.
Failure Mode Discovery
Identify hallucinations, bias, inconsistencies, and edge cases that automated systems miss. Surface the problems that benchmarks don't catch.
Benchmark Piloting
Test and refine early benchmark datasets. Validate evaluation models. Provide the structured human judgment that grounds the standard.
Clear thinking and attention to detail matter more than credentials or background.
All evaluators are calibrated using standardized rubrics and inter-rater reliability checks.
Evaluation Is the Next AI Frontier
Everyone is training models.
Few are verifying them.
As AI systems become agentic, autonomous, and embedded in decision-making, evaluation can no longer be an afterthought.
We believe evaluation is infrastructure, as foundational as training, deployment, or monitoring.
What's Coming
Initial schema release
YAML-based schema for defining evaluation criteria and scoring rubrics.
Reference benchmark
Canonical model for testing and validating eval.qa implementations.
First public validation cycle
Integration with task definition standards for workflow evaluation.
Open RFC process
Alpha, Beta, Public, structured refinement of the standard.
Drop Your Hat
If you care about AI trust, evaluation, or the future of responsible intelligence, we want you involved.
Apply to Join the Eval ArmySelected participants will receive early access to eval.qa drafts and pilots.
eval.qa is part of Alt.QA, the quality and truth infrastructure of TAO.ai and The Work Company.
For private evaluation engagements with model teams, see EvalQA.