Diraya · دراية

EXPERT DATA + PROFESSIONAL EVALUATION INFRASTRUCTURE

Know whether your AI can handle real professional work.

Diraya evaluates models and agents in realistic professional workflows, diagnoses where they fail, uses qualified experts to create the right corrective signals, and verifies whether performance improved.

Evaluate a capability See what exists today

THE PROBLEM

A benchmark score is not evidence the AI can do the job.

An agent can answer test questions and still fail the real workflow: use the wrong source, miss a condition, act with incomplete information, or fail to escalate.

STATIC BENCHMARK

Did it answer the question?

An isolated prompt, an expected answer, and a score.

REAL PROFESSIONAL WORK

Can it complete the work responsibly?

Tools, evidence, policies, missing information, uncertainty, and human escalation shape the result.

Diraya evaluates the capability in the workflow where it must actually work.

Loop

Evaluation should lead to improvement—and evidence.

Diraya connects evaluation to the intervention it reveals, then tests again on protected material to measure what changed.

  1. Evaluate

    Test the intended capability in representative professional work.

  2. Diagnose

    Identify what failed, why, and which expertise is required.

  3. Improve

    Create the right expert corrective signal, data, or system change.

  4. Re-evaluate

    Run protected evaluation after the intervention.

  5. Verify

    Determine what improved, what remains unreliable, and where oversight is required.

The original Arabic correction loop now operates as one proof layer inside this broader lifecycle.

Capabilities

Three capabilities. One closed loop.

Each capability feeds the next: evaluation reveals failures, experts create targeted improvement signals, and protected verification measures the result.

  1. 01

    Expert Data & Model Improvement

    Qualified experts create high-value corrective data and signals based on observed capability failures—not generic bulk labels.

  2. 02

    Professional Evaluation Environments

    Models and agents are tested in realistic tasks containing workflows, tools, evidence, constraints, and domain context.

  3. 03

    Capability Evaluation & Verification

    Determine where a system succeeds, where it fails, and what it can reliably be trusted to do.

Built through focused pilots from existing Arabic evaluation and expert-production groundwork.

REALISTIC EVALUATION

Test the AI in the conditions that shape the outcome.

We build realistic professional evaluation environments—what we call Worlds—where models and agents face the workflows, tools, evidence, and constraints they will encounter in practice.

Real work may require the AI to use defensible evidence, recognize missing information, handle uncertainty, follow rules, abstain appropriately, and escalate when human judgment is required.

Worlds are an infrastructure direction being developed through focused evaluations—not a catalogue of mature deployed products.

EXISTING TODAY

A real foundation—not a finished platform.

Diraya's Arabic prototype provides real structured evaluation artifacts, while the internal workbench implements expert production, review, adjudication, versioned submissions, sources, provenance, and structured exports.

  • EXISTING

    Arabic + expert workflow foundation

    Real prototype artifacts and an internal workflow supporting expert production, review, adjudication, versioning, and export.

  • PILOT CAPABILITY

    One focused capability evaluation

    Define, evaluate, diagnose, improve where appropriate, and run protected re-evaluation.

  • BEING BUILT

    Reusable professional evaluation infrastructure

    Worlds, verifiers, professional judgment infrastructure, and lifecycle capability evidence.

  1. Evaluation task
  2. Versioned expert work
  3. Review / adjudication
  4. Sources + provenance
  5. Structured export
Inspect the original Arabic prototype artifact

ARABIC PROTOTYPE ARTIFACT

A real failure Diraya diagnosed

A model answer, proposed correction, draft reviewer rationale, and structured output from the original Arabic reliability workflow.

  1. 1. PromptThe task
  2. 2. Rejected (A)Model failure
  3. 3. Chosen (B)Expert correction
  4. 4. RationaleWhy B · why not A

Existing prototype draft: Formal Arabic failure with formal expert audit pending

DIRAYA-AR-001 · Nahw

Prompt

أعرِبْ ما تحته خطّ إعرابًا تفصيليًّا موجَّهًا لطالب الصفّ الثاني عشر: «سارَ القائدُ والجبلَ حتّى أدركَ الحصنَ». هل تُعرَبُ «الجبلَ» مفعولًا معه أم معطوفًا؟ ومتى يمتنع العطفُ هنا؟ أجب بتشكيلٍ كاملٍ للكلمةِ المعرَبةِ وبيانِ العلامةِ.

Rejected (A)

الواوُ هنا تحتملُ العطفَ والمعيّةَ، والأفصحُ المدرسيُّ عندَ جمعٍ من المعربينَ حملُها على العطفِ؛ فـ«الجَبَلُ» معطوفٌ على «القائدُ» مرفوعٌ مثلهُ، على تأويلِ مصاحبةٍ مجازيةٍ. أمّا المفعولُ معهُ فيحتاجُ عندَهم إلى قرينةٍ أقوى تمنعُ التشريكَ منعًا قاطعًا، وهي غيرُ متعيّنةٍ هنا. التشكيلُ على هذا الوجهِ: الجَبَلُ.

Proposed correction (B)

الراجحُ في هذا التركيبِ أنْ تُعرَبَ «الجَبَلَ» مفعولًا معهُ منصوبًا وعلامةُ نصبِه الفتحةُ الظاهرةُ، والواوُ واوُ المعيّةِ لا عاطفةٌ. يمتنعُ العطفُ هنا (على القولِ المدرسيِّ الشائعِ) لأنَّ المعطوفَ يشاركُ المعطوفَ عليهِ في الحكمِ، والجبلُ لا يسيرُ مع القائدِ حقيقةً؛ فامتنعَ التشريكُ وتعيَّنَت المعيّةُ. التشكيلُ المطلوبُ: الجَبَلَ (لا الجَبَلُ). أمّا القولُ بجوازِ العطفِ على تأويلٍ مجازيٍّ فخارجٌ عن مستوى الإجابةِ المدرسيّةِ المعتمدةِ ما لم يُنَصَّ عليهِ.

Draft reviewer rationale

الاستجابةُ (b) هي الصحيحة ؛ لأن القاعدة تقول: يَمْتَنِعُ الْعَطْفُ، وَيَجِبُ إِعْرَابُ الِاسْمِ بَعْدَ الْوَاوِ مَفْعُولًا مَعَهُ، عِنْدَمَا لَا يَصِحُّ أَنْ يُشَارِكَ الِاسْمُ الثَّانِي الِاسْمَ الأَوَّلَ فِي الْفِعْلِ، مِثْلَ: «سِرْتُ وَالنِّيلَ». النِّيلَ: مَفْعُولٌ مَعَهُ مَنْصُوبٌ، وَعَلَامَةُ نَصْبِهِ الْفَتْحَةُ الظَّاهِرَةُ؛ لِأَنَّ النِّيلَ لَا يَسِيرُ، بَلْ سَارَ الْمُتَكَلِّمُ بِمُحَاذَاتِهِ.

Why this mistake matters

A single case mark on الجبل flips whether the mountain shares the verb (coordination) or is only accompaniment (واو المعية). In education eval that is a lost mark; in official and institutional Formal Arabic it quietly rewrites who participates in the act. Fluent-looking models often pick the “safer” رفع reading and change the meaning.

علامة إعراب واحدة على «الجبل» تقلب المعنى بين مشاركة الفعل (عطف) ومجرد المصاحبة (واو المعية). في الامتحانات المدرسية خسارة درجة؛ وفي نصّ المنهج الرسمي قد يتغيّر من يشارك في الفعل. النماذج فصيحة الظاهر غالبًا تختار الرفع «الآمن» فتغيّر المعنى دون ضجيج.

Technical output (JSON)

DIRAYA-AR-001 · seeds_v1_msa_formal.jsonl

Truncated for display. Full Arabic is in the carousel. Download the complete row below.

View technical output (JSON) 

{
  "aato_id": "DIRAYA-UAE-2026-001",
  "seed_id": "DIRAYA-AR-001",
  "domain": "Arabic_Linguistics",
  "subdomain": "Nahw_Irab_MafoolMaahu",
  "prompt": {
    "user_input": "أعرِبْ ما تحته خطّ إعرابًا تفصيليًّا موجَّهًا لطالب الصفّ الثاني عشر: «سارَ القائدُ والجبلَ حتّى أدركَ الحصنَ». هل تُعرَبُ «الجبلَ» مفعولًا معه أم معطوفًا؟ ومتى يمتنع العطفُ هنا؟ أجب بتشكيلٍ كاملٍ للكلمةِ المعرَبةِ وبيانِ العلامةِ…"
  },
  "candidates": [
    {
      "id": "c0_model_rejected",
      "text": "الواوُ هنا تحتملُ العطفَ والمعيّةَ، والأفصحُ المدرسيُّ عندَ جمعٍ من المعربينَ حملُها على العطفِ؛ فـ«الجَبَلُ» معطوفٌ على «القائدُ» مرفوعٌ مثلهُ…"
    },
    {
      "id": "c1_proposed_gold",
      "text": "الراجحُ في هذا التركيبِ أنْ تُعرَبَ «الجَبَلَ» مفعولًا معهُ منصوبًا وعلامةُ نصبِه الفتحةُ الظاهرةُ، والواوُ واوُ المعيّةِ لا عاطفةٌ…",
      "rationale": "الاستجابةُ (b) هي الصحيحة ؛ لأن القاعدة تقول: يَمْتَنِعُ الْعَطْفُ، وَيَجِبُ إِعْرَابُ الِاسْمِ بَعْدَ الْوَاوِ مَفْعُولًا مَعَهُ…"
    }
  ],
  "preference": {
    "chosen_id": "c1_proposed_gold",
    "rejected_id": "c0_model_rejected"
  }
}

ARABIC + UAE

Arabic is our proving ground—not our boundary.

Diraya began with Formal Arabic evaluation in the UAE, where language, domain standards, source authority, and context expose failures broad benchmarks miss. That foundation remains a specialization as the infrastructure expands to professional AI more broadly.

  • SPECIALIZATION

    Formal Arabic reliability

    Language-specific evaluation where grammar, morphology, terminology, and context can change meaning.

  • PROVING ENVIRONMENT

    UAE + GCC context

    A regional starting point for testing source authority, institutional language, and context-sensitive professional work.

  • BROADER DIRECTION

    Professional AI beyond Arabic

    The same evaluate, diagnose, improve, and verify infrastructure is designed to extend to other specialized workflows.

Explore the Arabic specialization