AI Skills Assessment: Test Work, Not Trivia

Written by (Tileo)

Updated October 202610 min read

In briefLearnAIthing's rubric tests a real workflow through evidence, exceptions, ownership, and handoff, so teams assess work instead of recall.

AI skills assessment scorecard showing a workflow brief, evidence, exceptions, ownership, and handoff

An AI skills assessment should show how a current employee handles one bounded, real workflow from their role. LearnAIthing's model reviews the work the employee produces under a clear brief. It is not a general hiring test, a trivia quiz, a permanent score, or a claim about the person's competence across every task.

The review covers the workflow, evidence, exceptions, ownership, and handoff. Its result guides the employee's next task and support.

What does the AI skills assessment scorecard measure?

LearnAIthing recommends assessing one bounded workflow from the person's real role. The scorecard records what a reviewer can inspect, not what the person says they understand.

LearnAIthing performance dimension

Evidence to inspect

Not yet observable

Observable with support

Observable independently for this bounded task

Frame the work

Workflow map, objective, inputs, constraints, and acceptance evidence

The work sample does not make the task or its boundaries inspectable.

The person frames the task with help from a reviewer.

The person makes the task, boundaries, and acceptance evidence inspectable.

Choose and constrain the AI use

Prompts, instructions or configuration, source references, and stated constraints

The work sample does not show where AI is used or constrained.

The person chooses and constrains the AI use with support.

The person makes the chosen AI use and its constraints inspectable.

Build the workflow

Workflow map, prompts, instructions or configuration, and sample outputs

The evidence does not show a reviewable workflow.

The person builds a reviewable workflow with support.

The person builds a reviewable workflow for the bounded task.

Evaluate outputs and handle exceptions

Sample outputs, evaluation notes, and exception record

The work sample does not show output evaluation or an exception path.

The person evaluates outputs and records exceptions with support.

The person evaluates outputs and handles the recorded exception path for the bounded task.

Document ownership and handoff

Runbook, named owner, and handoff recipient

The work sample does not make ownership or handoff inspectable.

The person documents ownership and handoff with support.

The person documents the owner, runbook, and handoff for the bounded task.

These five dimensions and the three scoring anchors are LearnAIthing's first-party operating model. They are not an industry standard. Record the role, workflow, brief, constraints, evidence requirements, and review date beside the scores. That context prevents a result for one task from becoming a claim about every task the person may face.

The European Commission's DigComp provides a common understanding of digital competence and a basis for digital-skills policy. European Commission Joint Research Centre, Digital Competence Framework DigComp describes digital competence as relevant to finding and critically interpreting information, creating, editing, and sharing digital content, managing privacy, safety, and wellbeing, and using digital platforms and services. European Commission Joint Research Centre, Digital Competence Framework

DigComp does not endorse LearnAIthing's five dimensions. LearnAIthing cites it only to support examining several observable competence areas instead of reducing digital competence to tool recall. European Commission Joint Research Centre, Digital Competence Framework

What should the assessment brief contain?

LearnAIthing recommends giving the person a brief before reviewing the work. Use one bounded workflow from the person's role and keep the assessment tied to that workflow.

Copy this assessment brief:

  1. Role and workflow. Name the person's role and the single workflow under assessment.
  2. Objective. State what the workflow must accomplish.
  3. Allowed inputs. List the inputs the person may use.
  4. Constraints. State the boundaries that apply to the work.
  5. Acceptance evidence. Define what the person must show so a reviewer can inspect the result.
  6. Exception path. State the exception the person must identify, record, or handle.
  7. Handoff recipient. Name the person who must be able to receive the work.
  8. Assessment date. Record when the evidence was reviewed.

The bounded task matters. LearnAIthing does not recommend generalizing from one workflow to universal competence. The assessment answers this task-specific question: can this person turn this role task into a bounded, reviewable AI-assisted system under these constraints?

What belongs in the evidence pack?

LearnAIthing's evidence pack makes the work inspectable after the person finishes the task. Collect the artifacts below:

  • A workflow map that shows the bounded task.
  • The prompts, instructions, or configuration used in the workflow.
  • Source references used for the work.
  • Sample outputs from the workflow.
  • Evaluation notes for those outputs.
  • An exception record.
  • A runbook for the handoff.
  • A named owner.

In LearnAIthing's model, do not substitute a confident explanation for missing artifacts. Confidence can inform support, but it does not replace the observable work sample. The same rule applies to tool vocabulary. Naming a prompt pattern, model, or product does not provide the workflow map, output evidence, exception record, or runbook that this model asks a reviewer to inspect.

The evidence pack ties each decision to visible work. It shows what remains absent or needs support without treating the score as a description of the whole person.

How do the five performance dimensions work?

The five dimensions below are part of LearnAIthing's first-party operating model. They describe what to inspect in one bounded work sample.

Frame the work

Inspect whether the person makes the objective, allowed inputs, constraints, acceptance evidence, exception path, and handoff recipient clear. The task boundary belongs in the work sample. A reviewer should not have to infer what the workflow is meant to do.

Choose and constrain the AI use

Inspect the prompts, instructions or configuration, and their source references. Check whether the evidence shows where AI is used and which constraints bound that use.

This dimension includes constraints and risks because LearnAIthing's model requires them to be part of the work. NIST developed the AI Risk Management Framework to help manage risks to individuals, organizations, and society that are associated with AI. NIST, AI Risk Management Framework AI RMF 1.0 is intended for voluntary use and helps organizations incorporate trustworthiness considerations into the design, development, use, and evaluation of AI products, services, and systems. NIST, AI Risk Management Framework

NIST's Generative AI Profile can help organizations identify generative AI risks and proposes risk-management actions aligned with organizational goals and priorities. NIST, AI Risk Management Framework

LearnAIthing cites these NIST resources only to support examining constraints, risks, outputs, and ownership as part of work. LearnAIthing does not present its rubric as a NIST framework. NIST, AI Risk Management Framework

Build the workflow

Inspect the workflow map, prompts, instructions or configuration, and sample outputs together. LearnAIthing recommends looking for a bounded system that another person can review. A loose collection of tool actions is not enough when the steps and outputs cannot be inspected as one workflow.

Evaluate outputs and handle exceptions

Inspect the sample outputs, evaluation notes, and exception record. The person needs to show how they reviewed the outputs and what happened on the exception path named in the brief.

Under LearnAIthing's decision rules, an unresolved quality or safety exception blocks independent-operation status for that workflow. Strong evidence in the other dimensions does not cancel the exception. The block applies to the workflow under review. It does not label the person or predict performance on unrelated work.

Document ownership and handoff

Inspect the runbook, the named owner, and the handoff recipient. LearnAIthing's model treats handoff as part of the work sample. The build is not independently observable for this bounded task if ownership and the instructions needed for handoff remain unclear.

For a team that wants capability transfer as the next step:

Explore the Deployed Business Operator path

Can self-assessment replace observed assessment?

No, not in LearnAIthing's model. A self-assessment can collect confidence and perceived gaps. It cannot replace the observable work sample.

Use the two forms of assessment for different evidence:

  • Self-assessment records how confident the person feels and where they believe they need support.
  • Observed assessment records what a reviewer can inspect in the brief, workflow, outputs, exception record, runbook, ownership, and handoff.

A mismatch between confidence and observed evidence is not a character judgment. It may show that the task needs tighter boundaries, support, or a smaller practice task.

For the capabilities that may sit behind a work sample, read what AI skills are.

How should you interpret outcomes without pass/fail theater?

LearnAIthing's three task-specific outcome anchors are part of its first-party operating model:

  • Ready for a bounded build. The required evidence is observable independently for this bounded task, with no unresolved quality or safety exception.
  • Ready with support. The work is observable when the person receives support on a dimension or an exception path.
  • Needs a smaller practice task. The current work sample does not yet make enough of the bounded task observable.

These outcomes guide the next task and support decision. They are not permanent labels.

Do not compress the scorecard into pass or fail. Keep each dimension, its evidence, and the recorded constraints visible. A person may build the workflow independently while still needing support with exceptions or handoff.

Use the evidence to size the next bounded task and identify support. An AI training plan for employees or AI training for teams can provide the next learning context.

How should managers calibrate the results?

Calibration turns a reviewed work sample into a next-work decision. The sequence and decision rules in this section are LearnAIthing's first-party operating model. They are not a research finding or an industry standard.

Use this calibration sequence:

  1. Confirm that reviewers assessed the same bounded workflow, brief, constraints, evidence requirements, and review date.
  2. Inspect the artifacts behind each dimension. Do not begin with a total score or a person's confidence.
  3. Record missing evidence and unresolved exceptions as separate evidence states. Do not average them into stronger dimensions.
  4. Use the decision table to choose the next bounded task, the support required, and the owner of that support.
  5. Keep the cited artifacts, the decision, and the review date together. Set the next review against the new task rather than carrying forward a permanent status.

LearnAIthing's operating rule is to compare like-for-like only. Compare results only when the bounded workflow, brief, constraints, evidence requirements, and review date are the same. If any of those items differ, keep the results separate. The difference may come from the task design rather than the employee's work.

Observed evidence state

Manager decision

Next bounded task

Support or owner required

Evidence to retain

Independently observable workflow, with all required evidence and no unresolved quality or safety exception

Record independent operation for this workflow only

Repeat the same bounded workflow under the recorded constraints, or assign an adjacent workflow under a new brief

Name the workflow owner and the next reviewer

Brief, workflow map, prompts or configuration, outputs, evaluation notes, exception record, runbook, owner, and review date

Workflow observable with support

Record ready with support for this workflow

Repeat the same workflow with the needed support made explicit

Name the reviewer, coach, or specialist who supplies that support

Supported artifacts, support provided, remaining gap, owner, and review date

Missing output evaluation

Do not record independent operation

Evaluate a defined set of outputs against the brief's acceptance evidence

Name a reviewer who can check the evaluation method and decision

Sample outputs, acceptance evidence, evaluation notes, reviewer decision, and review date

Unhandled quality or safety exception

Block independent-operation status for this workflow until the exception is handled

Run the named exception path and record the response

Name the person who owns the exception decision and the reviewer who confirms closure

Exception input, observed response, decision, corrective action, owner, and review date

Missing owner or handoff

Do not record the workflow as independently observable

Complete a handoff using the runbook with a named recipient

Name the workflow owner and the handoff recipient

Runbook, owner, recipient, handoff notes, open questions, and review date

This table preserves evidence instead of averaging people. An unresolved quality or safety exception remains attached to the workflow and blocks independent-operation status even when other dimensions are strong. It does not label the employee.

Managers can use this evidence map alongside an AI pilot program. Keep each result tied to its work sample. A team view can show available evidence, support needs, owners, and open exceptions. It should not rank people or imply universal ability.

FAQ

How do you test AI skills at work?

LearnAIthing recommends testing one bounded workflow from the person's real role. Give the person a brief with an objective, allowed inputs, constraints, acceptance evidence, an exception path, and a handoff recipient. Review the resulting evidence pack against LearnAIthing's five dimensions and three scoring anchors.

What are the top AI skills to assess?

In LearnAIthing's model, assess the ability to frame the work, choose and constrain the AI use, build the workflow, evaluate outputs and handle exceptions, and document ownership and handoff. These are first-party categories for a bounded work sample, not a universal list or industry standard.

Can I use an AI skills self-assessment?

Yes, to collect confidence and perceived gaps. In LearnAIthing's model, self-assessment does not replace observed evidence from the person's work sample.

How much does an AI skills assessment cost?

LearnAIthing's public model does not prescribe a price. The reusable brief, evidence pack, dimensions, and scoring anchors in this article define what to assess without turning the article into a vendor quote.

What happens if someone does not complete the task?

Treat the attempt as evidence about task sizing and support. The appropriate outcome may be ready with support or needs a smaller practice task. Do not turn the attempt into a permanent judgment about the person.

Should an assessment include practice questions and answers?

LearnAIthing's model does not score trivia answers. It scores observable work and the evidence attached to a real role workflow. A fake answer key would reward recall instead of showing whether the person can build and hand over the bounded system.

What should a manager do after the assessment?

Go to the calibration table and match the work sample to its observed evidence state. Use that row to choose the next bounded task, name the required support or owner, and retain the listed evidence. Compare the result with another result only when the workflow, brief, constraints, evidence requirements, and review date match.

For a team that wants capability transfer as the next step:

Explore the Deployed Business Operator path