Adaptive Assessment¶

The BLIM model¶

The Basic Local Independence Model (BLIM) defines the probability of a response given a knowledge state, using two parameters:

\(\beta\) (slip): \(P(\text{incorrect} \mid \text{item mastered})\)
\(\eta\) (guess): \(P(\text{correct} \mid \text{item not mastered})\)

The identifiability constraint is \(\beta + \eta < 1\).

Items vs instances¶

In KST/ALEKS, the distinction between items and instances is fundamental:

An item is a problem type / competency (the latent level).
An instance is a concrete question (the observed level).

Each item can have multiple instances of equivalent difficulty. The BLIM update happens at the item level; the engine selects and presents instances to avoid repeating the same question.

One-shot assessment¶

For batch assessment from known responses:

import knowledgespaces as ks

structure = ks.space_from_prerequisites(
    ["add", "sub", "mul"],
    [("add", "sub"), ("sub", "mul")],
)

# Single observation per item
result = ks.assess(structure, {"add": True, "sub": True, "mul": False})

# Multiple observations (different instances of the same item)
result = ks.assess(structure, [
    ("add", True), ("add", True),   # two instances of addition
    ("sub", True),
    ("mul", False),
])

print(result["state"])        # most likely knowledge state
print(result["probability"])  # posterior probability
print(result["mastery"])      # per-item mastery probabilities
print(result["outer_fringe"]) # what to learn next
print(result["inner_fringe"]) # most recently consolidated

Adaptive assessment¶

The engine selects the most informative question using Expected Information Gain (EIG):

\[\text{EIG}(q) = H(\pi) - \mathbb{E}[H(\pi \mid R_q)]\]

where \(H\) is Shannon entropy and the expectation is over both possible responses.

With instances (recommended)¶

result = ks.adaptive_assess(
    structure,
    ask_fn=lambda instance_id: ask_student(instance_id),
    instances={
        "add": ["3+2", "7+5", "12+9"],
        "sub": ["8-3", "15-7"],
        "mul": ["4*3", "6*7"],
    },
    beta=0.1,
    eta=0.2,
    threshold=0.85,
    max_questions=15,
)

Simple mode (one question per item)¶

result = ks.adaptive_assess(
    structure, lambda item: item in {"add", "sub"}
)

Low-level control¶

For full control over the assessment loop:

from knowledgespaces import BLIM, BLIMParams, StatePosterior
from knowledgespaces.assessment import select_item_eig, is_converged

blim = BLIM(structure, BLIMParams(beta=0.1, eta=0.2))
posterior = StatePosterior.uniform(blim)

posterior = posterior.update("add", True)
posterior = posterior.update("sub", True)

print(posterior.entropy)
print(posterior.most_likely_state)
print(posterior.marginal_mastery())

Instance-level selection¶

from knowledgespaces.assessment import InstancePool, select_instance_eig

pool = InstancePool.from_dict({
    "add": ["add_q1", "add_q2"],
    "sub": ["sub_q1", "sub_q2"],
    "mul": ["mul_q1"],
})

best = select_instance_eig(posterior, pool, asked={"add_q1"})
print(best.instance_id)  # e.g. "sub_q1"
print(best.item)         # "sub"
print(best.score)        # EIG value