PCN Questionnaire Augmentation

Validation Framework

2 min read

Validation Framework

Visual validation logic

flowchart LR
    B[Conventional questionnaire information] --> O[Outcome / criterion]
    B --> A[Augmented model]
    P[PCN information] --> A
    A --> C[Incremental value beyond questionnaire alone?]

See Figures for the full incremental-validity figure.

The project should define added value before examining specific network metrics. The central aim is to establish whether PCN augmentation contributes information beyond the underlying questionnaire.

1. Psychometric validity

Questions:

  • Are selected nodes stable?
  • Are perceived causal ratings reproducible?
  • Are global network characteristics more reliable than individual edges?
  • How much of the network reflects stable person-level structure versus momentary state?

Useful outcomes:

  • agreement in node selection,
  • edge-presence agreement,
  • correlations or ICCs for causal ratings,
  • rank stability of node-level centrality/strength,
  • stability of global network metrics.

2. Incremental validity

This is the central validation test.

Base model

Y ~ baseline Y + conventional questionnaire information

Augmented model

Y ~ baseline Y + conventional questionnaire information + PCN block

The PCN block should be prespecified and compact.

Candidate outcomes:

  • future symptom burden,
  • functional impairment,
  • symptom change,
  • treatment response,
  • help-seeking or treatment uptake,
  • subjective problem coherence.

Incremental value can be quantified using:

  • change in explained variance,
  • likelihood-ratio/model-comparison tests,
  • out-of-sample predictive performance,
  • calibration and prediction error,
  • preregistered block tests.

3. Person specificity

The augmentation should distinguish individuals who appear similar under conventional assessment.

A particularly intuitive demonstration is the Analysis/Same Symptoms Different Networks analysis: match individuals with similar questionnaire profiles and quantify heterogeneity in their causal networks.

4. Convergent and discriminant evidence

In nested validation samples, PCNs may be compared with:

  • narrative-derived networks,
  • EMA-derived temporal networks,
  • clinician formulations,
  • caregiver or informant models.

High structural identity should not automatically be expected because these methods assess different constructs. The question is instead whether there is theoretically meaningful convergence in key processes or network features.

5. Subjective representativeness

After network visualization, ask participants how well the network captures:

  • their problems,
  • relations among problems,
  • important drivers,
  • and their overall experience.

Representativeness can be studied both as an outcome and as a criterion for comparing alternative augmentation procedures.

6. Clinical or subjective utility

If networks are fed back to participants, evaluate whether they improve:

  • understanding,
  • coherence,
  • insight,
  • perceived personalization,
  • perceived control,
  • motivation,
  • and treatment relevance.

7. Generalizability

Because the project spans multiple questionnaires, analyses should distinguish:

  • instrument-specific effects, and
  • generic augmentation effects.

A multilevel or meta-analytic framework may allow network features to be tested across instruments while accounting for questionnaire and sample differences.

Graph View

Backlinks