BGPT: Paper Review: Learning about AI, learning with AI

Explore by Goal

Quick Explanation Copied

Paper focus (what’s being proposed)

The paper proposes integrating AI ethics reflection and AI-supported clinical skill practice across the University of Sydney’s Master of Clinical Psychology (MCP), redesigning two units: PSYC6089 (ethics via case-based learning + AI-related dilemmas) and PSYC6086 (skills via AI chatbot practice using Cogniti), aligned to TEQSA/USYD guidance on ethical, reflexive AI use and “two-lane” assessment design.

Long Explanation

Paper Review: Learning about AI, learning with AI

Rigorous, skeptical, visual critique of an AI-education curriculum proposal in clinical psychology training.

What the paper claims it will do

Integrate AI ethics reflection into PSYC6089 via scenario/vignette case-based learning and incorporate AI dilemmas into clinical viva assessment.
Integrate AI-supported clinical skill practice into PSYC6086 using Cogniti as an AI chatbot “learning partner” to simulate complex client interactions, requiring submission of a Cogniti transcript demonstrating skill engagement and learning.
Frame the overall approach using TEQSA/USYD guidance: promote ethical, reflexive AI engagement and use assessment structures that reduce risk while still enabling learning with AI (“two-lane” approach).

Visual 1 — Curriculum logic map (two units across two years)

Source: paper’s description of proposed unit changes and framing.

Key methodological observation (critical): it is a curriculum design proposal, not an empirical study

The manuscript provides a conceptual framework and assessment/unit redesign plan; the provided text does not include participant outcomes, effect sizes, or comparative evaluation results. Consequently, evidence strength is largely argumentative/theoretical rather than experimental.

Visual 2 — Assessment mapping to TEQSA-style “secure vs unsecured” framing

Paper links TEQSA guidance to a “two-lane” assessment approach and proposes viva-based AI ethics and transcript-based skills practice as its integration pattern.

Strengths (what looks scientifically “defensible”)

Explicit alignment to established educational policy framing: the paper ties its unit redesign to TEQSA guidance and USYD operational initiatives including reflexivity/ethical engagement and assessment redesign rather than blanket prohibition of AI use.
Mechanism-level pedagogical intent: “learning about AI” targets ethical decision-making with AI as an object of critique (via CBL and professional reasoning), while “learning with AI” targets skill acquisition through repeated interaction and feedback (via chatbot simulations and transcript demonstration).
Use of conceptual lenses: the paper grounds AI literacy as “more-than-digital” (postdigital) with distributed agency and a “fair position,” and additionally frames working with AI as engaging with black-box systems via relational epistemology and evaluative judgment.

Visual 3 — Risk map (what must be experimentally checked later)

Derived from the paper’s own stated challenges (e.g., staff readiness/cognitive load; authenticity/privacy constraints; black-box trust calibration).

Limitations / red flags (skeptical critique)

No reported empirical evaluation: the paper anticipates improved ethics reasoning and skill readiness, but the provided manuscript text does not report outcome data or validated measures to support causal claims.
Tool dependence risk: the “learning with AI” mechanism relies on Cogniti chatbot design/prompting quality and the agent’s fidelity for “complex/risky” client simulations. The paper flags staff capability/time as a barrier, implying the pedagogical effect may be confounded by implementation skill rather than curriculum theory.
Black-box calibration requires assessment design detail: while “black-box pedagogy” is cited as rationale, the proposal does not specify concrete rubrics for calibrating trust/verification behavior from students, beyond transcript submission. Without explicit rubrics/validation, students could learn superficial compliance (e.g., generating plausible reasoning) rather than rigorous epistemic standards.
Confidentiality constraint is acknowledged but not operationalized: the paper notes PSYC pedagogy previously excluded AI due to confidentiality concerns; the new design should specify data governance boundaries (what is processed, stored, and reviewed) to prevent “ethics-by-assertion.” The provided text mentions the concern and proposes integration, but the operational controls are not fully specified here.

What evidence would most change my mind (falsification points)

Ethics/clinical reasoning outcomes: demonstrate measurable improvement in AI-related ethical reasoning that generalizes beyond the curriculum tasks (not just improved viva talk). The mechanism claims draw on CBL and black-box pedagogy, but falsification would require robust measurement and transfer tests.
Simulation validity: show that chatbot-based practice improves downstream competencies compared to alternatives (e.g., non-AI role play/supervision) and does not produce harmful epistemic habits (e.g., over-trust). The paper relies on the general rationale that simulated practice can enable repeated learning without adverse consequences, but the actual comparator evidence is not provided here.
Governance robustness: prove confidentiality and privacy safeguards work under real student workflows; otherwise the ethical premise collapses under operational failure. The paper frames confidentiality concerns as a reason for prior exclusion, which heightens the need for explicit governance controls.

Visual 4 — Evidence hierarchy for this specific paper

This is a critique visualization based on whether the provided paper text contains outcome data and operational specifics, not a claim about the existence of data elsewhere (e.g., future evaluation).

Paper review metrics (requested)

Metric	Score (1–10)	Skeptical rationale (concise)
Novelty	7	Novel coupling of AI ethics reflection + AI chatbot skill practice specifically for MCP units; still builds on existing pedagogical framings.
Scientific quality	7	Argumentation is coherent and policy-referenced, but no empirical evaluation is reported in the provided text.
Generality	3	Highly contextualized to USYD MCP governance and Cogniti infrastructure; generalizability depends on implementing institutions’ policies and tool capacity.
Usefulness	7	Actionable blueprint for clinical psychology programs: unit-level goals, assessment types, and conceptual justification.
Reproducibility	4	Replicating depends on proprietary platform/tooling and staffing; details like rubrics/governance are not fully specified in the provided text.
Explanatory depth	7	Links educational mechanisms to epistemic/ethical theories (more-than-digital literacy; black-box pedagogy).

How to improve this kind of paper (actionable)

Provide an evaluation plan with pre-registered, validated instruments for (i) AI ethics reasoning quality, (ii) epistemic calibration (trust/verification behaviors), (iii) downstream clinical competence proxies, and (iv) privacy/governance operational details; include comparators and reporting standards to reduce implementation confounding.

Author review links (baked into BGPT)

Feedback:

Updated: April 15, 2026