Module 4: Apply DKT in Research/Practice

Deep Knowledge Tracing

Author

LASER Institute

Published

June 5, 2025

APPLYING DKT IN PRACTICE

LASER INSTITUTE KT TRACK BADGE ACTIVITY

  1. Please write a short essay and cite at least one paper in response to the following question. How could you apply Deep Knowledge Tracing to solve (just one) real problem in your research or in practice, that Bayesian Knowledge Tracing (BKT) wouldn’t be as good for?

    Guiding Questions

    • What specific problem are you addressing that requires DKT’s capabilities?

    • How does DKT’s ability to capture complex temporal patterns benefit your use case?

    • What aspects of your problem involve non-linear relationships that RNNs can model?

    • Why are BKT’s Markov assumptions insufficient for your chosen problem?

    • What type and scale of data would be available for training your DKT model?

  2. Input that essay to ChatGPT 4 or another Large Language Model of your preference, following the initial prompt:

    You are Chris Piech, a researcher with extremely detailed knowledge of Deep Knowledge Tracing (DKT) and an expert on its use within research and in intelligent tutoring systems. Please give feedback on the following essay written by an advanced doctoral student. Please give them feedback on the core ideas and their application on the research or real-world problem; do not give academic paper such as a recommendation to add citations. Please do not give recommendations around functionality that does not exist in DKT (e.g. engagement tracking), and do not make recommendations that apply to every knowledge tracing (e.g. paying attention to student privacy). Give only constructive, negative feedback:

  • What technical misconceptions about DKT were identified in your essay?

  • Which criticisms relate to the neural network architecture vs. the educational modeling?

  • How realistic were your assumptions about data requirements and model training?

  • What aspects of DKT’s limitations did you overlook?

  • Were your comparisons to BKT technically accurate?

  1. How would you modify your essay, if at all, in response to that feedback?


  1. Input your modified essay to ChatGPT 4 or another Large Language Model of your preference, following the initial prompt:

    You are an award-winning teacher who has developed curricula which leverage Deep Knowledge Tracing (DKT) and has spent thousands of hours watching and interacting with students using those curricula. Please give feedback on the following essay written by an advanced doctoral student. Please give them feedback on the core ideas and their application on the research or real-world problem; do not give academic paper such as a recommendation to add citations. Do not make scientifically inaccurate recommendations, such as recommending paying attention to learning styles. Give only constructive, negative feedback:

  • What practical implementation challenges did the teacher identify?

  • How does the “black box” nature of DKT affect classroom adoption?

  • What concerns about interpretability and actionability were raised?

  • How might teachers need to adapt their practices to work with DKT predictions?

  • What infrastructure or support would teachers need to implement your solution?

  1. How would you modify your essay, if at all, in response to that feedback?


  1. Input your modified essay to ChatGPT 4 or another Large Language Model of your preference, following the initial prompt:

    You are Maria, a thoughtful and highly intelligent 9th grade student who nonetheless sometimes forgets what she learned in class. You have just spent the last three weeks using a learning system in class, described by the following prompt. You have been asked to give your reflections on that learning system and how to make it better. Do not make scientifically inaccurate recommendations, such as recommending paying attention to learning styles. You say:

    (If this prompt does not fit your context of use, please modify it to make it fit!)

  • What aspects of the user experience did the student highlight?

  • How might DKT’s predictions feel different to students compared to simpler models?

  • What concerns about fairness or transparency emerged from the student perspective?

  • How could you make DKT-based recommendations more understandable to students?

  1. How would you modify your essay, if at all, in response to that feedback?