-
-
Notifications
You must be signed in to change notification settings - Fork 3.8k
Actions and issues
As a user plays through an exploration, we want to track their progress to identify any problems. If we do find problems with the exploration, we can raise those to the exploration creator. For example, if a user gets stuck in a cycle of cards or submits many incorrect answers to an interaction, the exploration could probably be improved.
We store the steps a user takes through an exploration as actions, and we create issues to keep track of the problems that these actions indicate. Both actions and issues are defined as schemas.
There are three actions defined at extensions/actions/
:
-
ExplorationStart
: This action is recorded when a learner starts an exploration. It stores the name of the state (i.e. card) they start at. -
AnswerSubmit
: This action is recorded when a learner submits a response to an interaction. It stores the ID of the interaction, the name of the state with the interaction, the answer that the learner submitted, the exploration's feedback for that answer, the next state they transition to, and how long the learner spent in the state. Note that the interaction ID is just its name, for exampleItemSelectionInput
. -
ExplorationQuit
: This action is recorded when a learner leaves an exploration. It stores the name of the state at which the learner quit and how long the user spent in the last state they visited.
These actions are recorded using the functions provided by the stats reporting service. For example, when a learner transitions into a terminal state (a card that ends the exploration), the conversation skin directive calls StatsReportingService.recordExplorationCompleted()
.
When a learner's journey through an exploration goes smoothly, we don't store the actions anywhere; however, when we detect a problem in a learner's journey, then we create an issue that stores the associated actions. Note that at no point during this process do we store or process any information about the learner's identity.
We define three different issues at extensions/issues/
:
-
MultipleIncorrectSubmissions
: This issue is recorded when a user submits 3 or more incorrect answers to an interaction. It stores the name of the state where the user provided incorrect answers and the number of incorrect answers that they submitted.The logic determining whether to file this issue lives in the
MultipleIncorrectAnswersTracker
class in the playthrough service. -
CyclicStateTransitions
: This issue is recorded when a user repeats a cycle of states 3 times in a row. Note that our cycle detection isn't particularly clever. As soon as we detect a cycle, we check whether it's equal to the most recent cycle we detected earlier. If it matches, we increment a counter. Otherwise, we reset the counter to 1. Here are some examples:-
A -> B -> A -> B -> A -> B -> A
: An issue would be filed because the cycleA -> B -> A
happened 3 times. -
A -> B -> A -> C -> A -> B -> A -> C -> A -> B -> A -> C -> A
: An issue would not be filed even though the cycleA -> B -> A -> C -> A
repeats 3 times. The reason is that we will only detect the smallerA -> B -> A
andA -> C -> A
cycles, which alternate. Since they alternate, we keep resetting our counter to 1 and so never reach 3 consecutive cycles.
The logic behind this cycle detection lives in the
CyclicStateTransitionsTracker
class in the playthrough service.This issue stores a list of the state names that make up the cycle.
-
-
EarlyQuit
: This issue is recorded whenever a user quits an exploration after fewer than 300 seconds. It records the name of the state where the user quit and the time they spent on the exploration.The logic behind this cycle detection lives in the
EarlyQuitTracker
class in the playthrough service.
Right now, neither issues nor their associated actions are surfaced to exploration creators. They used to be available through the improvements tab of the exploration editor, but that tab has been removed. We are currently working on building a new improvements tab.
Have an idea for how to improve the wiki? Please help make our documentation better by following our instructions for contributing to the wiki.
Core documentation
Developing Oppia
- FAQs
- How to get help
- Getting started with the project
- How the codebase is organized
- Making your first PR
- Debugging
- Testing
- Codebase policies and processes
- Guidelines for launching new features
- Guidelines for making an urgent fix (hotfix)
- Testing jobs and other features on production
- Guidelines for Developers with Write Access to oppia/oppia
- Release schedule and other information
- Revert and Regression Policy
- Privacy aware programming
- Code review:
- Project organization:
- QA Testing:
- Design docs:
- Team-Specific Guides
- LaCE/CD:
- Developer Workflow:
Developer Reference
- Oppiabot
- Git cheat sheet
- Frontend
- Backend
- Backend Type Annotations
- Writing state migrations
- Calculating statistics
- Storage models
- Coding for speed in GAE
- Adding a new page
- Adding static assets
- Wipeout Implementation
- Notes on NDB Datastore transactions
- How to handle merging of change lists for exploration properties
- Instructions for editing roles or actions
- Protocol buffers
- Webpack
- Third-party libraries
- Extension frameworks
- Oppia-ml Extension
- Mobile development
- Performance testing
- Build process
- Best practices for leading Oppia teams
- Past Events