Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: TagsDialog crashes #16356

Merged
merged 6 commits into from
May 21, 2024
Merged

fix: TagsDialog crashes #16356

merged 6 commits into from
May 21, 2024

Conversation

david-allison
Copy link
Member

@david-allison david-allison commented May 5, 2024

Purpose / Description

  • Fix a TransactionTooLargeException and an OutOfMemoryError

This is a tradeoff of stability vs performance. We should do much better here in a follow-up

Fixes

Approach

  • Serialize to a file, rather than use Args
  • Use asSequence to fix an OOM
    • This makes the dialog process 2x slower: we need to iterate the selected notes twice
    • This may actually be faster after the PR on some collection, as we perform select distinct nid BEFORE loading the notes
  • Add a progress dialog to help the user

How Has This Been Tested?

My S21, AnKing deck (31479 cards)

No longer crashes on AnKing
⚠️ It is still FAR too slow, needing to individually load 31k cards. This requires a follow-up issue

Checklist

  • You have a descriptive commit message with a short title (first line, max 50 chars).
  • You have commented your code, particularly in hard-to-understand areas
  • You have performed a self-review of your own code
  • UI changes: include screenshots of all affected screens (in particular showing any new or changed strings)
  • UI Changes: You have tested your change using the Google Accessibility Scanner

@david-allison

This comment was marked as outdated.

@david-allison
Copy link
Member Author

david-allison commented May 5, 2024

For future

Index: AnkiDroid/src/main/java/com/ichi2/anki/CardBrowser.kt
IDEA additional info:
Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP
<+>UTF-8
===================================================================
diff --git a/AnkiDroid/src/main/java/com/ichi2/anki/CardBrowser.kt b/AnkiDroid/src/main/java/com/ichi2/anki/CardBrowser.kt
--- a/AnkiDroid/src/main/java/com/ichi2/anki/CardBrowser.kt	(revision 3c3f39d7de6756959acdd2b1fdc9c89473a257ff)
+++ b/AnkiDroid/src/main/java/com/ichi2/anki/CardBrowser.kt	(date 1714909328656)
@@ -1295,14 +1295,21 @@
                 // TODO!! This is terribly slow on AnKing
                 // PERF: This MUST be combined with the above sequence - this becomes O(2n) on a
                 // database operation performed over 30k times
+                val potentialTags = allTags.toMutableSet()
                 val uncheckedTags = withCol {
                     selectedNoteIds
                         .asSequence() // reduce memory pressure
                         .flatMap { nid: NoteId ->
                             progress++
-                            val note = getNote(nid) // requires withCol
-                            val noteTags: List<String?> = note.tags
-                            allTags.filter { t: String? -> !noteTags.contains(t) }
+
+                            // avoid DB access if no more tags to check
+                            if (potentialTags.isEmpty()) return@flatMap emptyList<String>()
+
+                            val noteTags = getNote(nid).tags.toSet() // requires withCol
+                            val uncheckedTagsForNote = potentialTags - noteTags
+                            // PERF: if we're returning these tags, we don't need to check them again
+                            potentialTags -= uncheckedTagsForNote
+                            uncheckedTagsForNote
                         }
                         .distinct()
                         .toList()

@david-allison
Copy link
Member Author

I've added a progress dialog as well

Copy link
Member

@BrayanDSO BrayanDSO left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Performance-wise, it definitely can be improved and should be improved, but I agree with you that fixing and testing it should be the priority

Json.decodeFromString<TagsData>(inputStream.convertToString())
}

@Serializable
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is @Serializable necessary here? I don't see a TagsData instance being serialized anywhere.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

                                                                             kotlinx.serialization.SerializationException: Serializer for class 'TagsData' is not found.
                                                                             Please ensure that class is marked as '@Serializable' and that the serialization compiler plugin is applied.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that replacing this with @Parcelize should work (and adding : Parcelable to the class), and Parcelables tend to be faster than Serializables, but I'm not going to block on that

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I couldn't get this working, happy for someone to take on a follow-up

@mikehardy mikehardy added the Needs Author Reply Waiting for a reply from the original author label May 12, 2024
@mikehardy
Copy link
Member

@david-allison this one looks like it needs a look from you?

@david-allison
Copy link
Member Author

david-allison commented May 13, 2024

@mikehardy defer to 2.18.1 I feel. Nether of these are regressions in 2.18, and this needs more time

Copy link
Member

@mikehardy mikehardy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

esp with the progress dialog this seems like a huge improvement -

first - just not crashing
second - does have some user feedback now

seems very worthwhile to get in as a quick fix so AnKing users aren't crashing

@mikehardy mikehardy requested a review from BrayanDSO May 21, 2024 12:45
@mikehardy mikehardy added Needs Second Approval Has one approval, one more approval to merge and removed Needs Review labels May 21, 2024
Json.decodeFromString<TagsData>(inputStream.convertToString())
}

@Serializable
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that replacing this with @Parcelize should work (and adding : Parcelable to the class), and Parcelables tend to be faster than Serializables, but I'm not going to block on that

Since the dialog accepts all tags, this can crash
just by having too many tags

fix by using `TagsFile` serialized to disk

Note: using Parcelable due to API 25 issue: 15878

Fixes 16226 - too many cards
Fixes 16351 - too many tags
This fixes an OutOfMemoryException on 31479 cards
(AnKing)

It is still exceptionally slow, to the point of being unusable

Issue 16226
The operation is really slow

* show (some) feedback to user
* stop log spam, speeding it up
* remove ANR

* adds a `withContext`

Issue 16226
Better optimizations are possible
but I want test coverage first
@BrayanDSO BrayanDSO added Pending Merge Things with approval that are waiting future merge (e.g. targets a future release, CI wait, etc) and removed Needs Second Approval Has one approval, one more approval to merge labels May 21, 2024
@BrayanDSO BrayanDSO enabled auto-merge May 21, 2024 23:03
@BrayanDSO BrayanDSO added this pull request to the merge queue May 21, 2024
Merged via the queue into ankidroid:main with commit ecc777d May 21, 2024
8 checks passed
@github-actions github-actions bot removed the Pending Merge Things with approval that are waiting future merge (e.g. targets a future release, CI wait, etc) label May 21, 2024
@github-actions github-actions bot modified the milestones: 2.18.1 release, 2.19 release May 21, 2024
@mikehardy mikehardy mentioned this pull request May 23, 2024
5 tasks
@david-allison david-allison deleted the 16226-tags branch June 3, 2024 18:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Crash on switching app while edit tags dialog is open [BUG]: crash on clicking "Edit tags"
3 participants