[BUG]: The log event uploader does not check if there is internet connection

[masclot]

Describe the bug

We don't want events to be stored in local firebase, but rather sent directly to firebase servers. There is no way of enforcing this, as connectivity can drop at any time. Unfortunately, firebase library does not seem to tell give any information about what happens to an event after a FirebaseAnalytics#logEvent() is called.
To improve the situation we [indirectly] use NetworkUtils.isConnected() to check the network connection before sending an event to firebase. If there is no network connection, events are not sent to firebase. However, NetworkUtils only tells you if the phone is connected to a network, not whether Firebase servers are reachable.

For example, the network could have a firewall. Another example is when the phone connects to one of those "free" WiFi networks that intercept all network requests until the user signs in.
In this situation, the event log uploader will remove the events from the local cache and send them to firebase. Firebase will cache them locally until it runs out of storage (lose events), 72 hours have passed (lose events) or it can upload them itself (success).

This bug could be the culprit of issue #5063

Steps To Reproduce

This is theoretical. I haven't tried myself.
Use the Oppia app while offline. Connect to a Captive Portal Network without signing in. Wait for the LogUploadWorker to run. If the SyncStatusManager reports success, then all the events may be lost.

Expected Behavior

We cannot remove this problem entirely. We could make it better by checking real internet availability, instead of network connectivity.

Screenshots/Videos

No response

What device/emulator are you using?

No response

Which Android version is your device/emulator running?

No response

Which version of the Oppia Android app are you using?

No response

Additional Context

No response

[BenHenning]

This is a very good point, thanks for filing @masclot!

It actually reminds me also of #5064 as I think the true solution would also fix that problem. Firebase ideally needs to provide a way for us to monitor the success/failure of the event upload, then from our end we only use the internet connectivity as a signal to try rather than a signal to assume success (and subsequently only drop our records of events if the event upload succeeds).

Given the limitations in Firebase, we may need to consider uploading our events through Oppia web and having it proxy those events to whatever consolidate event framework we want to use long-term (this is something we're also discussing for feature flags & platform parameters to reduce the amount of infrastructure web needs to manage without also bloating or complicating the Android app). @seanlip curious to know your thoughts here--is the plan also for web to move toward Firebase for event tracking or keep everything within GA4?

FWIW I'm also assuming the same problem theoretically exists for web for both Firebase and GA4 (having no way to know if an event was successfully uploaded), but we could build a more resilient pipeline on web that does something like:

• Receive and queue events from frontends (Android + web).
• Periodically store events in a Firestore database (rather than Firebase analytics directly), or sends them to GA4 (though Firestore may be a better source of truth for GA4, too; something to consider), and mark these events as potentially uploaded.
• Periodically sweep & clean up events that are marked as uploaded by verifying they're in the datastore as expected (unless we can guarantee this with an atomic transaction in which case we can remove it during save). If the event doesn't get uploaded, mark it as not uploaded so that it gets picked up and tried again.

Some thoughts:

• One limitation here is that we'd have to use custom BigQuery queries and LookerStudio dashboards rather than getting to benefit directly from the built-in ones in Firebase (which really reduces the value proposition we receive from Firebase).
• We'd have to augment many of the properties we get automatically per event from Firebase.
• We could potentially have a BigQuery pipeline that attempts to backfill Firebase events from the database, but this probably won't work well since there's a lot of built-in properties set by Firebase we just don't know how to set (and would essentially be reinventing Firebase's Android frontend library itself).
• We could alternatively upload to both Firebase analytics & Firestore from the app using a unique GUID (or snowflake ID) per event that we then use to backfill missing events or remove duplicated events. This is a bit more robust, but more complex (and thus more moving parts that could fail in the future).
• We could bypass Oppia web and upload to Firestore directly in the client, but the confirmation of success is harder to do clientside and we could benefit from frontend consistency here (and ensure all Oppia analytics go to the same place which could have really great GTM & PM benefits).

This probably needs a lot more discussion to figure out a direction that we want to go.