thread '<unnamed>' panicked at 'called Option::unwrap() on a None value', in slideo/crates/matching-opencv/src/lib.rs:371:69 stack backtrace:

[LukasSchulz]

On some Videos/pdf-combinations I get this error on MacOS while slideo is processing frames: (This only happens on certain frames and always on the same ones)

Then, after completely processing the video the webserver isn't started and when the pdf is opened manually by executing slideo file.pdf the pdf doesn't have any timestamps.

Full Backtrace:

thread '<unnamed>' panicked at 'called `Option::unwrap()` on a `None` value', /Users/lukas/Documents/slideo/crates/matching-opencv/src/lib.rs:371:69
stack backtrace:
   0:        0x10bcf78de - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::h04abbaabf148650a
   1:        0x10bd3a98e - core::fmt::write::h14dac7cadec1cc70
   2:        0x10bcf708a - std::io::Write::write_fmt::hfaf2e10dfdcc61d8
   3:        0x10bd159e9 - std::panicking::default_hook::{{closure}}::h350fee0bf60f2674
   4:        0x10bd1555b - std::panicking::default_hook::h0b4e3bc46e6dcb8d
   5:        0x10bd15f7a - std::panicking::rust_panic_with_hook::h8cdc0a575f4a5a7b
   6:        0x10bcf8079 - std::panicking::begin_panic_handler::{{closure}}::h7a7b30fd1c313876
   7:        0x10bcf7a28 - std::sys_common::backtrace::__rust_end_short_backtrace::h2e099be83c81509d
   8:        0x10bd15af3 - _rust_begin_unwind
   9:        0x10bd473ef - core::panicking::panic_fmt::h4f63e4f6b62b650b
  10:        0x10bd47347 - core::panicking::panic::hb3c06f51e2e58141
  11:        0x10afa59a9 - core::option::Option<T>::unwrap::hb3706e21acd48f48
                               at /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_lang_rust/rust/work/rustc-1.51.0-src/library/core/src/option.rs:386:21
  12:        0x10b105cb2 - matching_opencv::OpenCVVideoMatcherTask<I>::match_images_with_frame::{{closure}}::h6ac2efa761b34801
                               at /Users/lukas/Documents/slideo/crates/matching-opencv/src/lib.rs:371:47
  13:        0x10b0e2242 - alloc::slice::<impl [T]>::sort_by::{{closure}}::h4e32b55bca503dbc
                               at /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_lang_rust/rust/work/rustc-1.51.0-src/library/alloc/src/slice.rs:311:33
  14:        0x10b0e0f62 - alloc::slice::insert_head::h22f59d896a4d25dc
                               at /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_lang_rust/rust/work/rustc-1.51.0-src/library/alloc/src/slice.rs:840:24
  15:        0x10b0dd846 - alloc::slice::merge_sort::h50931d4fdb6fb654
                               at /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_lang_rust/rust/work/rustc-1.51.0-src/library/alloc/src/slice.rs:1047:17
  16:        0x10b108531 - alloc::slice::<impl [T]>::sort_by::h296008c9323b96d6
                               at /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_lang_rust/rust/work/rustc-1.51.0-src/library/alloc/src/slice.rs:311:9
  17:        0x10b1054cb - matching_opencv::OpenCVVideoMatcherTask<I>::match_images_with_frame::h2e9e32b4065f5258
                               at /Users/lukas/Documents/slideo/crates/matching-opencv/src/lib.rs:371:13
  18:        0x10b103ea8 - <matching_opencv::OpenCVVideoMatcherTask<I> as matching::VideoMatcherTask<I>>::process::{{closure}}::{{closure}}::h69525bcbd55619bc
                               at /Users/lukas/Documents/slideo/crates/matching-opencv/src/lib.rs:214:36
  19:        0x10b0239d6 - rayon_core::scope::ScopeFifo::spawn_fifo::{{closure}}::{{closure}}::h14d50d9aafa378d9
                               at /Users/lukas/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.9.0/src/scope/mod.rs:495:47
  20:        0x10b116d31 - <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h82d2844e73051b6f
                               at /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_lang_rust/rust/work/rustc-1.51.0-src/library/std/src/panic.rs:344:9
  21:        0x10b00bf95 - std::panicking::try::do_call::h43dc9bd5aa331631
                               at /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_lang_rust/rust/work/rustc-1.51.0-src/library/std/src/panicking.rs:379:40
  22:        0x10b00de3d - ___rust_try
  23:        0x10b00a231 - std::panicking::try::h93de8b23b179aca6
                               at /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_lang_rust/rust/work/rustc-1.51.0-src/library/std/src/panicking.rs:343:19
  24:        0x10b1177e1 - std::panic::catch_unwind::haa922ddd00603855
                               at /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_lang_rust/rust/work/rustc-1.51.0-src/library/std/src/panic.rs:431:14
  25:        0x10b15dd11 - rayon_core::unwind::halt_unwinding::hb8f4d4b69b63ef43
                               at /Users/lukas/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.9.0/src/unwind.rs:17:5
  26:        0x10b023491 - rayon_core::scope::ScopeBase::execute_job_closure::h2b938e0ac14a5ca7
                               at /Users/lukas/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.9.0/src/scope/mod.rs:561:15
  27:        0x10b023425 - rayon_core::scope::ScopeBase::execute_job::h8bcba628953f50d9
                               at /Users/lukas/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.9.0/src/scope/mod.rs:549:29
  28:        0x10b02397d - rayon_core::scope::ScopeFifo::spawn_fifo::{{closure}}::hcd25299f0e0d3306
                               at /Users/lukas/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.9.0/src/scope/mod.rs:495:17
  29:        0x10b0d35dd - <rayon_core::job::HeapJob<BODY> as rayon_core::job::Job>::execute::ha6e2be962215648e
                               at /Users/lukas/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.9.0/src/job.rs:167:9
  30:        0x10b7532b5 - rayon_core::job::JobRef::execute::h9905210a4b77f273
                               at /Users/lukas/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.9.0/src/job.rs:59:9
  31:        0x10b7533c8 - <rayon_core::job::JobFifo as rayon_core::job::Job>::execute::ha3c023d7050ec006
                               at /Users/lukas/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.9.0/src/job.rs:211:50
  32:        0x10b7532b5 - rayon_core::job::JobRef::execute::h9905210a4b77f273
                               at /Users/lukas/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.9.0/src/job.rs:59:9
  33:        0x10b728e3d - rayon_core::registry::WorkerThread::execute::h52bdcf205839c2ce
                               at /Users/lukas/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.9.0/src/registry.rs:753:9
  34:        0x10bd41548 - rayon_core::registry::WorkerThread::wait_until_cold::h856430079a15650f
                               at /Users/lukas/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.9.0/src/registry.rs:730:17
  35:        0x10b728c78 - rayon_core::registry::WorkerThread::wait_until::ha9dbb2d013ef9bc2
                               at /Users/lukas/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.9.0/src/registry.rs:704:13
  36:        0x10b7295f3 - rayon_core::registry::main_loop::h2141c7566ae34282
                               at /Users/lukas/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.9.0/src/registry.rs:837:5
  37:        0x10b7264c0 - rayon_core::registry::ThreadBuilder::run::hccf2fa9a1df253e9
                               at /Users/lukas/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.9.0/src/registry.rs:56:18
  38:        0x10b7269d1 - <rayon_core::registry::DefaultSpawn as rayon_core::registry::ThreadSpawn>::spawn::{{closure}}::h70f699ea15ed07e4
                               at /Users/lukas/.cargo/registry/src/github.com-1ecc6299db9ec823/rayon-core-1.9.0/src/registry.rs:101:20
  39:        0x10b72f860 - std::sys_common::backtrace::__rust_begin_short_backtrace::hfdc1dafbe1be5b5f
                               at /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_lang_rust/rust/work/rustc-1.51.0-src/library/std/src/sys_common/backtrace.rs:125:18
  40:        0x10b73d8b1 - std::thread::Builder::spawn_unchecked::{{closure}}::{{closure}}::h8f274698cf915304
                               at /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_lang_rust/rust/work/rustc-1.51.0-src/library/std/src/thread/mod.rs:474:17
  41:        0x10b7534a4 - <std::panic::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h3f5ee8f562c0bcc0
                               at /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_lang_rust/rust/work/rustc-1.51.0-src/library/std/src/panic.rs:344:9
  42:        0x10b72ac45 - std::panicking::try::do_call::hc7ca871d6321fd0b
                               at /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_lang_rust/rust/work/rustc-1.51.0-src/library/std/src/panicking.rs:379:40
  43:        0x10b72cf8d - ___rust_try
  44:        0x10b72a6c1 - std::panicking::try::h67011bda65e0bc0b
                               at /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_lang_rust/rust/work/rustc-1.51.0-src/library/std/src/panicking.rs:343:19
  45:        0x10b753e94 - std::panic::catch_unwind::h03d7cd8b3a38bb63
                               at /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_lang_rust/rust/work/rustc-1.51.0-src/library/std/src/panic.rs:431:14
  46:        0x10b73d07e - std::thread::Builder::spawn_unchecked::{{closure}}::h7e67fa4f6020f6d7
                               at /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_lang_rust/rust/work/rustc-1.51.0-src/library/std/src/thread/mod.rs:473:30
  47:        0x10b741ed1 - core::ops::function::FnOnce::call_once{{vtable.shim}}::h9a7c2579ff92d2b4
                               at /opt/local/var/macports/build/_opt_bblocal_var_buildworker_ports_build_ports_lang_rust/rust/work/rustc-1.51.0-src/library/core/src/ops/function.rs:227:5
  48:        0x10bd2023b - std::sys::unix::thread::Thread::new::thread_start::h2ede5f70c14488e7
  49:     0x7fff6a7db109 - __pthread_start

Commenting out line 370 in the mentioined lib.rs seems to fix the issue even though I don't know why. The program also seems to still work fine after commenting out this line, because the same thing is written in line 329.

[hediet]

That is interesting.
This code applies the computed transformation to the frame so that the slide and the frame align. It then computes the similarity as f32 and compares them.
I don't understand why it cannot compare two f32s in this case - I guess one is NaN. But why?
Can you println!("{:?}", ...) the exact value that it cannot compare? Thanks!

Also, in general, how well does the tool work for you (besides these fixable hickups)?

[LukasSchulz]

Could you please tell me what exactly I should put where? I am not really that familiar with rust and couldn't get it to work with your instructions, sorry.

Otherwise the tool works really well. - And it is also only on the lectures I tried to run through slideo I get this specific error (maybe because the person is scrolling backwards in the slides? - I haven't really looked into you logic even though I read the background).

Sometimes it also seems to miss some pages if they look to similar to previous ones - at least that's what I think that happens, I don't know for sure. (Example, page 7 of this talk should be found at around minute 7 or 8 but it doesn't (at least for me). -> should I open another Issue for this?)

[hediet]

Could you please tell me what exactly I should put where? I am not really that familiar with rust and couldn't get it to work with your instructions, sorry.

Can you reproduce this issue with public slides/videos?

This statement instead of line 370 should report what goes wrong:

rated_best_matches.sort_by(|a, b| {
    b.2.partial_cmp(&a.2)
        .expect(&format!("Could not compare {:?} with {:?}", a.2, b.2))
});

I fear further debugging will be required though.

maybe because the person is scrolling backwards in the slides?

Slideo currently looks at each frame independently - reordering all frames should not change the result.

[hediet]

Sometimes it also seems to miss some pages if they look to similar to previous ones

If two consecutive frames look too similar (like only 1% of pixels change), the second frame is ignored. This is an optimization right after encoding the video.

Does this happen after you removed the sort_by? That sort_by should usually prevent this. Sometimes the transformation is not pixel-perfect and if all candidates are very similar, mismatches can happen.

[LukasSchulz]

Backtrace is the same as before (as it should be), output of println:
thread '<unnamed>' panicked at 'Could not compare NaN with 0.6858432', /Users/lukas/Documents/slideo/crates/matching-opencv/src/lib.rs:374:18

If you want I could send the files I am trying to process (or a link to them) via email - I don't want to post it here because it is not public.

[hediet]

Yeah, a link to the files would be helpful! (gh@hediet.de)

[LukasSchulz]

I should add, that I don't get the error running the slideo release in a windows-VM, so the problem seems to be somehow MacOS/Linux-related

[hediet]

Urgh... I don't have a Mac system available. I can try on WSL, but I don't have a rust system set up there.
I don't understand where NaN is coming from.

[hediet]

As you stated, I have no problems on windows. I'm positively surprised that slideo can analyze that video!
I guess the last slide causes the problem, as it has a very low similarity with the frame due to being mostly white, while the frame is mostly black.

I guess slideo could be optimized specifically for slides that build up incrementally (like slides that are written on video).

[LukasSchulz]

I have found another OS-specific difference:
On windows I don't get the problem with missed frames I described in a comment before. (I have put line 370 back to its original state on MacOS too) - maybe there is a OS-specific difference in opencv that also caused the NaN-Error?

Also I replaced line 370 with the following code (I don't know how much sense this makes, but it seems to work fine and still finding all the slide timestamps even in the videos I sent you):

rated_best_matches.sort_by(|a, b| {
            if a.2.is_nan() {
                b.2.partial_cmp(&b.2)
                    .expect(&format!("Could not compare {:?} with {:?}", b.2, b.2))
                }
                else {
                    b.2.partial_cmp(&a.2)
                        .expect(&format!("Could not compare {:?} with {:?}", a.2, b.2))
                }
            });