Assertion 'pri_ref != PRIMARY_REF_NONE' failed in dav1d_submit_frame() src/decode.c

Reproduced with commit 12337860

Steps to reproduce:

1. replay testcase with ./dav1d_fuzzer testcase.ivf

testcase.ivf

dav1d_fuzzer: src/decode.c:3198: int dav1d_submit_frame(Dav1dContext *const): Assertion `pri_ref != PRIMARY_REF_NONE' failed.

==11343==ERROR: AddressSanitizer: ABRT on unknown address 0x03e800002c4f (pc 0x7fc9ffe91e97 bp 0x7fca0000a7d8 sp 0x7ffe0a2f0e70 T0)
    #0 0x7fc9ffe91e96 in __libc_signal_restore_set /build/glibc-OTsEL5/glibc-2.27/signal/../sysdeps/unix/sysv/linux/nptl-signals.h:80
    #1 0x7fc9ffe91e96 in gsignal /build/glibc-OTsEL5/glibc-2.27/signal/../sysdeps/unix/sysv/linux/raise.c:48
    #2 0x7fc9ffe93800 in abort /build/glibc-OTsEL5/glibc-2.27/stdlib/abort.c:79
    #3 0x7fc9ffe83399 in __assert_fail_base /build/glibc-OTsEL5/glibc-2.27/assert/assert.c:92
    #4 0x7fc9ffe83411 in __assert_fail /build/glibc-OTsEL5/glibc-2.27/assert/assert.c:101
    #5 0x564ac8 in dav1d_submit_frame src/decode.c:3198:13
    #6 0x531fc2 in dav1d_parse_obus src/obu.c:1292:20
    #7 0x52cb30 in dav1d_get_picture src/lib.c:251:20
    #8 0x526630 in LLVMFuzzerTestOneInput tests/libfuzzer/dav1d_fuzzer.c:117:19
    #9 0x526d7f in main tests/libfuzzer/main.c:116:13
    #10 0x7fc9ffe74b96 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:310
    #11 0x41aa19 in _start (dav1d_fuzzer+0x41aa19)