Potential multiplication overflow and incorrect malloc sizes
dav1d makes several calls to malloc
, but it doesn't check for if the malloc size overflows (e.g., the expression sizeof(*f->frame_thread.b) * f->sb128w * f->sb128h * 32 * 32
from decode.c). This means a malicious AV1 file could cause a multiplication overflow and for the malloc
to be too small, leading to all sorts of chaos.
dav1d should be checking for overflow when computing these sizes.