qt: better synchronization of video and interface window
This depends on !6805 (merged).
This is not a replacement for !5682, which is about video window -> qt quick window synchronization, such as starting to play a new video. This is rather about qt quick (interface) window -> video window synchronization.
Basically, I delay video window updates until just before the interface window (Qt Quick) swaps the frame which is the time the user is presented with the changes for the interface side (obviously with additional delay for VSync if the frame was rendered too quick). In !6805 (merged), I proposed doing that (updating the video window / synchronizing) during rendering stage (only for position update there, where that merge request is about). In this merge request, I delay it further to the point where UI window is about to swap the frame. This is probably the best it can get until we start dealing with VSync. This should help with ghosting, which is observed when PIP player is moved around or window is resized.
Currently, the synchronization occurs during polishing stage from the GUI (Qt) thread, this is not so optimal because then the video window might get updated before the interface window update is presented to the user. I tried to show this in the SG graph [1]:
This should be safe as long as we can update the video window from Qt's rendering thread. I have tried to check the video integrations (so-called "compositor"s), and it seemed fine to me:
- Direct composition uses
IDCompositionVisual::SetOffsetX()
andIDCompositionVisual::SetOffsetY()
. Direct Composition functions are reportedly thread safe. - Compositor platform uses
QWindow::resize()
andQWindow::setPosition()
for the video window, that should be fine because that is the video window and not the UI window which should not be modified from the GUI thread at the same time. - Wayland uses
wl_subsurface_set_position()
,wl_surface_set_buffer_scale()
wp_viewport_set_destination()
(if fractional), I'm not sure if they are thread safe but these are called for the video window and the video window should not be modified from the GUI thread at the same time. - X11 uses
QWindow::resize()
, the rest seems to be thread safe.resize()
should be fine because video window should not be modified from the GUI thread at the same time.
This approach makes WindowResizer
less relevant:
- Now that size change can be requested from the UI at most once per UI frame rendered, WindowResizer has become less relevant.
- !1978 (merged) (vout: Use a fair display lock and interrupt wait) already reduced the waiting time considerably.
Since the UI uses VSync (except CompositorX11), it should be fine to have blocking stuff that only take couple of milliseconds to execute.
Request review @chub.
[1] https://doc.qt.io/qt-6/qtquick-visualcanvas-scenegraph.html#threaded-render-loop-threaded