Python cv2, changing camera resolution

I am trying to get a very simple VideoCapture going with OpenCV, where I am able to change the resolution in between pictures.

My setup:

• Debian GNU/Linux 11 (bullseye)
• Python 3.9.2
• OpenCV 4.5.1

Already on the simplest step, OpenCV is giving me warnings I do not understand

import cv2
cap = cv2.VideoCapture(0)

Results in

[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (961) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1

I searched this warning, but I only find links to an OpenCV issue that should already be resolved (see for example here).

Anyway the warning is not prohibitive, as I am able to see images from the camera, and I am able to downsize the camera rresolution from 3264x2448 to 640x480 using cap.set(cv2.CAP_PROP_FRAME_WIDTH, 640) and cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 480).
The true issue occurs when I try to change the frame size back to the original:

print(cap.get(cv2.CAP_PROP_FRAME_WIDTH), cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
# Prints: 3264.0 2448.0

org_w = cap.get(cv2.CAP_PROP_FRAME_WIDTH)
org_h = cap.get(cv2.CAP_PROP_FRAME_HEIGHT)

cap.set(cv2.CAP_PROP_FRAME_WIDTH, 640)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 480)

print(cap.get(cv2.CAP_PROP_FRAME_WIDTH), cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
cap.set(cv2.CAP_PROP_FRAME_WIDTH, org_w)
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, org_h)
print(cap.get(cv2.CAP_PROP_FRAME_WIDTH), cap.get(cv2.CAP_PROP_FRAME_HEIGHT))

This results in a bunch of warnings and the width and height values are set to 0x0 instead:

[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (1824) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Internal data stream error.
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (536) startPipeline OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (1085) setProperty OpenCV | GStreamer warning: no pipeline
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (992) getProperty OpenCV | GStreamer warning: GStreamer: no pipeline
[ WARN:0] global ../modules/videoio/src/cap_gstreamer.cpp (992) getProperty OpenCV | GStreamer warning: GStreamer: no pipeline
0.0 0.0

And then cv2.imshow fails ofc.
Is this not the proper way to change camera resolution between pictures? Should I leave the camera in the original 3264x2448 and just use cv2.resize to reduce the image size after the image is taken?