Absolutely, the main potential issue I see is that generally the AI (or other downstream processing) can't keep up with the full rtsp frame rate unless only a single stream is being processed or its a low rate stream, so many frames need to be "dropped". OpenCV seems to do this internally if the read() method is not called often enough. My reading threads drop the frame if the queue that feeds the AI threads is full and after a short delay (time.sleep(0.001)) to force a context switch, resumes.
You can't easily use rbe on binary buffers or you quickly run out of memory -- the issue that got me on this forum initially that only came to light recently (Memory leak, what am I doing wrong?)
OpenCV also has settings to adjust the size of its internal queue, but it seems version dependent on if its honored or not.