I understand your concern.
I think you use ROI which includes anterior portion of hips and lateral portion of hips to the midcoronal plane and not much moving skin of abdomen (to eliminate the errors caused by breathing motion and daily fat displacement), which is demonstrated in the pocketbook called “Reference guide, drawing an isocenter region of interest, VisionRT”. With that ROI I have done isocenter comparison between AlignRT- and tattoo setup in pelvic RT (n=15+15). AlignRT improved isocenter accuracy in LAT direction in both systematic and random errors (p=0.02), but not in LNG direction(p=0.98). Vertical direction was based on mean couch value, taken from first three day image matches, leading to small isocenter variation (random error 0.17-0.20cm). I do not have own data of rotations between CBCT and AlignRT. That kind of fresh data could be very useful and interesting if someone acquires daily CBCTs and perform 6D corrections based on automatch in daily practice with the ROI VisionRT suggests.
(Comparison of surface matching and target matching for image‐guided pelvic radiation therapy for both supine and prone patient positions, Zhao et al., 2016.).
Are the errors systematical? If the are, can you use such VRT surface in the next day setup, which includes corrected isocenter errors and corrected residual structure errors based on IGRT. This may need retrospective analysis of the images, so that you see that the VRT surface includes such corrections that have been typical with that patient earlier days. If the named large errors are not systematical, it is a bit risky to use AlignRT for setup, if you cannot correct the errors in 6D. This is possible with the obese patients, but on the other hand this is possible also with the tattoo setup (Effect of Body Mass Index on Magnitude of Setup Errors in Patients Treated With Adjuvant Radiotherapy for Endometrial Cancer With Daily Image Guidance, Lin et al 2012.). Therefore also such study comparison could be interesting where we compare the residual structure errors in the CBCT with setup based on 1) AlignRT 2) tattoos 3) absolute couch values with indexed fixation without AlignRT or tattoos, just with checking the approximate straightness before IGRT (Eliminating Daily Shifts, Tattoos, and Skin Marks: Streamlining Isocenter Localization With Treatment Plan Embedded Couch Values for External Beam Radiation Therapy, 2019).
We use AlignRT in pelvis for both setup and intrafactional monitoring. The main focus is in intrafractional control and we acquire new VRT surface daily after image guidance. (Three-dimensional surface imaging for detection of intra-fraction setup variations during radiotherapy of pelvic tumors, Apicella et al. 2016).
After all this, I would not look behind, instead try to improve the existing, one patient at the time.