Evaluation of Auto ROI Automated Region of Interest Generation Tool in AlignRT: Preliminary Findings from a Testing Site

Tanith Lott (00:04):

Hi everyone. So we’re delighted to be here and we’re going to be presenting, as we said on the evaluation of the Auto ROI, Automated Region of Interest Generation tool in AlignRT. And we’ll be sharing with you our preliminary findings from a testing site. So this was very much a collaborative project between our project team at home in the Beacon and Vision RT. So, before I dive into the auto ROI tool itself and the evaluation of it, I might just provide a bit of background on the Align RT and how we use it at the Beacon Hospital. So in 2014, we introduced SGRT for our cranial SRS sites. This was then expanded in 2017 to include the vast majority of our sites, extremities, thorax, abdomen, and pelvis. Then, in 2018 2019, we began our tattooless journey in conjunction with breast DRBH. This was then expanded further in 2019, and by 2020, we had gone completely tattoo-less and used SGRT for all our treatment sites.

Tanith Lott (01:07):

Our most recent updates were in 2021 to AlignRT. So as you can see, we use AlignRT quite comp comprehensively across treatment sites. So this provides quite a good testing bed for this Auto ROI within our department. Just to note that we use a Varian Edge and, most recently TrueBeam with SGRT AlignRT on both machines. So if we get stuck in the OIS in SGRT, we all know how crucial they are for patient tracking setup and subsequent treatment delivery. They need to be an accurate surrogate for the target. They need to contain enough topographic landmarks. If they’re too large, there’s the low frame rate, and they’re insensitive to local changes, whereas if they’re too small, they’ll give you insufficient topography and potential camera obstruction. There have been extensive studies showing that the selection of the ROI can directly affect patient setup and, therefore OAR and target dose.

Tanith Lott (02:06):

Even small changes in the definition of the actual ROI can lead to a decrease in setup error. So why might, why might we then move to motivate to automate the ROI definition? So it’s time-saving. We all know that the creation of manual ROIs is resource-intensive. It will increase consistency between patients. Currently, the manual process is quite user-dependent and can depend on the varying level of experience between TE therapists within a department, and it’s also better utilises resources and can lead to more streamlined workflows within departments. So, what does this auto ROI module designed to do? So it can define the ROI for most sites, pelvis, breast, brain, chest, and abdomen. It’s based on deep feedback from multiple clinical sites and adaptive-based modelling on successful ROIs. So it’s not AI. So then, a little bit about our actual study design. So we structured this in two parts. It was done on the prototype system. So the first part took place between November 2024 and January 25 and it was for pelvic breast, and thorax sites. And then in two the 2025, June, July of this year, we extended that to look at our SRS patient cohort.

Tanith Lott (03:36):

So this is just a video of what it looks like in clinical practice. So this interface should be very similar. So if you select your protocol of where you’re going to be treating your anatomical site, that will then import your surface for you. You’ll be given an option in two seconds to generate an Auto ROI. And under that there will be a dropdown menu for the different variants available for that site. So again, if we scroll through them, this is for a left breast; you can see that it is the inverted T that some centers might use. That’s just a general breast chest band. There’s a chest wall excluding breasts, which you might need, need for more pendulum breast treatments, your contralateral strip, depending on your department protocols, which or OY you use. And lastly, for the superclub treatment extended superiorly up into the neck area. So once you’ve picked the most clinically appropriate for what you’re trying to achieve in your departmental protocol, you can then save that. And that is the process. So as you’ve seen, once you pick your protocol so your anatomical site, there’s a dropdown menu of the different auto ROIs that are available for that site. So I’m not going to go through them all. You’ve seen in the video the breast DIBH one. So there’s one for the abdomen, pelvis and brain. And you can just see the different variants of what is available to use within your departments.

Tanith Lott (05:07):

So here are just a few examples of how the auto IROI performs. So this is done from the clinical version of the program, and this is for breast. So this is what we would use in our department. So, it’s the breast with a contralateral strip. If I show you, sorry, if I show an example, then for the pelvis, you can see this is again what we would use in our departments, focusing on stable bony landmarks within the pelvis. And that’s generated by the Auto ROI. Then, if we move on to the thorax, again, this is very similar to what we would do, and it focuses on the rigid structures in the thorax, up into the sternum, the rib cage, excluding any abdominal topography to exclude respiratory motion. And then if we look lastly at our brain, this is our SRS treatment sites. So we scan them with an open face mask. So, you can see the Auto ROI does a good job at contouring around the mask and around the kind of the nasal area up to the cheek level. So I’m going to pass you over to my colleague Kiron, or Rob, sorry, Kiran, sitting in front of me, who is going to go through the evaluation process and all the data.

Robert Nolan (06:23):

So as Tanith mentioned there, I’m going to go into more detail on our prototype testing, which we did at the end of last year and just the start of this year. So just to note for this first part, we did this study using a prototype tool of auto ROI on our own patients. So this is just a big spreadsheet showing kind of all the data we recorded. So we had kind of a mixed methods approach to evaluating auto or wise. So we had just a simple qualitative metric where we used this evaluation table on a scale of one to five, from unusable all the way up to perfect. And then in terms of quantitative data we also asked our Ortiz during treatment to flick between their manually generated ROI and the auto-generated ROI, and just to record the deltas for each.

Robert Nolan (07:17):

So we could do a, a quick comparison of both just to get some sort of quantitative comparison for this. But for this kind of part of the prototype testing, we mainly just wanted to identify any gross differences between what we drew manually and what the Auto ROI is giving us. So in terms of the study co cohort for this part of the testing we had a mix between mostly breast and pelvis. And in terms of the distributions of the ratings from one to five as we’d expected for a prototype tool, we have mostly three out of five. So, on the scale that just means minor edits required. So we have very few at the two range, which would be significant editing required. And very few kinds of further up. Maybe there are a few fours there, usable as is, but none.

Robert Nolan (08:08):

Perfect. So what should one expect from a prototype tool? Some small edits need to be made for these or wise, but very promising and considering this is quite early on at the end of last year, this is pretty impressive for us. And in terms of the quantitative data, so this is a comparison between the deltas when we’re using our manual and then we just change our ROI to the unedited Auto ROI and to see if there’s much of a difference in the deltas. In terms of the translational deltas, there’s very small differences in the very Latin long and you can see you kind of a breakdown here between pelvis and breast. So quite similar there for Latin long and then some kind of bigger differences in terms of rotation and especially role and pitch. But we think when there’s any sort of difference between your manual roi and Auto ROI, the deltas that would be most sensitive to any differences would be these rotational deltas.

Robert Nolan (09:10):

So as ETH mentioned this, there was a second part to this prototype testing. And this is actually done not on our patients, but on some ROI’s generated on anonymized patients is patients sent to us by Vision RT. So we did a study of 87 cranial SRS patients. So as Thomas mentioned, our center is a long history of using SGRT for cranial SRS patients. So since 2014, this was quite an important part of Auto ROI that we would’ve liked to implement clinically. So we had a different scale here from one to four, and then we have these descriptor boxes, which we can use to describe how we would improve these ROIs if we were using them clinically. And we just have a simple front on and two side views of each ROI.

Robert Nolan (10:00):

So again, the distribution of the ratings here. So quite similar to what we saw in the other type prototype version for the standard sites. We have kind of very few ROI’s that we would deem as being a one or a two, so requiring significant editing or unusable with the vast majority following in that three out of four range with an average of 2.91 for the cohort of 87 patients. So we are pretty happy with the results there. So in terms of the descriptors, how would we improve the ROIs that we were given for this prototype version? So the vast majority of edits that would’ve been required would’ve been insufficient coverage or topography. So basically, increasing the extent of the ROI is an issue that was present in this prototype, which we, we haven’t seen in the clinical version for cranial sites is a number of holes kind of presenting across the OOI.

Robert Nolan (10:57):

Again, we haven’t seen that in the clinical version, so that’s something that’s been fixed in the prototype. And then the kind of third most mentioned for these ROIs was the edges requiring editing. So possibly some rough edges that need to be cleaned out, but very few indications of any of the other descriptors. So just to give an example of these sorts of minor edits that might have been required for these ROIs for the cranial sites. So you can see in the far left, there we have at the superior extent of the ROI, we have some of the masks included which we don’t do clinically. Also, just an example there of the small holes that we saw in the ROI in the prototype version. So again, a very quick fixed just something we’d rather not have there.

Robert Nolan (11:43):

And then our third example here, which is you, you might see a few times is slightly too much of the cheek concluded that we wouldn’t do it clinically again. So, some early observations. So, from our center’s experience testing ROI in the prototype version and some kind of preliminary work with the clinical version the prototype testing shows that the vast majority of ROI is only require minor edits. So we get very few cases where we have an auto OOI that we have to do significant editing to that would take similar time to essentially using a manual ROI. So very promising. And the performance of OT ROI is is good and it’s consistent across all sites, which is quite reassuring. So, depending on whether it’s pelvis, breast or cranial sites, we saw very similar performance.

Robert Nolan (12:35):

And then yes, in, in terms of the cranial. So, insufficient coverage and topography holes in the oi are the most common reasons for any sort of edits. But again, again, that’s the prototype version. So just moving on to our next steps. We’ll have to do more testing on the clinical version to see if this performs differently from our experience with the prototype version. So actually, in the next few weeks our center’s planning on upgrading to AlignRT version eight. So we’ll be doing more testing on the clinical version of auto ROI, and we hope to implement it clinically for all sites very soon. So just some, some acknowledgements. It wasn’t just me and Tanith Lott working on the project. We also had a radiation therapist, Smith and Claire Day, and physicist Luke Rock. And we also had a lot of help from Vision rt, especially Connie and Andy. New and yeah, thanks for listening. And do you have any questions? Thank you.