Assessment of PET scanner quantitative factors: Analysis and validation of GATE Monte Carlo simulation with experimental study

Corresponding Author:

Sedighe Sina
Radiation Research Center
Shiraz University
Shiraz
Iran
E-mail: samirasina@shirazu.ac.ir

Abstract

The quality control procedure would also be a requirement for any department to achieve a level of formal accreditation. The QC of the PET images was performed using NEMA and Jaszczak phantoms. This article describes the experimental and simulation quantitative protocols of acquiring PET imaging parameters to validate and analyze the result to use in complex geometry, patient dosimetry, and the optimal value of scanner performance in comparison studies. The QC factors such as signal-to-noise ratio, SNR, contrast to noise ratio, CNR, standardized uptake values, SUV, and spatial resolution were estimated from the hot and cold PET-CT images of the phantoms containing various activities. GATE Monte Carlo code was used for simulation of the phantoms. The images were reconstructed by STIR software, and the QC parameters of the simulation-based images were obtained. The maximum differences between SNR results of the Monte Carlo simulation, and the experimental results were found to be 2.29 and 2.93, and the maximum differences in CNR values were reported to be 2.79 and 2.2 for 20 and 40 MBq/ml activates respectively. The spatial resolution of the images was obtained by the size of spheres and rods of the Jaszczak phantom. According to the results 12.7 and 15.9 mm sphere diameters and 8 mm rod diameters were invisible in 20 and 40 MBq/ml cold scans. The simulation results show that the GATE Monte Carlo code is a beneficial tool for the simulation of PET images. These findings, while preliminary, suggest that analyzing the factors in patient dosimetry with a relation of the pixel intensity value of images with activity variation.

Keywords

Quality control ▪ GATE ▪ PET ▪ NEMA ▪ Jaszczak

Introduction

Positron emission tomography, PET, has brought significant quantitative measurement for in vivo study. In this issue, F18- FDG radionuclide imaging is considered an extensive and routine image procedure in clinical scans which is an applicant in diagnostic imaging, staging, and evaluating treatment response [1]. Concerning the fact that PET imaging contained significant limitations such as low spatial resolution and signal-to-noise (SNR) ratio, quality control parameters play an important role in addressing the issue of scanner performance. Spatial resolution, CNR, SNR, SUV, uniformity critically vital factors in the quality assurance of PET imaging procedure. The measurement of these parameters has been described as quantitative and reproducible values dependent on regular quality tests that analyze the performance of imaging units [2]. The application of phantom studies is fundamental to the assessment of Image Quality (IQ) in PET which is a challenging subject affected by Quality Control (QC) parameters. These preliminary tests are characterized as the scanner performance assessment unit that guarantees and controls patient image quality. The National Electrical Manufacturers Association (NEMA) recommends QC protocols for PET scanners to assess the performance of the systems [3]. In particular, Quality control of PET systems approaches in many different tasks including attenuation correction in PET/MR hybrid imaging, image reconstructions evaluating, patient-based image quality tests [4]. Therefore, quantitative phantom-based tests of PET scanners and analyzing these data is the bold issue. In this way, Monte Carlo simulation is the necessary tool for simulation medical imaging devices and performance scanner assessment in complex or high-dose risk positions. The cause of simulation results offers a better point of view about scanner performance in the optimal conditions, allowing researchers to compare experimental data with simulated tests as the referenced value. Given the importance of this issue, image quantification and quality control factors should be obtained by scanner and image reconstruction procedure [5- 7]. The utilization of a simulation code dedicated to design specific PET systems and a very high level of confidence is also an important process to improve image quantification. The essential problem about Monte Carlo simulation concerns its performances, versatility, and validation. Several studies to investigate the quantitative factors of PET scanners with phantom have been reported. According to the research, one of the main obstacles to improving data analysis is simulating the quality test of a scanner. This paper aims to perform an image quality procedure of a PET scanner in support of simulation results. quantitative factors were estimated by NEMA, and Jaszczak [8,9], phantoms with Monte Carlo code used for simulating the procedures of PET imaging, and the images reconstructed by STIR software. Finally, the result in accordance with the Gate simulation was compared with experimental data, and scanner performance in routine conditions was checked with simulation outputs. Simulation results analyzed in the preliminary step were shown the applicant inpatient imaging and dosimetry studies.

Materials and Methods

A glance at the diagram provided in FIGURE 1, reveals different steps of the study, including experimental, and simulation tests. According to FIGURE 1, we investigate whether and to what extent affects image quality that contains cold and hot spot tests to obtain quantitative parameters and survey the accordance of simulation values.

▪ Description of experimental study

At the first, the uniformity of the PET scan was checked out by the water cylindrical uniform phantom. Then the experimental test was performed in two steps (FIGURE 2) by using PHILIPS PET/CT scanner, PET/CT series Ingenuity TF 64 slices with LYSO crystal type, 18 cm axial FOV, crystal size 4 × 4 × 22 mm³, crystal array per module 23 × 44, detector ring diameter 90.3 cm. To access whether the intensity value is related to activity range, a series of experiments hot and cold study was conducted by NEMA and Jaszczak phantoms.

1. The hot-spot tests by NEMA phantom

2. The cold-spot test by Jaszczak phantom

The four scans of hot-spot tests were performed with NEMA phantom designed using six hollow spheres with different diameters and used for obtaining QC PET factors [10]. The specifications of the phantom are shown in FIGURE 2. Various activities of 18-F Fluoro Deoxy Glucose (FDG) were injected into each sphere and the phantom body was filled with water. The scans were performed in 3 min/ bedtime acquisition for all the activity tests. The characterizations of each scan are summarized in TABLE 1 that shows, the injected activity values in spheres of the same scans were near with ±1.5 uncertainty and vary in 10, 20, 30, 40 MBq/ml.

TABLE 1: The injected activity in each scan (MBq/ml).

The cold-spot scans were implemented by Standard Thereupon Jaszczak (high spatial resolution) phantom with two different activities, 20 MBq/ml and 40 MBq/ml. The Jaszczak phantom is constructed of Acrylic Plexiglas, consisting of six spheres with different diameters and 6 groups of hollow rods with various dimensions. The Jaszczak phantom was used for obtaining cold-spot tests with the 2 different activities. The body of the phantom was filled with water plus FDG solution and the PET/CT scans were applied in 3 min/ bedtime. The injected activities in the phantoms were chosen concerning the common values of injected activities to a patient in the PET/CT scans. Attenuation correction was considered in the scans and images were reconstructed by the OSEM algorithm.

▪ Description of a simulation study

In the second, the PHILIPS PET scanner was simulated (FIGURE 3) with GATE, 8.1, Monte Carlo code. GATE encapsulates the Geant4 libraries [11], that are the useful and accurate Monte Carlo simulation platform for modeling PET systems, and nuclear medicine imaging. Scanner and phantom geometries, physical processes, material definitions are covered by the GATE toolkit, Geant4 physics, and material tables. The specifications of simulated scanner geometry are shown in TABLE 2. After that, the root results of the simulation codes were reconstructed by the reconstruction algorithm. For this purpose, STIR software was used which is an open-source library executed in C++ for Tomographic image reconstruction [12]. GATE to STIR module was implemented for PET image reconstruction. The STIR software template properties are briefly shown in TABLE 3. To better compare simulation study to experimental test, OSEM reconstruction algorithm with 9 iterations, and random and scattered options were considered. The image scaling factor and time frames were set as one and at the end, the projection was saved and the STIR file was created.

TABLE 2: Scanner geometry.

TABLE 3: STIR template properties.

▪ Image quality assessment

In the way of getting results, 3D slicer version 4.8.1 was used for image quantification and analysis. The principal factors in the quantification of PET imaging modality such as SNR, CNR, SUV, Spatial resolution were analyzed by the image processing software. For this purpose, the imported DICOM images of the phantom in each scan and image processing filters were used to segment the best ROI (Region of Interest) or to create the best Volume of Interest (VOI). The six spheres were labeled in each slice and used the segment quantification and PET standard uptake (SUV) modules for estimating the image quality factors. Each factor plays an important role in image quality [13]. The quantitative PET factors and analysis results by NEMA and Jaszczak phantoms are obtained as shown in the following section.

The Signal to Noise Ratio (SNR): The Signal to Noise Ratio (SNR) was calculated as the ratio of Mean value to Standard Deviation (SD) in the VOI (Volume of Interest) [14].

As following the formulas, first, the sphere volumes were specified, then the SNR factors were estimated according to the Mean and SD of the VOI which is the volume of interest around the solitary sphere that was determined with a fixed threshold near 40% of SUV max. The SNR factor depends on the time of the scan, attenuation, and activity. In this study, the injected activity varied in each scan, while the attention and acquisition time was fixed.

The Contrast to Noise Ratio (CNR): The Contrast to Noise Ratio (CNR) was measured as the ratio of signal level in the presence of the noise [15- 16].

We had labeled the sphere and the background area of the phantom before we estimated the CNR factors. Then the table of Mean (I sphere), Mean (background), and SD (background) were estimated and the CNR formula was applied. Also, we compared the contrast of the spheres to each other as formula 5.

5) Contrast ratio between spheres= (Mean sphere( i)-Mean sphere(j))/(SD(background))

The CNR in hot-spot scans were estimated according to equation 4, while this factor was measured in cold-spot scans as follow.

6) CNR (cold) = (Max pixel value in phantom cylinder-Min pixel value in cold sphere)/(Max pixel value in phantom cylinder)

We had segmented the cold spheres and had calculated Min, Max, and Mean pixel values of each label. Finally, we had estimated the CNR factor as equation 6.

Standard Uptake Value (SUV): Standard Uptake Value (SUV) is a quantitative measure calculated from the determination of sphere activity obtained from a PET study. In the patient's body, 18-F Fluorodeoxyglucose (FDG) SUV correlates with the metabolic rate of glucose and the number of tumor cells.

7) SUV= (sphere concentration of radioactive tracer × sphere weight)/( injected dose)

For calculating the SUV of each sphere, first, we had imported the PET images then the VOI of each sphere was labeled. The SUV factor was estimated according to equation 7 and the SUV module was used by 3Dslicer software [15].

Spatial resolution: Spatial resolution defines the ability to distinguish two-point sources as distinct in the reconstructed image. It is typically defined as the Full-Width at Half-Maximum (FWHM) of a Point Spread Function (PSF). We should check the cold and hot-spot scans and also simulated the point source in the sphere [17].

Comparison of the experimental and simulated values: The validation of the simulation study was performed by comparing the results with the experimental data. (see equation 8)

8) Difference percentage of exp and sim values

Results

▪ Experimental results

FIGURE 4 shows two groups of figures concerning the hot and cold-spot scan. The images have shown in the first row provide different hot-spot images obtained from different activities with the NEMA phantom, while the second row showed the cold-spot images by Jaszczak phantom. The different slices of scans were reconstructed by OSEM algorithms with a Sheep-Logan-Hanning filter with the normalized pixel in the range of 0 to 255 value. The values of SNR, CNR, and SUV were calculated with respect to the corresponding formulas for each activity. The first set of questions aimed to survey CNR factor and variation pattern.

▪ Contrast to Noise Ratio (CNR)

The CNR values calculated hot-spot and coldspot scans of phantoms contained different activities shown in FIGURES 5 and 6 respectively. As can be seen from the figures, the minimum CNR of the injected activities in the NEMA phantom (hot-spot test) reported 68,87,110,136 and the maximum CNR was 102,131,151,195. The maximum value of the Contrast to noise ratio was observed as 195 for the highest activity. As it is evident from (FIGURE 5), all six curves have an increasing trend. As expected, the CNR increases by raising the activity. There was a significant correlation between activity and diameter variations. TABLE 4 shows beneficial image quality information about the comparison of sphere contrasts to each other with the same injected activity as consideration for pixel values normalized between 0 to 255. The values of the table were calculated according to the mean differences of two spheres divided by the standard deviation (std) of the background. The minimum value of the table was 0.42 in 10 MBq/ ml activity that shows the contrast difference between spheres 1, and 2 and the maximum contrast value was 250.45 in 40 MBq/ml activity between spheres 1 and 6. This comparison carried out the informative data from patient body associated with activity changes in different size of lesions and could be computed pixel intensity conversion to activity range file that is a crucial step of a patient simulation.

TABLE 4:The contrast ratio between the spheres of the NEMA phantom.

It is apparent from the cold-spot curves, obtained from the Jaszczak phantom, that CNR values have also a rising pattern. When 20 MBq/ml activity was injected into the phantom body, the sphere with the minimum diameter (12.7 mm) could not be distinguished clearly that caused the spatial resolution of the PET scanner. The second sphere, 15.9 mm in diameter, could hardly be distinguished. For 40 MBq/ml injected activity, it is hard to detect the smallest sphere. The minimum CNR values in cold-spot scans with the two activities were reported as 0.48 and 0.56, while the maximum values were found to be 0.93 and 0.99 respectively. The most striking result to emerge from data is cognition of patient image when recognizing smallest lesion is important.

▪ Signal to Noise Ratio (SNR)

FIGURE 7 shown the variations of SNR values.

An increasing trend is observed for the curves by augmenting the sphere diameters and counts. The minimum and maximum SNR values in hot-spot scans were found to be 60.5 and 200.2 with 10 and 40 MBq/ml activities respectively. The mean value of each labeled sphere was fixed and the signals were increased with higher activity and sphere diameters. The SNR estimation using the mean of the signal and standard deviation value of the noises was obtained by equation 3 in each image, and the results indicate more signal to noise.

▪ Standard uptake volume SUV results

The variation of mean, maximum and minimum Standard Uptake Volume (SUV) is shown in FIGURES 8-10. The SUV min, SUV max, SUV mean are the important quantities inpatient study. The factors were calculated according to the Volume of Interest (VOI) spheres containing each activity. An important parameter in the SUV formula is the sphere weight which was obtained as the VOI weight. The volumes of interest were segmented on the CT images before importing the PET DICOM images by 3Dslicer software, and the SUV values were estimated. An overview of experimental data on SUV factor constructs the useful analysis of patient dosimetry by image.

▪ Spatial resolution

Spatial resolution refers to the ability to differentiate small structures. FIGURE 11 indicates the visible sphere diameters in cold and hot-spot scans. In hot-spot scans, the rods of different diameters were filled by FDG in 20 MBq/ml activities. According to the hot-spot images with the activity of 20 MBq/ml, the 8 mm hot rod, and from cold-spot images the spheres with diameters 15.9, and 12.7 mm were invisible. However, the results changed for other activities. For the 40 MBq/ml, 12.7 mm diameter sphere, and the 8 mm hot rod were disappeared. TABLES 5 and 6 compares the actual sizes of the spheres and rods in the cold and hot scans and were shown the accuracy of the illustration of the cold and hot sources.

TABLE 5:Comparison of the sphere diameters dimension in 2 cold scans with actual diameters.

TABLE 6: Comparison of the rod diameters dimension in 2 cold scans with actual diameters.

▪ Monte Carlo simulation results

FIGURE 12 shows the acquired data from AMIDE software. The root files were simulated by Gate toolkit and reconstructed images were performed by STIR reconstruction software which was collected in this figure. The images were reconstructed in 9 iterations and SNR factors were estimated for iterations 9 and 8 (FIGURE 13). The slices of iteration 9 were selected due to the better SNR value. We compared the SNR and CNR values of the simulated NEMA phantom in iteration 9 with the experimental data of 20 and 40 MBq/ml activities. FIGURES 14 and 15 shows the SNR and CNR curves of the simulated study have increasing trends and the values are in close agreement with the experimental data. Interestingly, this correlation is related to considering the simulated parameters near to experimental variable. TABLE 7 shows the percentage difference between CNR and SNR simulated values and experimental data. The maximum SNR difference in 20 and 40 MBq/ml activities was found to be 2.29 and 2.93 respectively and the maximum CNR values were reported as 2.79 and 2.2 for 20 and 40 MBq/ml activities respectively. In the last part of a simulation study, six-point sources were simulated with the size of the NEMA phantom spheres. The reconstructed images of simulation and experimental data were compared with actual source sizes and differences were surveyed. The purpose of the point-source simulation was to investigate the point-speared function effects. FIGURE 16 shows the differences between the shape of the sources and indicates the outputs from impulse inputs. If we now turn to compare the shape of the outputs with point source size consideration, the results are inconsistent with the actual size and experimental shapes. The sizes of each sphere were varied in different tests and the point speared function was changed.

TABLE 7: Percentage Difference between CNR and SNR Simulated and Experimental Values.

Discussion

The quantitative parameters of medical imaging have an important role in performing quality assurance in Positron Emission Tomography. In this study, QC parameters, such as SNR, CNR, spatial resolution, and SUV were assessed experimentally, and the results were compared with the GATE Monte Carlo simulation results. With regard to the importance of preliminary QC test of scanner performance, the research was undertaken in two steps concerning the fact that the simulation study will be supporting complex geometry and patient dosimetry. The image parameters are very complex and depend on conditions. However, evaluation and audit of the performance of the PET unit are vital to ensure the appropriate performance of scanners. NEMA and Jaszczak phantoms are the necessary tools for the quality control of PET scanners. In this study, we focused on the variation of the quality factors of the PET images, with different parameters. This part is obtained both experimentally, and by using the Monte Carlo simulations.

First of all, the hot-spot scans were performed for four different activities injected inside the NEMA phantom. The values of SNR, CNR, and SUV parameters were obtained. The variation of the CNR variation with activity has an increasing trend for all sphere diameters. Considering TABLE 4 which shows the contrast ratio between the spheres, one can see that the maximum contrast value is 250 between spheres 1 and 6. This is caused by the different diameters in 40 MBq/ml activity. The SNR values have also an increasing pattern with raising the activity and the sphere diameters. The SUV factor depends on the injected activity and sphere weight and according to this fact, the curves of the SUV showed a different trend, i.e. decreasing trend with the activity. The segmented VOI in the PET DICOM images were used to estimate the SUV values. Therefore, the segmentation of the VOI is a very important factor affecting the accuracy of the calculated SUV. The results are in accord with data obtained in previous studies. However, we have surveyed meticulously on analyzing parameters.

TABLE 4:The contrast ratio between the spheres of the NEMA phantom.

Another important part of the experimental study was the cold-spot scan with the Jaszczak phantom. The tests were provided with 2 activities. The CNR values have an increasing trend with the activity. The small cold rods and spheres of the Jaszczak phantom are the best tools for assessing spatial resolution. In the 20 MBq/ ml activity scan, we couldn't find spheres 5 and 6. Also, the dimensions of other spheres were seen as lower than their actual sizes. The minimum distinguishable diameter was 19.1 mm. In the 40 MBq/ml activity Jaszczak phantom scan, the values are lower than the actual diameter and the minimum visible diameter is 15.9 mm. The second part of the study was the simulation process. We had chosen the common activity in PET scan but the repetition of the tests carried much dose-related to higher activity. Concerning this issue, the simulation study is a beneficial tool to repeat and validate the tests. First, the reconstructed images were analyzed by SNR factor in each iteration and the best-reconstructed iteration images were chosen. The variation of the QC parameters was found to be the same as the experimental data. In conclusion, we can validate the simulated output and repeat the cold and hot scans with higher activity by a simulation study. In the last part of the study, small spheres were simulated, point sources and the response of the simulation were compared with the simulated volume sources and the experimental outputs. As it is evident from (FIGURE 16), according to the partial volume effect the actual physical size of the sources couldn't be obtained from the images. The simulated findings can be extrapolated to reference values of the best scanner performance that can be tuned to the parameters near the optimal values. It could be argued that quantitative results can help patient dosimetry simulation procedure whether and when the activity range file must be related to pixel intensity in dosimetry steps. It is important to bear in mind the possible bias when we quantize the phantom image and the existence of the source of uncertainty in simulation and experimental determinations. The findings, while in the preliminary step of measurements, have shown reliable protocols and suggestions for patient simulation in image analyzing and dosimetry phases.

Conclusion

The quality control parameters of PET imaging were obtained both experimentally, and by Monte Carlo simulations. The results indicate that the Monte Carlo simulation using GATE can be used effectively in PET simulation, and Quality control of PET images. Simulation of quality control process may be a utile tool in complex physics such as the patient body and can be used as reference data in optimal scanner performance. Such procedures would also be a requirement for any department to achieve a level of formal accreditation.

References

1. Munk OL, Tolbod LP, Hansen SB, Bogsrud T. Point-spread function reconstructed PET images of sub-centimeter lesions are not quantitative. EJNMMI physics. 4(1), 1-2 (2017).
2. Ziegler S, Jakoby BW, Braun H, Paulus DH, Quick HH. NEMA image quality phantom measurements and attenuation correction in integrated PET/MR hybrid imaging. EJNMMI physics. 2(1), 1-4 (2015).
3. N. E. M. Association. Performance measurements of positron emission tomography. NEMA Standards Publication NU 2 (2007).
4. Reynés-Llompart G, Sabaté-Llobera A, Llinares-Tello E, Martí-Climent JM, Gámez-Cenzano C. Image quality evaluation in a modern PET system: impact of new reconstructions methods and a radiomics approach. Scientific reports. 9(1), 1-9 (2019).
5. Ljungberg M. Simulation techniques and phantoms. Emission tomography: The fundamentals of PET and SPECT. Academic Press Inc. (2000).
6. Ljungberg M, Strand SE. A Monte Carlo program for the simulation of scintillation camera characteristics. Comput Methods Programs Biomed. 29(4), 257-272 (1989).
7. Zaidi H. Relevance of accurate Monte Carlo modeling in nuclear medical imaging. Medical physics. 26(4), 574-608 (1999).
8. Groch MW, Erwin WD. Single-photon emission computed tomography in the year 2001: instrumentation and quality control. J Nucl Med Technol. 29(1), 12-8 (2001).
9. Pirayesh IJ, Bahreyni TM, Momennezhad M, Naseri S, Ljungberg M. Simulation of a quality control Jaszczak phantom with SIMIND Monte Carlo and adding the phantom as an accessory to the program. Iran J Med Phys. 9(2), 135-140 (2012).
10. Quality Assurance for PET and PET/CT Systems. Vienna: International Atomic Energy Agency. (2009).
11. Geometry and Tracking (GEANT4) Transport Code.
12. Thielemans K, Tsoumpas C, Mustafovic S et al. STIR: software for tomographic image reconstruction release 2. Phys. Med. Biol. 57(4), 867-883 (2012).
13. Lopez BP, Jordan DW, Kemp BJ et al. PET/CT acceptance testing and quality assurance: Executive summary of AAPM Task Group 126 Report. Med Phys. 48(2), e31-e35 (2021).
14. Yan J, Schaefferkoetter J, Conti M, Townsend D. A method to assess image quality for low-dose PET: analysis of SNR, CNR, bias and image noise. Cancer Imaging. 16(1), 1-2 (2016).
15. Thie JA. Understanding the standardized uptake value, its methods, and implications for usage. J Nucl Med. 45(9):1431-1434 (2004).
16. Toossi MB, Islamian JP, Momennezhad M, Ljungberg M, Naseri SH. SIMIND Monte Carlo simulation of a single photon emission CT. J Med Phys. 35(1), 42-47 (2010).
17. Holstensson M, Hindorf C, Ljungberg M, Partridge M, Flux GD. Optimization of energy-window settings for scatter correction in quantitative 111in imaging: comparison of measurements and Monte Carlo simulations. Cancer Biother Radiopharm. 22(1), 136-142(2007).