• Users Online: 14150
  • Print this page
  • Email this page


 
 
Table of Contents
LETTER TO EDITOR
Year : 2021  |  Volume : 4  |  Issue : 3  |  Page : 598-599

Authors' reply to Kulkarni


1 Endimension Technology Private Limited, Mumbai, Maharashtra, India
2 Department of Radiodiagnosis, Tata Memorial Hospital, Mumbai, Maharashtra, India

Date of Submission31-Aug-2021
Date of Decision04-Sep-2021
Date of Acceptance06-Sep-2021
Date of Web Publication08-Oct-2021

Correspondence Address:
Vivek Punia
CM 03, KReSIT Building, IIT Bombay, Mumbai - 400 076, Maharashtra
India
Login to access the Email id

Source of Support: None, Conflict of Interest: None


DOI: 10.4103/crst.crst_213_21

Get Permissions


How to cite this article:
Bharadwaj K S, Punia V, Apparao M L, Mahajan A. Authors' reply to Kulkarni. Cancer Res Stat Treat 2021;4:598-9

How to cite this URL:
Bharadwaj K S, Punia V, Apparao M L, Mahajan A. Authors' reply to Kulkarni. Cancer Res Stat Treat [serial online] 2021 [cited 2021 Dec 9];4:598-9. Available from: https://www.crstonline.com/text.asp?2021/4/3/598/327794



We thank Kulkarni[1] for his comments on our article titled, “Novel artificial intelligence (AI) algorithm for automatic detection of COVID-19 abnormalities in computed tomography images.”[2] We will attempt to respond to the queries raised in the letter.

Regarding the question whether a pre-trained model was used, and the request to provide additional details, we have used a pre-trained RetinaNet model which used imagenet data for training and in addition, performed transfer learning for medical images. Due to the unavailability of pre-trained models on COVID-19 image data at the time of our study, we did not use specifically pre-trained models. In our future studies, we will perform the analysis according to insightful suggestions by the reviewer.

With regard to the question about the baseline model, suggestions can be used for comparison for different CNN models. The comparison will also show the effectiveness of the RetinaNet model as compared to other architectures like ResNets, DenseNet, VGG, etc. However, we have used RetinaNet, because it had single-shot detection capabilities and effectiveness on object detection as compared to other methods. Since the objective of our study was to estimate the effectiveness and baseline performance of deep learning in the detection of COVID-19 infection and to play the role of assistive technology,[3],[4] we have not conducted such comparisons. We plan to include such comparisons in our future studies.

Regarding the question about the confusion matrix, we agree with Kulkarni on the need for extensive quantitative evaluation of the results to help medical practitioners in making an informed decision. At the time of our study, we had limited data and extensive analysis was not possible. Since the objective was to measure the quantitative and qualitative effectiveness of deep learning-based AI in a clinical study, we had compared the images with radiologist labels and added a qualitative analysis of the results. Detailed quantitative analysis based on demography, medical history and AI mistakes will be useful for new insights on AI capabilities. However, this would require additional annotations. With the availability of more data and corresponding annotations, we plan to conduct such a study in the future.

Regarding our decision to use RetinaNet rather than other models, at the time of our study (late 2020), our objective was to provide the effectiveness of the method for the purpose of COVID-19 detection and to evaluate the use of AI in the setting of a clinical study. The suggestion from Kulkarni highlights a very crucial point and it can be used to measure the performance of RetinaNet against other models. Since the performance of the model in training and validation was satisfactory, we have treated this model as the baseline and conducted the study. However, we agree with Kulkarni, and in future, we plan to compare the performance of different models to further improve the performance.

Regarding the comment that there may have been bias in the dataset on account of the fact that we collected the normal (non-COVID) images from those of patients of a tertiary cancer care center, we have mentioned process of the COVID-19 negative data collection in the data creation section of the manuscript. We collected all the data of normal and non-COVID-19 patients (other abnormalities) during the period of the study. The collected data were used without any filter on these patients; hence, it demonstrates close to the real-world performance at the Tata Memorial Hospital, Mumbai, India. Regarding the biases due to the demographic profiles of the patients, we had already discussed this in the paper, and we plan to conduct more extensive studies to quantitatively estimate the bias in the AI.

Regarding the bounding box performance measures, we agree with Kulkarni on the qualitative measure of the bounding box. However, we have used the bounding box predictions for qualitative evaluation of the AI. We have added the results of the same to the discussion in our paper. Since this requires an additional annotation effort, we plan to conduct such extensive studies in future to estimate the performance of the bounding box predictions.

Finally, we agree with Kulkarni that cross-validation will provide a more detailed and robust performance of the model. However, in this study, we used fixed training and validation data for evaluation. The results were further qualitatively verified by the radiologist to measure the effectiveness.[5],[6] For future studies, we will include the cross-validation as suggested.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.



 
  References Top

1.
Kulkarni K. Analysis of RetinaNet experiments for COVID-19 detection using CT scan images. Cancer Res Stat Treat 2021;4:596-7.  Back to cited text no. 1
  [Full text]  
2.
Bharadwaj KS, Pawar V, Punia V, Apparao ML, Mahajan A. Novel artificial intelligence algorithm for automatic detection of COVID-19 abnormalities in computed tomography images. Cancer Res Stat Treat 2021;4:256-61.  Back to cited text no. 2
  [Full text]  
3.
Sharma PJ, Mahajan A, Rane S, Bhattacharjee A. Assessment of COVID-19 severity using computed tomography imaging: A systematic review and meta-analysis. Cancer Res Stat Treat 2021;4:78-87   Back to cited text no. 3
    
4.
Bothra M, Mahajan A. Mining artificial intelligence in oncology: Tata Memorial Hospital journey. Cancer Res Stat Treat 2020;3:622-4  Back to cited text no. 4
    
5.
Mahajan A, Vaidya T, Gupta A, Rane S, Gupta S. Artificial intelligence in healthcare in developing nations: The beginning of a transformative journey. Cancer Res Stat Treat 2019;2:182.  Back to cited text no. 5
  [Full text]  
6.
Ahuja A, Mahajan A. Imaging and COVID-19: Preparing the radiologist for the pandemic. Cancer Res Stat Treat 2020;3, Suppl S1:80-5.  Back to cited text no. 6
    




 

Top
 
  Search
 
    Similar in PUBMED
   Search Pubmed for
   Search in Google Scholar for
    Access Statistics
    Email Alert *
    Add to My List *
* Registration required (free)  

 
  In this article
References

 Article Access Statistics
    Viewed86    
    Printed4    
    Emailed0    
    PDF Downloaded6    
    Comments [Add]    

Recommend this journal