Image Classification Integrated with Chatbot

  • Yash Rana
  • Hardik Vyas
  • Sapan Naik
Keywords: Chatbot, image classification, Convolution Neural Network, Recurrent Neural Network, Dialogflow, Inception


To get desired information from Internet is not that easy as so lot of information available. Images are one of the important source of information and search desired image is again a difficult task. To search a desired image from the system, one can use Chatbot as interface between user and system. So, using Chatbot and image as media of communication, one can get required information from Internet. Chatbot is implemented here with image classification. Image classification is implemented to understand the type and details of image, while Chatbot is used for communication purpose. For image classification, convolutional neural network is used. As far as working is concern, when Chatbot receives images, it classifies those images and search information in database based on it and produce full description of image.


[1] Gabrys, Bogdan, and Lina Petrakieva. "Combining labelled and unlabelled data in the design of pattern classification systems q." International Journal of Approximate Reasoning, 35 (2004): 251-273.
[2] Kalra, Kanika, Anil Kumar Goswami, and Rhythm Gupta. "A comparative study of supervised image classification algorithms for satellite images." International Journal of Electrical, Electronics and Data Communication 1.10 (2013): 10-16.
[3] Bhandare, Ashwin, et al. "Applications of Convolutional Neural Networks." International Journal of Computer Science and Information Technologies (2016): 2206-2215.
[4] Yu, Zhou, Ziyu Xu, Alan W. Black, and Alexander Rudnicky. "Chatbot evaluation and database expansion via crowdsourcing." In Proceedings of the chatbot workshop of LREC, vol. 63 (2016) p. 102.
[5] Lee, Jongpil, et al. "Samplecnn: End-to-end deep convolutional neural networks using very small filters for music classification." Applied Sciences 8.1 (2018): 150.
[6] Kalyanpur, Aditya, and J. William Murdock. "Unsupervised entity-relation analysis in IBM watson." In Proceedings of the Third Annual Conference on Advances in Cognitive Systems ACS, (2015): 12.

[7] Behera, Bibek. "Chappie-a semi-automatic intelligent chatbot." (2016): 1-5.
[8] Ren, Shaoqing, Kaiming He, Ross Girshick, and Jian Sun. "Faster r-cnn: Towards real-time object detection with region proposal networks." In Advances in neural information processing systems, (2015): 91-99.
[9] Khvostikov, Alexander, et al. "3D CNN-based classification using sMRI and MD-DTI images for Alzheimer disease studies." arXiv preprint arXiv:1801.05968 (2018).
[10] Zhang, Qi-xing, et al. "Wildland Forest Fire Smoke Detection Based on Faster R-CNN using Synthetic Smoke Images." Procedia engineering 211 (2018): 441-446.
[11] Chollet, Francois. Deep learning with python. Manning Publications Co., 2017.

[12] Ciresan, Dan C., Ueli Meier, Jonathan Masci, Luca Maria Gambardella, and Jürgen Schmidhuber. "Flexible, high performance convolutional neural networks for image classification." In IJCAI Proceedings-International Joint Conference on Artificial Intelligence, vol. 22, no. 1, (2011):1237.

[13] Szegedy, Christian, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. "Rethinking the inception architecture for computer vision." In Proceedings of the IEEE conference on computer vision and pattern recognition, (2016): 2818-2826.

[14] Abadi, Martín, Michael Isard, and Derek G. Murray. "A computational model for TensorFlow: an introduction." In Proceedings of the 1st ACM SIGPLAN International Workshop on Machine Learning and Programming Languages, ACM, (2017):1-7.

[15] Sharif Razavian, Ali, Hossein Azizpour, Josephine Sullivan, and Stefan Carlsson. "CNN features off-the-shelf: an astounding baseline for recognition." In Proceedings of the IEEE conference on computer vision and pattern recognition workshops, (2014): 806-813.

[16] Girshick, Ross. "Fast r-cnn." In Proceedings of the IEEE international conference on computer vision, (2015): 1440-1448.
[17] Shawar, Bayan Abu, and Eric Atwell. A comparison between Alice and Elizabeth chatbot systems. University of Leeds, School of Computing research report 2002.19, 2002.
[18] Heller, Bob, Mike Proctor, Dean Mah, Lisa Jewell, and Bill Cheung. "Freudbot: An investigation of chatbot technology in distance education." In EdMedia: World Conference on Educational Media and Technology, Association for the Advancement of Computing in Education (AACE), (2015): 3913-3918.
[19] Shawar, Bayan Abu, and Eric Atwell. "Using dialogue corpora to train a chatbot." In Proceedings of the Corpus Linguistics 2003 conference, (2003): 681-690.
[22] Tang, JingLei, Dong Wang, ZhiGuang Zhang, LiJun He, Jing Xin, and Yang Xu, "Weed identification based on K-means feature learning combined with convolutional neural network," Computers and electronics in agriculture 135, (2017): 63-70.
[23] Nielsen, Michael A. "Neural networks and deep learning (2015)," [Online]. Available: http://neuralnetworksanddeeplearning. com (2016).