Robot-Based Image Analysis for Evaluating Rehabilitation after Brain Surgery

  • Jan Kohout
  • Jan Crha
  • Katerina Trnkova
  • Karel Sticha
  • Jan Mares
  • Martin Chovanec
Keywords: brain surgery, computer vision, gait disorders, facial disorders, Kinect


After certain types of brain surgery, patients are often affected by changes in both their dynamic balance and facial disorder. Because rehabilitation takes several months, it is important that both doctors and patients are able to monitor progress quantitatively. At present, such quantification is subjective and highly dependent on the doctor’s opinion. Thus, we here investigate the use of robot-based image analysis for measuring rehabilitation. To evaluate a patient’s dynamic balance, we developed a mobile robotic platform that uses a stereovision camera (MS Kinect) to capture a video of the subject walking along a hospital corridor. To evaluate a patient’s facial disorders, the same camera is used in a static mode to detect and capture precise facial movements that the subject is asked to perform. From these videos, specific patterns can be extracted for rehabilitation process description.


Subaihi, A., Almanqur, L., Muhamadali, H., AlMasoud, N., Ellis, D.I., Trivedi, D.K., Hollywood, K.A., Xu, Y., Goodacre, R.: Rapid, accurate, and quantitative detection of propranolol in multiple human biofluids via surface-enhanced raman scattering. Analytical chemistry 88(22) (2016) 10884–10892

Lin, D., Ge, X., Lin, X., Chen, G., Chen, R.: Blood surface-enhanced raman spectroscopy based on ag and au nanoparticles for nasopharyngeal cancer detection. Laser Physics 26(5) (2016) 055601

Wang, J., Lin, D., Lin, J., Yu, Y., Huang, Z., Chen, Y., Lin, J., Feng, S., Li, B., Liu, N., et al.: Label-free detection of serum proteins using surface-enhanced raman spectroscopy for colorectal cancer screening. Journal of biomedical optics 19(8) (2014) 087003

Wang, J., Zeng, Y., Lin, J., Lin, L., Wang, X., Chen, G., Huang, Z., Li, B., Zeng, H., Chen, R.: Sers spectroscopy and multivariate analysis of globulin in human blood. Laser Physics 24(6) (2014) 065602

Rekha, P., Aruna, P., Daniel, A., Prasanna, S.W., UdayaKumar, K., Ganesan, S., Bharanidharan, G., David, B.: Raman spectroscopic characterization of blood plasma of oral cancer. In: Photonics (ICP), 2013 IEEE 4th International Conference on, IEEE (2013) 135–137

Feng, S., Chen, R., Lin, J., Pan, J., Wu, Y., Li, Y., Chen, J., Zeng, H.: Gastric cancer detection based on blood plasma surface-enhanced raman spectroscopy excited by polarized laser light. Biosensors and Bioelectronics 26(7) (2011) 3167–3174

Jermyn, M., Desroches, J., Mercier, J., Tremblay, M.A., St-Arnaud, K., Guiot, M.C., Petrecca, K., Leblond, F.: Neural networks improve brain cancer detection with raman spectroscopy in the presence of operating room light artifacts. Journal of biomedical optics 21(9) (2016) 094002

Liu, T., Chen, C., Shi, X., Liu, C.: Evaluation of raman spectra of human brain tumor tissue using the learning vector quantization neural network. Laser Physics 26(5) (2016) 055606

Jermyn, M., Desroches, J., Mercier, J., St-Arnaud, K., Guiot, M.C., Petrecca, K., Leblond, F.: Neural networks improve brain cancer detection with raman spectroscopy in the presence of light artifacts. In: Clinical and Translational Neurophotonics; Neural Imaging and Sensing; and Optogenetics and Optical Manipulation. Volume 9690., International Society for Optics and Photonics (2016) 96900B

Yang, T., Li, X., Yu, T., Sun, R., Li, S.: Spectral discrimination of serum from liver cancer and liver cirrhosis using raman spectroscopy. In: European Conference on Biomedical Optics, Optical Society of America (2011) 808720

Harris, A.T., Rennie, A., Waqar-Uddin, H., Wheatley, S.R., Ghosh, S.K., Martin-Hirsch, D.P., Fisher, S.E., High, A.S., Kirkham, J., Upile, T.: Raman spectroscopy in head and neck cancer. Head & neck oncology 2(1) (2010) 26

Machida, E., Cao, M., Murao, T., Hashimoto, H.: Human motion tracking of mobile robot with kinect 3d sensor. In: SICE Annual Conference (SICE), 2012 Proceedings of, IEEE (2012) 2207–2211

Stone, E.E., Skubic, M.: Unobtrusive, continuous, in-home gait measurement using the microsoft kinect. IEEE Transactions on Biomedical Engineering 60(10) (2013) 2925–2932

Boyd, J.E., Godbout, A., Thornton, C.: In situ motion capture of speed skating: Escaping the treadmill. In: Computer and Robot Vision (CRV), 2012 Ninth Conference on, IEEE (2012) 460–467

Gonzalez, R.C., Woods, R.E., Eddins, S.L.: Digital Image Processing Using MATLAB○R . Gatesmark Publishing (2009)

Ringer, M., Lasenby, J.: Modelling and tracking articulated motion from multiple camera views. In: BMVC. Volume 2000. (2000) 172–181

Boulic, R., Fua, P., Herda, L., Silaghi, M., Monzani, J.S., Nedel, L., Thalmann, D.: An anatomic human body for motion capture. In: Proc. EMMSEC. Volume 98. (1998)

Walther, L.E.: Current diagnostic procedures for diagnosing vertigo and dizziness. GMS current topics in otorhinolaryngology, head and neck surgery 16 (2017)

Sagers, J.E., Brown, A.S., Vasilijic, S., Lewis, R., Sahin, M.I., Landegger, L.D., Perlis, R.H., Kohane, I.S., Welling, D.B., Patel, C.J., et al.: Computational repositioning and preclinical validation of mifepristone for human vestibular schwannoma. Scientific reports 8(1) (2018) 5437

Nam, G.S., Jung, C.M., Kim, J.H., Son, E.J.: Relationship of vertigo and postural instability in patients with vestibular schwannoma. life 8 (2018) 9

Rosahl, S., Bohr, C., Lell, M., Hamm, K., Iro, H.: Diagnostics and therapy of vestibular schwannomas–an interdisciplinary challenge. GMS current topics in otorhinolaryngology, head and neck surgery 16 (2017)

Becker, W., Naumann, H.H., Pfaltz, C.: Ear, nose, and throat diseases. Stuttgart: Thieme, ISBN 978-3-13-671203-0 (2009)

Alicandri-Ciufelli, M., Piccinini, A., Grammatica, A., Salafia, F., Ciancimino, C., Cunsolo, E., Pingani, L., Rigatelli, M., Genovese, E., Monzani, D., et al.: A step backward: The rough facial nerve grading system. Journal of Cranio-Maxillo-Facial Surgery 41(7) (2013) e175–e179

Sun, J., Wang, Y., Li, J., Wan, W., Cheng, D., Zhang, H.: View-invariant gait recognition based on kinect skeleton feature. Multimedia Tools and Applications (2018) 1–27

Li, Q., Wang, Y., Sharf, A., Cao, Y., Tu, C., Chen, B., Yu, S.: Classification of gait anomalies from kinect. The Visual Computer 34(2) (2018) 229–241

How to Cite
Kohout, J., Crha, J., Trnkova, K., Sticha, K., Mares, J. and Chovanec, M. 2018. Robot-Based Image Analysis for Evaluating Rehabilitation after Brain Surgery. MENDEL. 24, 1 (Jun. 2018), 159-164. DOI:
Research articles