Journal of Health Care Communications Open Access

Survey-Based Evaluation of an Innovative Speech-Enabled Translator in Emergency Settings: A Phase II Cohort Study

Antony Akash Janakiram^1*, Johanna Gerlach², Pierrette Bouillon² and Hervé Spechbach^{1 1}Department of Community Medicine, Geneva University Hospitals, Switzerland
²Department of Translation and Interpreting, University of Geneva, Switzerland
^*Correspondence: Antony Akash Janakiram, Department of Community Medicine, Geneva University Hospitals, Switzerland, Email:

Received: 01-Nov-2022, Manuscript No. IPJHCC-22-14676; Editor assigned: 03-Nov-2022, Pre QC No. IPJHCC-22-14676 (PQ); Reviewed: 17-Nov-2022, QC No. IPJHCC-22-14676; Revised: 22-Nov-2022, Manuscript No. IPJHCC-22-14676 (R); Published: 29-Nov-2022, DOI: 10.36846/2472-1654-7.70048

Abbrevation

ED: Emergency Department

INTRODUCTION

Migration is a worldwide phenomenon. The United Nations estimated that 272 million individuals migrated internationally in 2019 for different reasons (work, family, study, conflict, persecution or disaster) [1]. In the USA, over 300 languages are spoken and more than 25 million residents have limited English proficiency [2]. Healthcare services are confronted today with a linguistic and cultural diversity. Data from a multilingual country such as Switzerland show that almost 17.2% of the population is most familiar with a language that is not one of the four national languages [3]. At Geneva University Hospitals, more than 50 different languages are spoken by patients and at least one out of 12 is unable to express themselves in the local language [4].

For patients with a migration background, the emergency department (ED) is often the first point of entry in the healthcare system [5]. A clinical review focusing on ED settings reported that patients with limited English proficiency are less satisfied with medical encounters, have more para-clinical examinations to achieve a diagnosis, and receive less explanations and follow- up [6]. Several studies show that linguistic disparities lead to negative outcomes in emergency contexts [7-12]. However, multiple solutions exist to overcome these language disparities. Patients often rely on ad hoc interpreters (e.g. acquaintances or family members), which can lead to miscommunication or information omissions [13]. Another option is to use professionally trained interpreters, considered as the gold standard for medical translation [14]. However, such services are often underused, particularly in an emergency context [15-21]. In a recent review, it was suggested that barriers to the use of professional interpreters were due to the reliance of physicians on untrained ad hoc interpreters, perceived time and labor associated with obtaining and working with a professional interpreter, and costs associated with these services [6].

Another approach is to use computer-based tools, which are now part of daily life, as the widespread prevalence of telemedicine and telehealth has led to an increasing acceptance and positive attitude towards technology and its benefits in healthcare [22-28]. Several well-known translation tools exist, such as Google Translate (https://translate.google.com/) or DeepL (https://www.deepl.com/translator), and allow the user to translate automatically text or speech from one language to another. However, these non-specialized tools are not adapted to a medical setting and do not include access to a full medical repository [29-31]. Additionally, data protection is questionable. Another type of system allows the user to translate a set of fixed phrases pre-translated into different languages. Indeed, it has been reported that patients have a preference for a domain-specific system with fixed questions [32]. These systems are categorized as “phraselators” and developed for a specific purpose and context. In the medical field, tools such as MediBabble (http://www.medibabble.com/) or Universal Doctor (https://www.universaldoctor.com/) are designed for medical consultations and show benefits during diagnostic interviews with allophone patients [5].

To overcome the problem of the language barrier, Geneva University Hospitals, in collaboration with the University of Geneva Faculty of Translation and Interpreting, have developed a new type of phraselator named “Babel Dr.”. Unlike others, Babel Dr. incorporates speech recognition technology, thus allowing healthcare staff to communicate more naturally with patients. In this study, we describe an evaluation of the three main features (oral interaction, speech recognition system and selection of sentences in a list) of Babel Dr. to assess perceived satisfaction, efficacy and usefulness among patients and physicians in a real life clinical setting. We aimed to answer the following questions to determine whether the use of a speech-enabled fixed-phrase translation system is adapted or not for medical care in an emergency context: 1) are patients and physicians satisfied with this type of fixed-phrase translator?; 2) can Babel Dr. be considered as an efficient tool in an emergency context and are physicians able to reach a diagnosis with the tool?; and 3) are the three technical features appreciated by physicians?

Materials and Methods

Population and Setting

We conducted a phase II cohort study using a survey-based design to evaluate the perceived satisfaction and efficacy of a speech-enabled, fixed-phrase translation tool (“Babel Dr.”) among patients and physicians in an emergency department. All patients were recruited during office hours between 29 November 2018 and 31 January 2020 at the outpatient emergency unit of Geneva University Hospitals. Patients and physicians were selected on a voluntary basis. The inclusion criterion was patients with no understanding of French, thus making a diagnostic interview by the physicians impossible. Exclusion criteria were defined as patients who shared a common language with the physicians and/or were not able to read in their native language.

Tool Presentation

Babel Dr. understands approximately 10,000 pre-translated sentences made from rules (patterns with variables) and can be mapped to one billion possible variations. The tool includes one input language (French) and five output languages (Albanian, Arabic, Farsi, Spanish, and Tigrinya). It also translates to sign language for French-speaking Switzerland, a world first, but coverage development is still in progress. Each sentence is translated for male and female patients and different translation formats are presented (written, spoken, and video for sign language). Babel Dr. includes a speech recognition system allowing the user to speak through the tool. This system is also coupled with a back-translation feature. Once the spoken sentence is recognized, the tool associates it with one of its core sentences. A written feedback is then proposed to the physician who can approve the sentence before translating it to the patient. This pairing guarantees the reliability about what is translated to the patient and participates in the performance of the translation process. Additionally, sentences can be translated directly from an ordered list, which is scrolled automatically to present the most pertinent sentences, based on the dialogue context derived from the previous sentences translated for the patient. Babel Dr. is available at all times at our hospital, with data protection guaranteed on local servers.

Intervention

The intervention consisted of the use of the translation tool during a standard emergency consultation. Prior to participation, non-French speaking patients were asked to read an information leaflet about the study in their own language and give informed written consent for their participation (Appendix 1). All volunteer physicians followed a short 15-30 minutes training on Babel Dr. and were required to use the tool for the diagnostic interview. The tool includes introductory sentences to explain to the patient how to interact by using head tilting movements for “yes” and “no” or gestures. These indications were provided by the physicians at the beginning of the consultation. During the consultation, physicians could translate questions related to the patient’s medical history using either the speech recognition system or by directly selecting the sentences in the list sorted by the domain of interest. Either party could interrupt the use of the translation tool at any moment. At the end of each session, both patients and physicians completed a satisfaction survey. Physicians received a French survey and patients received a translated version in their own language. If requested, help was provided to fill out the survey without any influence on the answers.

The patient survey included 11 questions covering demographic data (5 items) and two dimensions: communication aspects (5 items) and satisfaction (1 item) (Appendix 2). The physicians’ survey included 21 questions covering patient care information (3 items), perceived efficacy of the tool (5 items), past experience with translation tools (6 items), tool interruption (3 items), technical features’ appreciation (3 items), and satisfaction (1 item) (Appendix 3). Responses to survey items were either an open-ended field, a “yes/no” or multiple choice, and two used a 4-point Likert scale (“not at all”, “rather no”, “rather yes” and “completely”; and “rather not satisfied”, rather satisfied” and “completely satisfied”). These data contributed to a qualitative answer to the research questions mentioned above.

Outcomes

The main outcomes were the evaluation of patient and physician satisfaction and perceived efficacy of the technical features (oral interaction, speech recognition system, and selection of sentences in a list) of the tool measured through two separate surveys.

Statistical Analyses

Qualitative analyses were conducted. Analyses were done on IBM SPSS Statistics Version 26 (IBM SPSS Statistics, RRID: SCR_019096). Logistic regressions were performed on the physicians’ survey to evaluate the association between the technical features’ appreciation items and satisfaction. Due to the low number of physicians recruited, only univariate logistic regressions were performed.

Results

Of 42 eligible patients who were willing to participate during the study period, 30 were recruited (mean age, 38.2 [standard deviation, 16.49] years; 53.33% were male). Language distribution was as follows: Albanian (1); Arabic (4); Farsi (7); Spanish (11); and Tigrinya (7) (Figure 1). Patient diagnosis varied and range from digestive issues (gastritis, colitis), respiratory problems (bronchitis), cutaneous infection (eczema, phlebitis), genital complications (epidimyte, orchitis), urinary infection and articular problems. Log data showed that physicians translated a total of 948 (mean, 31.6) sentences with the tool: 560 (mean, 18.7) using the speech recognition system, and 388 (mean, 12.9) using the list of sentences in the tool.

Figure 1: Overview of the language distribution of the patient study population

Ninety percent (n=27) of patients expressed a positive level of satisfaction (item 11) (Figure 2). Furthermore, 90% (n=27) felt able to tell the physicians why they came to the emergency room (item 6) and 93% (n=28) expressed that they understood the tool’s translations (item 8). At the physician level, 86.7% (n=26) were satisfied with the tool (Figure 3). Regarding the perceived efficacy, 93.3% (n=28) of physicians reported understanding the patient’s health problem (item 4) and 80% (n=24) were able to make a diagnosis (item 7). Additionally, 60% (n=18) were able to provide a medical care plan for the patient (item 8). Twenty percent (n=6) of cases required assistance from a telephone-based interpreter (item 16). Of note, 53.3% (n=16) of patients would have preferred to use an interpreter during the consultation, 40% would not have preferred to use an interpreter, and 6.7% (n=2) did not answer (item 10). Finally, only 33% (n=10) of physicians were able to ask all their questions (item 5). The evaluation of the technical features of the system showed that 86.7% (n=26) of physicians appreciated the possibility to interact orally with the system (item 18), 93.3% (n=27) considered that the questions were well understood by the tool (item 19), and 93.3% (n=27) appreciated translating sentences directly from the list (item 20). Lastly, 30% (n=9) of physicians had to stop the use of Babel Dr. during the consultation due to a technical problem, having difficulty in using the tool, or at the patient’s request (item 15).

Figure 2: Distribution of patients (n=30) for the question: “How satisfied are you with this translation tool?”

Figure 3: Distribution of doctors (n=30) for the question: “Concerning the use of this software, would you say that you are...?”

The association between the physicians appreciation to be able to interact orally with the system (item 18) and their satisfaction with the tool (item 21) was statistically significant (P=0.005; odds ratio 75 [95% confidence interval 3.7-1536]; (Table 1). Logistic regression showed a significant relationship between the feeling that the spoken questions were well understood by the tool (item 19) and physicians’ satisfaction (item 21) (P=0.024; odds ratio 25 [95% confidence interval 1.5- 410.9]; (Table 2). Finally, there was a significant association between the physicians’ appreciation to be able to translate sentences directly from the list (item 20) and their satisfaction with the tool (item 21) (P=0.024; odds ratio 25 [95% confidence interval 1.5-410.9]; (Table 3).

Table 1: Results of the univariate logistic regression model regarding the appreciation to interact orally with the system as predictor for a dichotomized response to physicians’ satisfaction.

Table 2: Results of the univariate logistic regression model with the appreciation that the sentences were understood by the system as a predictor for a dichotomized response to physicians’ satisfaction

Table 3: Results of the univariate logistic regression model with the appreciation of translating directly from the list of sentences as a predictor for a dichotomized response to physicians’ satisfaction

Discussion

To our knowledge, this is the first study to evaluate the diagnostic performance of a computer-based, fixed-phrase translation tool in a clinical setting with real patients. Descriptive results showed that the large majority of users, both physicians and patients, were satisfied with the tool. Importantly, the quality of translation was appreciated by the patients and they felt able to express their problem despite non-verbal answers. On a perceived efficacy level, most physicians were able to reach a diagnosis using Babel Dr. Moreover, we observed that less than one-half of all patients would not have preferred to use an interpreter during the consultation. Nevertheless, many physicians were not able to translate all of their questions. Finally, evaluation of the technical components was very positive and features such as the speech recognition system were appreciated by the vast majority of physicians.

In this study, we were able to emphasize the benefits of fixedphrase translations with a speech recognition system in a real- life emergency setting. Even though this type of translation feels restrictive, it guarantees an accurate and controlled transfer of information that is specific to the field in question. So far, experimental studies with such tools have shown promising results in terms of translation quality and physicians’ satisfaction [31,33]. This positive tendency towards fixed-phrase translators was also recently reported in a study by Panayiotou et al where participants seemed more favorable towards fixedphrase translation apps than real-time, voice-to-voice mobile translation apps [34]. Furthermore, a recent study comparing Quick Speak (an emergency medical service with fixed-phrase translation software) to a free-text translation tool (such as Google Translation) showed that patients with limited English proficiency had a preference toward a domain-specific system with fixed questions [32].

We were able to confirm these findings in a clinical setting and also evaluate the patient perspective, which proved to be positive. The latter could be explained by the fact that cultural norms and values were taken into account in the translations. Notably, fixed-phrase translation allows to incorporate culturally- specific registers, thus enhancing the communication experience. For example, topics such as sexual health are taboo in certain languages and the communication of this medical information requires some adjustments. In Farsi, the question “Have you ever undergone a breast biopsy?” would be expressed in the following way by a Persian physician “Has a physician ever taken a sample of your chest for diagnosis” (literal translation). For some languages, these types of adaptations are essential for patient-physician communication (Halimi S, Razieh A, Bouillon P, Spechbach H. Pee or urinate? A corpus-based analysis of medical communication for context-specific responses. Manuscript in preparation). However, a literal translation of elliptical utterances is rarely possible without affecting communication and can lead to translation problems [35].

Another interesting aspect raised by our study was related to the use of non-verbal answers by patients. At first glance, it could be supposed that this answer modality is too restrictive and frustrating for patients and thus diminishes the user experience. However, our results showed the contrary. Thus, we can speculate that the documentation of the patient history led by the physician in an emergency setting with close-ended questions was precise and sufficient to identify the patient’s problem. An additional result was that not all patients would prefer to have an interpreter during their consultation, despite being described as the gold standard [14].

Moreover, our study highlights technical features that may help future system developers. First, we focused on user-friendliness, which was facilitated by a speech recognition system. Qualitative analysis showed that this feature was very much appreciated by physicians and significantly predicted their satisfaction with the tool. This result is consistent with the literature [36]. Furthermore, data extracted from the log showed that of 948 translated sentences, physicians translated 59.1% of sentences using the speech recognition system and 40.9% through the list of sentences present in the tool. The speech recognition system provides numerous benefits [37]. It allows a faster interaction with the system and therefore can lead to faster care for the patient. A previous study comparing Babel Dr. with a traditional phraselator (MediBabble) showed a shorter time of use with an average time in favour of Babel Dr. by 11 seconds (95% confidence interval 4.6-17.3; P<0.001) [38]. Phraselators have been often criticized for their lack of flexibility [32]. However, the use of a speech recognition system allowed integrating a new level of flexibility compared to a traditional phraselator. Since numerous spoken variations are coded to the same core sentence, this allows the physicians more flexibility during their inquiry. This large mapping could permit a better efficacy in the use of the tool and adds a sense of freedom that ultimately “smooths” the user experience. This assumption was confirmed by our study as results showed that physicians considered that the tool well understood the spoken questions. Statistical analyses also showed that a positive attitude towards the tool’s understanding of the spoken questions can predict a higher satisfaction rate among physicians. Therefore, this technical feature provides the users more freedom in speech. A last feature worth mentioning is the physicians’ positive appreciation to translate directly from a list of selectable sentences. Analyses showed that this feature can also predict physician satisfaction. This well demonstrates that all three technical features incorporated in Babel Dr.’s architecture are important and lead towards a positive satisfaction as a medical translation app, with a slight edge for the ability to be able to interact orally with the system. Consistent with the suggestions of Turner et al for an ideal translation tool, pairing the accuracy and clarity of fixed-phrase translations with the flexibility of a speech recognition system can be a guide for future translation tools [32].

In clinical practice, Babel Dr. can be considered as a potential valuable resource, especially in the ED where time management is critical. From a public health perspective, Babel Dr.’s goal is to reduce healthcare disparities between populations. It allows extending patient care to certain minorities, such as allophone or deaf-mute individuals, who can be reluctant to consult due to the language barrier. It is a known fact that language mismatch between patients and medical staff can lead to a negative effect [4,39-44]. Another benefit of Babel Dr. is that it can be used anywhere, such as secluded areas or in large migration camps where access to an interpreter is difficult. Babel Dr. is also cost-effective and can help reduce unnecessary expenses in different departments.

Limitations

Our study has some limitations. First, we must address our limited sample size. This study was restricted at present to the five output language available on Babel Dr. and limited to only one input language (French). Our study was conducted during the Covid-19 confinement period which reduced by far the affluence of people visiting the ED. Second, the study was conducted in a single center during office hours, thus excluding night-time situations where professional interpreters are more difficult to reach. Third, Albanian was added late and explains why there was only one patient identified for this study. Fourth, technical difficulties (microphone issues, loud surroundings, or glitches) may have hindered the use of the speech recognition system compared to the selection of sentences in the list. Finally, the study was conducted under real-life conditions and some physicians were rushed to visit the patient due to a heavy work load and therefore skipped additional training on the use of the tool. This may explain some of the negative feedback encountered concerning the fact that only a few physicians were able to ask all of their questions.

Unanswered Questions and Future Research

This study paves the way for several questions. A future study comparing the attitudes towards translation tools such as Babel Dr. and professional interpreters would allow a better grasp of all aspects linked to communication during consultations. Another interesting perspective is to determine additional ways, other than speech, for patients to express themselves and which could be generalized to all populations. Concerning future developments, the goal is to evolve Babel Dr. towards an even more flexible tool, incorporating more content, more languages (such as Turkish, Romanian and Chinese), and even bypass close-ended questions. These additions will depend on the specific demand and on migratory flows. Moreover, we aim to reach a global stage by adding English as a new input language, thus allowing a widespread use of Babel Dr. worldwide. Using latest neuronal technologies and its unique architecture, Babel Dr. can be tailored to different situations and adapted to any context. Our next step is to develop and evaluate a new bidirectional version of Babel Dr. using open-ended questions where patients can contribute additional details about their condition by selecting between numerous pictograms (Figure 4). A recent study has shown promising results in the use of pictograms to communicate in a medical context [44]. This could be the first step in the development of a new universal medical language with the potential to be implemented anywhere and understood across many cultures, such as the universal road sign language, which is understood by any driver.

Figure 4: Babel Dr. bidirectional beta version

Conclusion

Evaluation of the performance of Babel Dr., a novel medical fixed-phrase translation tool, in a clinical patient-physician interaction in an emergency context showed that it was able to provide high quality and reliable translations for the patient and consequently allow for better care. Satisfaction ratings were high for both patients and physicians. Patients could express their problem despite being restricted to non-verbal responses and physicians were able to make different diagnoses with the translation tool. Babel Dr. overcame different drawbacks existing in other translation methods and can be an effective alternative solution in an emergency setting where interpreters are difficult to reach.

Acknowledgement

We thank the Private Foundation of Geneva University Hospitals for its contribution to this project. We thank Rosemary Sudan for editorial assistance.

Funding

This work was funded by the Private Foundation of Geneva University Hospitals.

Availability of Data and Materials

Relevant anonymised patient level data available on reasonable request from Dr. Herve Spechbach (Herve.Spechbach@ hcuge.ch).

Competing Interests

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Consent for Publication

Informed consent was provided by all the participants of this study.

Authors' Contribution

Dr Herve Spechbach led and supervised this study. Johanna Gerlach and Pre Pierrette Bouillon read and commented on the manuscript. Antony Akash Janakiram wrote the manuscript.

References

1. International Organization for Migration (2019) World Migration Report 2020. Geneva, Switzerland.
2. Pandya C, McHugh M, Batalova J (2011) Limited english proficient individuals in the united states: Number, share, growth, and linguistic diversity. LEP Data Brief. Migration Policy Institute.

   [Google Scholar]

3. World Migration Report 2020
4. Jacobs E, Chen AH, Karliner LS, Agger-Gupta N, Mutha S (2006) The need for more research on language barriers in health care: A proposed research agenda. Milbank Q. 84(1):111-133.

   [Crossref] [Google Scholar] [PubMed]

5. Cox A, Maryns K (2021) Multilingual consultations in urgent medical care. The Translator. 27(1): 1-19.

   [Crossref] [Google Scholar] [Research Gate]

6. Ramirez D, Engel KG, Tang TS (2008) Language interpreter utilization in the emergency department setting: A clinical review. J Health Care Poor Underserved. 19(2):352-362.

   [Crossref] [Google Scholar] [PubMed]

7. Waxman MA, Levitt MA (2000) Are diagnostic testing and admission rates higher in non-English speaking versus English-speaking patients in the emergency department? Ann Emerg Med. 36(5):456-461.

   [Crossref] [Google Scholar] [PubMed]

8. Hampers LC, Cha S, Gutglass DJ, Binns HJ, Krug SE (1999) Language barriers and resource utilization in a pediatric emergency department. Pediatrics. 103(6):1253-1256.

   [Crossref] [Google Scholar] [PubMed]

9. Manson A (1988) Language concordance as a determinant of patient compliance and emergency room use in patients with asthma. Med Care. 26(12):1119-1128.

   [Crossref] [Google Scholar] [PubMed]

10. Ozolins LL, Hjelm K (2003) Nurses’ experiences of problematic situations with migrants in emergency care in Sweden. Clin Eff Nurs. 7(2):84-93.

    [Crossref] [Google Scholar] [Research Gate]

11. Sarver J, Baker DW (2000) Effect of language barriers on follow-up appointments after an emergency department visit. J Gen Intern Med. 15(4):256-264.

    [Crossref] [Google Scholar] [PubMed]

12. Carrasquillo O, Orav EJ, Brennan TA, Burstin HR (1999) Impact of language barriers on patient satisfaction in an emergency department. J Gen Intern Med. 14(2):82-87.

    [Crossref] [Google Scholar] [PubMed]

13. Cox A, Rosenberg E, Thommeret-Carrière AS, Huyghens L, Humblé P, et al. (2019) Using patient companions as interpreters in the Emergency Department: An interdisciplinary quantitative and qualitative assessment. Patient Educ Couns. 102(8):1439- 1445.

    [Crossref] [Google Scholar] [PubMed]

14. Flores G (2005) The impact of medical interpreter services on the quality of health care: A systematic review. Med Care Res Rev. 62(3):255-299.

    [Crossref] [Google Scholar] [PubMed]

15. Karliner LS, Jacobs EA, Chen AH, Mutha S (2007) Do professional interpreters improve clinical care for patients with limited english proficiency? A systematic review of the literature. Health Serv Res. 42(2):727-754.

    [Crossref] [Google Scholar] [PubMed]

16. Kazzi GB, Cooper C (2003) Barriers to the use of interpreters in emergency room paediatric consultations. J Paediatr Child Health. 39(4):259–263.

    [Crossref] [Google Scholar] [PubMed]

17. Jaeger FN, Pellaud N, Laville B, Klauser P (2019) The migration-related language barrier and professional interpreter use in primary health care in Switzerland. BMC Health Serv Res. 19(1):429.

    [Crossref] [Google Scholar] [PubMed]

18. Sandler R, Myers L, Springgate B (2014) Resident physicians’ opinions and behaviors regarding the use of interpreters in New Orleans. South Med J. 107(11):698–702.

    [Crossref] [Google Scholar] [PubMed]

19. Baker DW, Parker RM, Williams MV, Coates WC, Pitkin K (1996) Use and effectiveness of Interpreters in an emergency department. JAMA. 275(10):783.

    [Crossref] [Google Scholar] [PubMed]

20. Cox A, Lázaro Gutiérrez R (2016) Interpreting in the emergency department: How context matters for practice. Mediating Emergencies and Conflicts. 33–58.

    [Crossref] [Google Scholar]

21. Blay N, Ioannou S, Seremetkoska M, Morris J, Holters G, et al. (2018) Healthcare interpreter utilization: Analysis of health administrative data. BMC Health Serv Res. 18(1):348.

    [Crossref] [Google Scholar] [PubMed]

22. Gan SK-E, Koshy C, Nguyen P-V, Haw Y-X (2016) An overview of clinically and healthcare related apps in Google and Apple app stores: Connecting patients, drugs, and clinicians. Scient Phn Apps Mob Devices. 2(1):8.

    [Google Scholar]

23. Koehler N, Vujovic O, McMenamin C (2013) Healthcare professionals’ use of mobile phones and the internet in clinical practice. J mob tech med. 2(1):3–13.

    [Google Scholar]

24. Quinn CC, Clough SS, Minor JM, Lender D, Okafor MC, et al. (2008) Welldoctm mobile diabetes management randomized controlled trial: Change in clinical and behavioral outcomes and patient and physician satisfaction. Diabetes Technol Ther. 10(3):160–8.

    [Crossref] [Google Scholar] [PubMed]

25. Fortier MA, Chung WW, Martinez A, Gago-Masague S, Sender L (2016) Pain buddy: A novel use of m-health in the management of children’s cancer pain. Comput Biol Med. 76:202–14.

    [Crossref] [Google Scholar] [PubMed]

26. Mahmood A, Blaizy V, Verma A, Stephen Sequeira J, et al. (2019) Acceptability and attitude towards a mobile-based home exercise program among stroke survivors and caregivers: A cross-sectional study. Int J Telemed Appl. 2019:e5903106.

    [Crossref] [Google Scholar] [PubMed]

27. Kebede M, Zeleke A, Asemahagn M, Fritz F (2015) Willingness to receive text message medication reminders among patients on antiretroviral treatment in North West Ethiopia: A cross-sectional study. BMC Med Inform Decis Mak. 15(1):65.

    [Crossref] [Google Scholar] [PubMed]

28. Mira JJ, Navarro I, Botella F, Borrás F, Nuño-Solinís R, et al. (2014) A spanish pillbox app for elderly patients taking multiple medications: Randomized controlled trial. J Med Internet Res. 16(4):e3269.

    [Crossref] [Google Scholar] [PubMed]

29. Patil S, Davies P (2014) Use of Google Translate in medical communication: Evaluation of accuracy. BMJ. 15:349:g7392

    [Crossref] [Google Scholar] [PubMed]

30. Khoong EC, Steinbrook E, Brown C, Fernandez A (2019) Assessing the use of Google translate for Spanish and Chinese translations of emergency department discharge instructions. JAMA Intern Med. 179(4):580.

    [Crossref] [Google Scholar] [PubMed]

31. Bouillon P, Gerlach J, Spechbach H, Tsourakis N, Halimi Mallem IS. BabelDr vs Google Translate: A user study at Geneva University Hospitals (HUG).

    [Google Scholar]

32. Turner AM, Choi YK, Dew K, Tsai M-T, Bosold AL, et al. (2019) Evaluating the usefulness of translation technologies for emergency response communication: A scenario-based study. JMIR Public Health Surveill. 5(1):e11171.

    [Crossref] [Google Scholar] [PubMed]

33. Spechbach H, Gerlach J, Karker SM, Tsourakis N, Combescure C, et al. (2019) A speech-enabled fixed-phrase translator for emergency settings: Crossover study. JMIR Med Inform. 7(2):e13167.

    [Crossref] [Google Scholar] [PubMed]

34. Panayiotou A, Hwang K, Williams S, Chong TWH, LoGiudice D, et al. (2020) The perceptions of translation apps for everyday health care in healthcare workers and older people: A multi-method study. J Clin Nurs. 29(17–18):3516–26.

    [Crossref] [Google Scholar] [PubMed]

35. Mutal JD, Gerlach J, Bouillon P, Spechbach H (2020) Ellipsis translation for a medical speech to speech translation system.

    [Google Scholar]

36. Lyons JP, Sanders SA, Fredrick Cesene D, Palmer C, Mihalik VL, et al. (2016) Speech recognition acceptance by physicians: A temporal replication of a survey of expectations and experiences. Health Informatics J. 22(3):768–78.

    [Crossref] [Google Scholar] [PubMed]

37. Johnson M, Lapkin S, Long V, Sanchez P, Suominen H, et al. (2014) A systematic review of speech recognition technology in health care. BMC Med Inform Decis Mak. 14(1):94.

    [Crossref] [Google Scholar] [PubMed]

38. Boujon V, Bouillon P, Spechbach H, Gerlach J, Strasly I (2018) Can speech-enabled phraselators improve healthcare accessibility? A case study comparing babeldr with medibabble for anamnesis in emergency settings.

    [Google Scholar]

39. van Rosse F, de Bruijne M, Suurmond J, Essink-Bot M-L, Wagner C (2016) Language barriers and patient safety risks in hospital care. A mixed methods study. Int J Nurs Stud. 54:45–53.

    [Crossref] [Google Scholar] [PubMed]

40. Cohen AL, Rivara F, Marcuse EK, McPhillips H, Davis R (2005) Are language barriers associated with serious medical events in hospitalized pediatric patients? Pediatrics. 116(3):575–9.

    [Crossref] [Google Scholar] [PubMed]

41. Karliner LS, Kim SE, Meltzer DO, Auerbach AD (2010) Influence of language barriers on outcomes of hospital care for general medicine inpatients. J Hosp Med. 5(5):276–82.

    [Crossref] [Google Scholar] [PubMed]

42. David RA, Rhee M (1998) The impact of language as a barrier to effective health care in an underserved urban Hispanic community. Mt Sinai J Med. 65(5–6):393–7.

    [Google Scholar] [PubMed]

43. Divi C, Koss RG, Schmaltz SP, Loeb JM (2007) Language proficiency and adverse events in US hospitals: A pilot study. Int J Qual Health Care. 19(2):60–7.

    [Crossref] [Google Scholar] [PubMed]

44. Norré M, Bouillon P, Gerlach J, Spechbach H (2020) Évaluation de la compréhension de pictogrammes Arasaac et Sclera pour améliorer l’accessibilité du système de traduction médicale BabelDr

    [Google Scholar]