|
|
ENQUIRE PROJECT DETAILS BY GENERAL PUBLIC |
| Project Details |
| Funding Scheme : | General Research Fund | ||||||||||||||||||||||||||||||
| Project Number : | 16202917 | ||||||||||||||||||||||||||||||
| Project Title(English) : | Enabling Developers and Manufacturers to Tame Android Compatibility Issues | ||||||||||||||||||||||||||||||
| Project Title(Chinese) : | 協助開發和製造商克服Android兼容性問題之研究 | ||||||||||||||||||||||||||||||
| Principal Investigator(English) : | Prof Cheung, Shing-chi | ||||||||||||||||||||||||||||||
| Principal Investigator(Chinese) : | |||||||||||||||||||||||||||||||
| Department : | Dept of Computer Science & Engineering | ||||||||||||||||||||||||||||||
| Institution : | The Hong Kong University of Science and Technology | ||||||||||||||||||||||||||||||
| E-mail Address : | scc@cse.ust.hk | ||||||||||||||||||||||||||||||
| Tel : | 2358 7016 | ||||||||||||||||||||||||||||||
| Co - Investigator(s) : |
|
||||||||||||||||||||||||||||||
| Panel : | Engineering | ||||||||||||||||||||||||||||||
| Subject Area : | Computing Science & Information Technology | ||||||||||||||||||||||||||||||
| Exercise Year : | 2017 / 18 | ||||||||||||||||||||||||||||||
| Fund Approved : | 700,000 | ||||||||||||||||||||||||||||||
| Project Status : | Completed | ||||||||||||||||||||||||||||||
| Completion Date : | 30-6-2021 | ||||||||||||||||||||||||||||||
| Project Objectives : |
|
||||||||||||||||||||||||||||||
| Abstract as per original application (English/Chinese): |
Android是受歡迎的移動操作系統，支持數百萬個應用。然而，Android生態系統的沉重分裂引起了各種Android版本和設備之間應用程序兼容性的一些主要擔憂。在具有不兼容的Android版本或硬件驅動程序的設備上運行時，應用程序會出現意外的行為，甚至無法運行。為了解決這些兼容性問題，開發人員需要分析Android版本和設備型號的那些組合可能導致意外的行為並相應地自定義代碼。現有研究報告說，Android兼容性問題是普遍的：通常發現用戶評論抱怨應用程序在其設備上或系統更新後無效。造成這種現象的四個原因。首先，Android生態系統是分散的，使兼容性分析變得昂貴。在超過24,000個不同的設備型號上部署了超過50種Android版本，具有不同的硬件和驅動程序。兼容性問題可能來自於這些的任何組合，但在實踐中，開發人員只能靠這些組合的一小部分來測試他們的應用程序。第二，Android兼容性問題沒有很好的記錄。開發人員主要從經驗或非結構化的討論中了解問題。第三，隨著新的Android版本和設備型號的發布，新的問題不斷出現。很難通過不同的非結構化來源及時跟踪這些問題。最後，沒有相關的自動化工具。 我們建議使用大數據在三個維度上解決兼容性問題：問題表徵，知識庫構建和工具自動化。為了提高對兼容性問題的分散理解，我們將進行實證研究，以基於代碼存儲庫和2000多個開源Android應用程序的問題跟踪系統來表徵問題，構建兼容性問題的知識庫。由於這些問題隨著新版本的Android版本，庫和設備模型的發布而發生變化，因此我們可以通過使用人群數據開發代碼挖掘框架來自動識別新出現的問題，並根據需要更新知識庫。我們將開發自動化技術和工具，以幫助檢測和修補Android應用程序中的兼容性問題。我們將使用流行的真實世界的Android應用程序評估我們的方法。 Android應用開發是中國和香港信息產業的重要組成部分。本項目開發的知識庫和工具將提高開發人員的生產力。他們還將通過解決開發新產品時的兼容性問題來惠及移動設備供應商。 |
||||||||||||||||||||||||||||||
| Realisation of objectives: | We have fully achieved the three stated objectives by carrying out the following research activities. Objective 1: The heavily fragmented Android ecosystem has induced various compatibility issues in Android apps. We made empirical studies whether compatibilities issues arise from callback APIs in Android apps and how these apps respond to events induced by incompatible callback APIs. We examined Android documentations and conducted an empirical study on 100 real-world callback compatibility issues to investigate how these issues were induced by callback API evolutions. Based on our empirical findings, we have proposed a graph-based model to capture the control flow inconsistencies caused by API evolutions on how callback APIs are invoked in response to various events. We also extended our empirical studies to cover those hybrid Android apps with web browser capabilities. We found that the interaction between an app’s native code and web code induces new types of bugs, referred to as WebView-induced bugs, in Android apps. Our study identified new characteristics of these bugs. In addition, we also studied the issues induced by conflicts induced by the compatibilities across different versions of third-party libraries. Four patterns of such compatibilities were identified. These patterns allow for future systematic studies of such compatibilities. The above findings were disseminated at three top-tier research venues [Conference-3,4,5]. We followed up these findings with further real-world compatibility issues collected from five popular open-source Android apps. We interviewed Android practitioners and conducted an online survey to gain insights from real practices. We also made attempts to collect and analyze resource leakages occurring in Android apps since some of those leakages may arise from compatibilities across API levels and devices [Journal-2]. During the process of analyzing the root causes of compatibility issues, we often needed to locate from the old program versions the bug-inducing commits that introduced bugs repaired later by a fixing commit. We found that the process of locating bug-inducing commits has not been well-supported by existing methodologies. We conducted an exploratory study of the problem and disseminated our finding at a top-tier research venue [Conference-9]. Objective 2: We commenced our study to develop a framework after submission of proposal. We studied the high false positive rates of popular static analysis tools available in Android studio for the detection of software defects, including compatibility issues. A novel technique was developed to effectively reduce the false positive rates by augmenting the detected issues with user reviews and knowledge graph. We also developed techniques to further reduce false positive warnings based on failure clustering by means of crash points and crash origins. We developed a methodology to help developers locate their earlier code changes that are accountable for a given software issue. Automated detection techniques for compatibility issues arising from Android callback APIs and hybrid Android apps were developed. An automated technique was proposed to fix detected program issues. The results were disseminated at four top-tier research conferences [Conference-1,2,3,4]. We mined these issues by exploring the search space in three dimensions: device models, Android OS versions, and Android APIs. We will address two challenges. First, the issues arising from device models, evolve quickly with the frequent release of new device models to the market. Second, new issues are mostly undocumented. We developed a technique that automatically learns API-device correlations of FIC issues from existing Android apps. The technique extracts and prioritizes API-device correlations from a given corpus of Android apps. The result was disseminated at a top-tier research conference [Conference-1]. Besides Android apps, we also explored the detection of compatibility issues across Java projects [Conference-7], API libraries [Conference-8], and API documentation [Journal-1]. Objective 3: We implemented tool supports of our techniques and released our collected representative real-world Android applications as well as evaluation results to the public. The URLs of our publicly accessible project web sites were published. We evaluated our technique with large-scale open-source Android apps and evaluated if it can detect compatibility and uncover previously unknown issues. We have constructed a publicly available project website (http://castle.cse.ust.hk/ficfinder/index.html) and released our tools together with benchmark datasets at five open-source repositories hosted at GitHub. - PIVOT: https://github.com/FICIssuePivot/Pivot - CIDER: https://cideranalyzer.github.io/index.html - wDroid: https://github.com/RichardHoOoOo/wDroid - DECCA: https://github.com/DeccaDC/Decca - InduceBenchmark: https://github.com/justinwm/InduceBenchmark The research artifact of PIVOT was considered very useful to the community and awarded the ACM SIGSOFT Distinguished Artifact at ICSE 2019. | ||||||||||||||||||||||||||||||
| Summary of objectives addressed: |
|
||||||||||||||||||||||||||||||
| Research Outcome | |||||||||||||||||||||||||||||||
| Major findings and research outcome: | This project led to the training of 3 research students, publication of 12 top-tier research articles and 5 public project repositories at GitHub. The student, Ming Wen (2019 graduate), is an associate professor at Huazhong University of Science and Technology. Another student, Lili Wei (2020 graduate), is a postdoc supported by the RGC Postdoctoral Fellowship award. In recognition of her outstanding research, she received PhD fellowships from Google and Microsoft Research Asia. A feature interview of her by HKUST is available at https://seng.hkust.edu.hk/news/20191031/leveraging-computer-science-turn-imagination-reality. She will join McGill University as an assistant professor. The student, Hang Xu (2021 graduate), is a technical staff member at Tencent. The research artifact submitted to ICSE 2019 received the ACM SIGSOFT Distinguished Artifact award. We summarize major findings as follows. - Compatibility issues are not documented as the Android framework developers are likely unaware of them. These issues escape most of the unit tests. Another property of these issues is that they keep evolving. It means issues that have been detected in previous Android framework versions may no longer be applicable to the current or future Android framework versions. [Conference-2, 5] - API documentation are often outdated. The documentation may not accurately describe the behavior of the corresponding implementation. [Journal-1] - Compatibility issues can be broadly categorized into device non-specific and device specific. The former are issues that arise from the core Android framework components. They occur across different device models. It means that the fix to an issue can be applied to many device models. The latter are issues that arise from the components customized by Android device vendors. It means that the fix to an issue is applicable only to specific device models. [Conference-5] - The control flows of Android apps are largely driven by WebView and the protocols that govern how callback APIs are invoked in response to various events. These WebView and callback APIs evolve along with the Android framework, resulting in many compatibility issues. [Conference-2, 3] - Compatibility issues are often introduced by various vendors when they customize the Android implementation for their products. Such device specific issues are hard because most vendors do not publish such issues. App developers need to execute their applications on such device models to realize the issues. Motivated by the difficulties, we propose a mining technique to automatically extract the possible compatibility fixes from the latest obfuscated APK distribution of the close-source apps. These fixes are then be abstracted into common device specific compatibility patterns, which can then be applied to detect related compatibility issues in other apps. [Conference-6] - Many of the issues in Android apps result in resource leaks, which escape the code checking and testing by their developers. [Journal-2] | ||||||||||||||||||||||||||||||
| Potential for further development of the research and the proposed course of action: |
Android fragmentation is an outstanding problem intrinsic to the Android ecosystem. Our project laid the foundation work on the characterization and detection of the compatibility issues induced by Android fragmentation. This project mainly focused on the issues occurring at Java apps. Recently, apps are built using multiple platforms, including XML for graphical user interface rendering and WebView for web content rendering, in addition to Java. A promising extension of the work is to extend compatibility issues to apps built on top of multiple platforms. Another interesting direction is to explore the automation of repair for these issues. A major challenge is to work out the best repair strategy when multiple platforms are involved. Indeed, compatibility issues are not restricted to the Android ecosystem. We also found the issues are also pervasive in Java, Python, Go and Python ecosystems. A possible direction is to transfer the methodology developed in this project to the detection of compatibility issues in other ecosystems, including the emerging ones that support the development of artificial intelligence applications. | ||||||||||||||||||||||||||||||
| Layman's Summary of Completion Report: | An Android app that works on a particular Android version and device model may no longer work when users deploy the app on another Android version or device model. This is because the support by the underlying Android framework varies across Android versions and device models. In most cases, app developers rarely have the resources to test their apps on all active Android versions and all major device models before releasing their apps. Indeed, such compatibility issues escape most unit tests and undocumented. In addition, they keep evolving as the upgrade of Android versions and device models. Addressing the compatibility issues is a well-known hard problem in the Android community. In this project, we conducted a systematic study, including interviews with experienced Android app developers, to characterize the form of these compatibility issues and the patterns by which the issues can be effectively detected. We developed the first benchmark dataset on Android compatibility issues and initiated five GitHub projects. The research findings were disseminated by 12 top-tier research articles. Three research students were trained. | ||||||||||||||||||||||||||||||
| Research Output | |||||||||||||||||||||||||||||||
| Peer-reviewed journal publication(s) arising directly from this research project : (* denotes the corresponding author) |
|
||||||||||||||||||||||||||||||
| Recognized international conference(s) in which paper(s) related to this research project was/were delivered : |
|
||||||||||||||||||||||||||||||
| Other impact (e.g. award of patents or prizes, collaboration with other research institutions, technology transfer, etc.): |
|||||||||||||||||||||||||||||||
| SCREEN ID: SCRRM00542 |