Dataset:
Task guidelines and the CDF questionnaire instrument of security API usability study

ac.person.orcid 0000-0001-9192-0993
ac.person.orcid 0000-0001-5332-5341
ac.person.position Staff
ac.person.position Staff
dc.date.accessioned 2021-11-26T10:46:47Z
dc.date.available 2021-11-26T10:46:47Z
dc.date.issued 2019 en_US
dc.description.abstract Research in the field of usable security and privacy suffered in recent times due to the lack of systematic approach for evaluating the usability of security APIs. As a solution to this problem, we proposed a Cognitive Dimensions Framework (CDF) based methodology to evaluate the usability of security APIs [1,2]. Thereafter, we conducted an empirical investigation to evaluate the proposed methodology. In this evaluation, we employed the proposed methodology to evaluate the usability of four security APIs (Java Secure Socket Extension (JSSE) API, Bouncy Castle lightweight crypto API, OWASP Enterprise Security API, Google authentication API). We expected to evaluate the proposed methodology by measuring its thoroughness, validity, effectiveness, and reliability. This data set includes the study material used in this experiment, which includes the CDF based questionnaire and the task guidelines (including code artifacts) used for each component of the study. Results revealed that the proposed methodology identified 80% of the usability issues of each APIs with only seven participant programmers [3]. It was also observed that the validity of the issues identified was between 60% - 80% in all four evaluations [3]. In addition to these findings about the proposed usability evaluation methodology, the results of this study revealed interesting details about usability issues that exist in the four security APIs that were mentioned above [4-7]. 1. Chamila Wijayarathna, Nalin Asanka Gamagedara Arachchilage, and Jill Slay. ``A generic cognitive dimensions questionnaire to evaluate the usability of security APIs." International Conference on Human Aspects of Information Security, Privacy, and Trust. Springer, Cham, 2017. 2. Chamila Wijayarathna, and Nalin Asanka Gamagedara Arachchilage. ``A methodology to evaluate the usability of security APIs." Proceedings of the 9th IEEE International Conference on Information and Automation for Sustainability, 2018. 3. Chamila Wijayarathna, and Nalin Asanka Gamagedara Arachchilage. ``Using cognitive dimensions to evaluate the usability of security APIs: An empirical investigation." Information and Software Technology 115 (2019): 5-19. 4. Chamila Wijayarathna, and Nalin Asanka Gamagedara Arachchilage. ``Why Johnny can’t develop a secure application? A usability analysis of Java Secure Socket Extension API." Computers \& Security 80 (2019): 54-73. 5. Chamila Wijayarathna, and Nalin Asanka Gamagedara Arachchilage. ``Why Johnny can't store passwords securely?: A usability evaluation of Bouncy Castle password hashing." Proceedings of the 22nd International Conference on Evaluation and Assessment in Software Engineering 2018. ACM (2018). 6. Chamila Wijayarathna, and Nalin Asanka Gamagedara Arachchilage. ``Fighting against XSS attacks: A usability evaluation of OWASP ESAPI output encoding." Proceedings of the 52nd Hawaii International Conference on System Sciences (HICSS) (2019). 7. Chamila Wijayarathna, and Nalin Asanka Gamagedara Arachchilage. ``An empirical usability analysis of the Google authentication API." Proceedings of the Evaluation and Assessment on Software Engineering. ACM (2019). en_US
dc.identifier.uri http://hdl.handle.net/1959.4/resource/collection/resdatac_931/1
dc.language English
dc.language.iso EN en_US
dc.rights CC-BY
dc.rights.uri https://creativecommons.org/licenses/by/4.0/ en_US
dc.subject.other usable security and privacy en_US
dc.subject.other security API usability en_US
dc.subject.other secure development en_US
dc.title Task guidelines and the CDF questionnaire instrument of security API usability study en_US
dc.type Dataset en_US
dcterms.accessRights open access
dcterms.accrualMethod This study attempted to evaluate the proposed Cognitive Dimensions Framework (CDF) based Usability Evaluation Methodology (UEM) by evaluating its thoroughness, validity, effectiveness and methodology. Therefore, the proposed UEM was employed to evaluate the usability of four security APIs (Java Secure Socket Extension (JSSE) API, Bouncy Castle lightweight crypto API, OWASP Enterprise Security API, Google authentication API). When employing the proposed UEM, we recruited several programmers per each study and asked them to complete a task that involves the corresponding security API. Participants participated remotely, therefore, the corresponding task guidelines were emailed to them. They participated remotely using their own equipment. Participants were asked to think-aloud while completing the task. Furthermore, they were asked to record their computer screens with the voice to record the procedures they followed and their think-aloud data. After completing the task, they completed the CDF questionnaire instrument via Google forms and then emailed the code they developed and their recordings. Questionnaire responses were analyzed by referring to the CDF to identify usability issues that each programmer has encountered. In addition to that, recordings and code artifacts were also analyzed to identify usability issues. Usability issues identified by the two methods were compared in order to measure the thoroughness, validity, reliability and effectiveness of using the CDF questionnaire to evaluate the usability of security APIs [1]. 1. Chamila Wijayarathna, and Nalin Asanka Gamagedara Arachchilage. ``Using cognitive dimensions to evaluate the usability of security APIs: An empirical investigation." Information and Software Technology 115 (2019): 5-19. en_US
dcterms.accrualMethod https://www.sciencedirect.com/science/article/pii/S0950584919301624
dspace.entity.type Dataset en_US
unsw.accessRights.uri https://purl.org/coar/access_right/c_abf2
unsw.contributor.leadChiefInvestigator Diwelwatta Gamage, Chamila Dilshan en_US
unsw.contributor.researchDataCreator Gamagedara Arachchilage, Nalin Asanka en_US
unsw.description.storageplace UNSW-Canberra en_US
unsw.identifier.doi https://doi.org/10.26190/5da65c5981f7c en_US
unsw.isPublicationRelatedToDataset https://doi.org/10.1016/j.infsof.2019.07.007 en_US
unsw.isPublicationRelatedToDataset Wijayarathna, C & Arachchilage, N 2018, 'Why Johnny Can't store passwords securely?: A usability evaluation of bouncycastle password hashing', in ACM International Conference Proceeding Series, https://doi.org/10.1145/3210459.3210483 en_US
unsw.isPublicationRelatedToDataset https://doi.org/10.1016/j.cose.2018.09.007 en_US
unsw.isPublicationRelatedToDataset Chamila Wijayarathna, and Nalin Asanka Gamagedara Arachchilage. ``Fighting against XSS attacks: A usability evaluation of OWASP ESAPI output encoding." Proceedings of the 52nd Hawaii International Conference on System Sciences (HICSS) (2019) https://doi.org/10.24251/HICSS.2019.877 en_US
unsw.isPublicationRelatedToDataset https://doi.org/10.1145/3319008.3319350 en_US
unsw.relation.OriginalPublicationAffiliation Diwelwatta Gamage, Chamila Dilshan, Sch of Engineering & IT (Sum), UNSW Canberra, en_US
unsw.relation.OriginalPublicationAffiliation Gamagedara Arachchilage, Nalin Asanka, , This record is inactive, as the person is not currently at UNSW., en_US
unsw.relation.faculty UNSW Canberra
unsw.relation.projectDesc The aim of this research is to develop a systematic approach to evaluate the usability of Security APIs. Past studies have shown that less usability of security APIs result in programmers unintentionally introducing security vulnerabilities to the applications they develop. Therefore, the usability of security APIs is considered important for creating applications and data secure. To develop more usable security APIs, those should be evaluated for usability, identify usability issues existing and fix those issues before making it available to use by other programmers. However, currently, there is no methodology to evaluate usability of security APIs and its one of the main barriers to develop more usable security APIs. Hence, the objective of this study is to develop a systematic approach which can be used by API developers to evaluate the usability of security APIs they develop. en_US
unsw.relation.projectStartDate 2016-07-25 en_US
unsw.relation.projectTitle Developing a systematic approach to evaluate the usability of security APIs en_US
unsw.relation.school School of Engineering and Information Technology
unsw.relation.school School of Engineering and Information Technology
unsw.subject.fieldofresearchcode 0803 Computer Software en_US
unsw.subject.fieldofresearchcode 080602 Computer-Human Interaction en_US
Files
Original bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
Research Instrument.zip
Size:
6.44 MB
Format:
application/zip
Description:
Resource type