ASIACCS 2019 Paper #303 Reviews and Comments =========================================================================== Paper #303 "I Don't Think I Can Share My Health Information ..." : Understanding Users' Risk Perceptions about Personal Health Records Shared on Social Networking Services Review #303A =========================================================================== Overall merit ------------- 4. Accept Reviewer expertise ------------------ 3. Knowledgeable Paper summary ------------- This paper presents a user study that explores users' privacy expectations and preferences when sharing PHRs on social networks. Strengths --------- - Very well executed user study. The qualitative aspects are also expertly handled. - Clean and informative presentation. All my "what if?"s were answered before I could finish asking the questions. Results are interpreted with one concise sentence without unnecessary speculation. Weaknesses ---------- - In general novelty is low, but results are useful nonetheless, so this is not a deal breaker. - The machine learning section is black magic. The paper could be better without it. See comments. Comments for author ------------------- - I'm always happy to see a well done user study or otherwise investigation of human factors at a systems security conference. Presenting the qualitative exploratory analysis in a scientific framework (good to see grounded theory!) is a big plus. Also, thank you for providing the details in the Appendices. - The obvious point of comparison is "Sharing Health Information on Facebook: Practices, Preferences, and Risk Perceptions of North American Users" published in SOUPS 2016. This submission is largely incremental and complementary on previous results. The differences and focus on more specific medical data here are good enough differentiators, and confirming/challenging previous findings isn't a negative either. I was a bit surprised how much the authors were influenced by the methodology of the SOUPS paper, though, as the similarities are extensive. - The big negative: I'm not convinced that the proposed classifier works. The bag-of-words approach is often used as a cop-out technique when researchers don't have a good understanding of what features are meaningful in the given context, and trying arbitrary classifiers in a brute-force manner until you get good accuracy doesn't help either. The results you present *suggest* that devising a method to assist users with their privacy decisions *may be possible*. You claim to "propose a practical privacy setting method"; but what you have is not a complete "method," and you've shown no evidence that it is practical. Please tone down these claims. - "All our participants were the residents of the United States" This is an obvious bias that must be addressed in the abstract and intro. For reference, the aforementioned SOUPS paper even includes this important information in the title. - Table 1: How did you determine R2 and R4 to be "key factors", what's your criteria? Same question for Table 2, 3, 4, and 5. Please explain. - "In summary, we found that more than half of participants perceived sharing PHR is risky, even though 9 of them actually still did share their PHR on SNS." What's the overlap between the two? - R6 not labeled with (Motivate) in Table 1. - "To identify main factors that influence the sharing of "U2: Diseases and illnesses on SNS"" Do you mean U1? - "With the crawled posts, we asked MTurkers in the confirmatory study to label sensitive posts that can be shared with everyone on SNS." What does "sensitive" mean in this context? - Personal nit: You *really* don't need the first part of the title. Review #303B =========================================================================== Overall merit ------------- 2. Weak reject Reviewer expertise ------------------ 3. Knowledgeable Paper summary ------------- The paper presented a survey to understand users’ risk perceptions about sharing their PHR on SNS. It proposed a privacy setting method to automatically determine whether users’ posts can be shared with everyone on SNS by analyzing the keywords frequently occurred in health-related posts. Strengths --------- The last part of the paper (privacy setting based on automatic classification by topic modelling) is of interest. Weaknesses ---------- There is not enough technical details in the paper. It is also not clear how useful the survey could be. Comments for author ------------------- The paper presented a survey to understand users’ risk perceptions about sharing their PHR on SNS. It proposed a privacy setting method to automatically determine whether users’ posts can be shared with everyone on SNS by analyzing the keywords frequently occurred in health-related posts. There is not enough technical details in the paper. It is also not clear how useful the survey could be. The last part of the paper (privacy setting based on automatic classification by topic modelling) is of interest. The author should focus on the bag-of-words techniques, and how to tune the algorithm in order to improve accuracy in this domain, Review #303C =========================================================================== Overall merit ------------- 3. Weak accept Reviewer expertise ------------------ 2. Some familiarity Paper summary ------------- The paper tries to understand why people share or not share health information on social networking services. First, the authors conducted an interview with a small number of participants to have qualitative understanding. Then, the authors recruited hundreds of participants from Amazon Mechanical Turk to quantitatively understand the reasons and factors of health information sharing. Finally, the authors use machine learning methods to predict whether a user is willing to share a post to public or not. Strengths --------- + Interesting problem. + Extensive study on the topic. Weaknesses ---------- - Results are not surprising. - Machine learning evaluation is misleading. Comments for author ------------------- The paper addresses an interesting problem, trying to understand whether and why people share health information on social networking services. The authors seem following a principled user study approach to study this problem. The authors also try to build a machine learning classifier to predict whether a user wants a post to be shared to everyone. 1. The machine learning results are misleading. The results are cross-validation results, which are more like training errors. In particular, you can tune the hyperparameters of a machine learning method to have a high accuracy for cross-validation. It is better to split the data as training, validation, and testing. The testing data should not be involved in tuning the parameters/hyperparameters in any way. 2. Is the privacy preference prediction personalized? Looks like the authors predict whether a post should be shared to everyone. But this decision may be user-dependent. So, a classifier should be built for each user. 3. Security concerns. Can you explain more details on the security concerns of sharing health information.