Assessment of Sexual Violence Risk Perception in Men Who Have Sex With Men: Proposal for the Development and Validation of “G-Date”

Background Sexual violence (SV) is a significant problem for sexual minorities, including men who have sex with men (MSM). The limited research suggests SV is associated with a host of syndemic conditions. These factors tend to cluster and interact to worsen one another. Unfortunately, while much work has been conducted to examine these factors in heterosexual women, there is a lack of research examining MSM, especially their SV risk perception. Further, MSM are active users of dating and sexual networking (DSN) mobile apps, and this technology has demonstrated usefulness for creating safe spaces for MSM to meet and engage partners. However, mounting data demonstrate that DSN app use is associated with an increased risk for SV, especially given the higher likelihood of using alcohol and other drugs before sex. By contrast, some researchers have demonstrated that DSN technology can be harnessed as a prevention tool for HIV; unfortunately, no such work has progressed regarding SV. Objective This study aims to (1) use qualitative and quantitative methods to tailor an existing laboratory paradigm of SV risk perception in women for MSM using a DSN mobile app framework and (2) subject this novel paradigm to a rigorous validation study to confirm its usefulness in predicting SV, with the potential for use in future prevention endeavors. Methods To tailor the paradigm for MSM, a team of computer scientists created an initial DSN app (G-Date) and incorporated ongoing feedback about the usability, feasibility, and realism of this tool from a representative sample of MSM. We used focus groups and interviews to assist in the development of G-Date, including by identifying relevant stimuli, developing the cover story, and establishing the appropriate study language. To confirm the paradigm’s usefulness, we are conducting an experimental study with web-based and face-to-face participants to determine the content, concurrent, and predictive validities of G-Date. We will evaluate whether certain correlates of SV informed by syndemics and minority stress theories (eg, history of SV and alcohol and drug use) affect the ability of MSM to detect SV risk within G-Date and how paradigm engagement influences behavior in actual DSN app use contexts. Results This study received funding from the National Institute on Alcohol Abuse and Alcoholism on September 10, 2020, and ethics approval on October 19, 2020, and we began app development for aim 1 immediately thereafter. We began data collection for the aim 2 validation study in December 2022. Initial results from the validation study are expected to be available after December 2025. Conclusions We hope that G-Date will enhance our understanding of factors associated with SV risk and serve as a useful step in creating prevention programs for this susceptible population.

1 R15 AA028637-01 3 HDEP ANGELONE, D PUBLIC HEALTH RELEVANCE: Sexual violence (SV) is highly prevalent for men who have sex with men (MSM), and this risk is associated with serious negative health outcomes.The proposed project will use qualitative and quantitative methods to tailor an existing laboratory analogue of SV in heterosexual women to be used with MSM.Subsequently, this paradigm will be subjected to a rigorous validation study to confirm its usefulness in predicting risk for SV, with the potential for use in future prevention endeavors.

CRITIQUE 1
Significance: 3 Investigator(s): 3 Innovation: 4 Approach: 4 Environment: 1 Overall Impact: This R15 proposal seeks to develop a laboratory protocol for education and intervention for sexual violence among MSM.The study team is well qualified and presents a sound rationale for the value of the study.The study design is quite good and builds for a similar approach for SV among cis-gender heterosexual women.The plan to assess feasibility and validate the new app that will be developed is sound.There were moderate weaknesses in some areas.The investigators failed to account for implicit bias in their design, and to present a sound rationale for the exclusion of transmen who have sex with men.Despite these deficits, the overall proposal is strong and includes a good for the inclusion of students.

Significance Strengths
• Sexual violence (SV) is a major public health issue.
• MSM SV is a major issue that we lack true understanding of and have few tools to intervene.
• Syndemic factors may impact MSM ability to see risks of SV.
• Apps and digital technologies are a growing area of SV risk.Weaknesses • While the investigators discuss MSM SV among ethnic/racial minorities, the intersectional nature of MSM SV seems to be underappreciated.

Investigator(s) Strengths
• PI Angelone is a psychologist with prior expertise in sexual violence research.He has prior seed and state funding and has served as a consultant on an R44.• Dr. Fife is a psychologist who will serve as study statistician.Dr. Fife has served in this role on other NIH studies.• Dr. Mitchell is a psychologist with expertise in SV and has collaborated with Dr. Angelone before.Dr. Mitchell has had prior state and federal service and evaluation grants.• Dr. Wells is an expert in sexual violence.Dr. Well has served as a Co-I on an NIH R01 and PI for an R03.• Dr. Kurkovsky brings expertise in mobile app development as a software engineer.

Weaknesses
• The team, despite significant content expertise, does not mention or show experience in mentorship of undergraduates.This is an important component to the R15 mechanism.

Strengths
• The G-Date app will be built on the model created by the EduDate model which has been used in prior research on SV among heterosexual women.This line of work lends innovation to the proposed study as it indexes a successful prior deployment of a related model.
• The creation of a laboratory paradigm has the potential to create a safe way to educate MSM about SV risk and perception.
• The training afford to undergraduates has the potential to be both scientifically salient and socially engaged -there is a high likelihood that this work will draw students into clinicalresearch careers.

Weaknesses
• The research paradigm could be better expanded to capture more about intersectionality.Age, masculinity and race/ethnicity among the presented date models on G-Date could be components of risk perception -measures of implicit bias would be helpful in understanding the experience SV risk perception.

Approach Strengths
• The proposal indicates that the research will begin with app development to mirror the EduDate paradigm with translation to the mobile environment and with adaptations for MSM.
• Participant recruitment will take place locally near the university, in Camden and Philadelphia.
The investigators propose to use LGBT centers, coffee shops and social media.
• Ages 18-30 will be included who are cis-gender males.Focus groups will be done to guide development of G-Date.
• App feasibility will be established in Phase 2 where 194 cis-gender MSM will be recruited to use the mock app.
• Well established scales will be used to examine the various psychological constructs that are relevant to the study.
• 12-month follow-up will be done.

Weaknesses
• Transmen are excluded from the study.It is unclear why transmen who have sex with men would be excluded given the high rates of SV against this subpopulation -a true intersectional framework would allow for the inclusion (and possible oversampling for analysis) of this population.
• The lack of inclusion of a measure of implicit bias among the participants is a major flaw in this study.Participants are apt to perceive risk in highly racialized terms, and capturing implicit bias is essential to the analysis behind this study.

Environment Strengths
• Rowan University appears to be well positioned to support this study.
• The current resources for Dr. Angelone show a commitment to high quality research.
• The student population is well described and fits well with R15 objectives.

Study Timeline Strengths
• None noted by reviewer.

Weaknesses
• None noted by reviewer.

Protections for Human Subjects
Acceptable Risks and/or Adequate Protections  Overall Impact: This innovative application, has a very strong team, highly relevant pilot data, and proposes a rigorous method for evaluating a highly innovative 21 st century paradigm for assessing a very significant problem in MSM (alcohol related dating violence).I am highly enthusiastic about this application.My only minor concern is feasibility of a 12 month follow up in a three-year grant.This is an outstanding study that can and should be done.

Significance Strengths
• Alcohol related dating victimization is a significant public health problem for MSM.
• Tools for assessing this risk are outdated (e.g., laboratory bars).
• Development of an app that assesses this risk is innovative.
• The pilot data presented indicates that they have an app that can be adapted for MSM.
• The premise for this study is strong Weaknesses • None

Investigator(s) Strengths
• The PI has a strong background in this area • The PI has developed the app that will be developed • The team includes consultants who are leaders in this field Weaknesses • None

Innovation Strengths
• This is a very innovative and timely approach to measure risk of dating violence in this population.
• The intensive development to assure that the tool is applicable to different races and ethnicities is important.

Approach Strengths
• The approach is highly rigorous and well described.
• The app modification/development approach is well described.
• The outcome assessment will include sufficient data for a larger trial, including data for up to 12 months on dating violence.

Weaknesses
• The 12-month follow up is a strength but in a three-year grant, it may not be feasible.

Environment Strengths
• The team has developed the resources needed to carry out the project.

Weaknesses
• None

Study Timeline Strengths
• Appropriate Weaknesses • Might be challenging to get 12 month follow up in 3 years (Angelone) has expertise in the area of sexual violence and has worked extensively with undergraduate and graduate students.Additional investigators and consultants provide content and research management experience.The application is innovative in its target population focus and in the extent to which students will be involved.Weaknesses in the approach include the lack of adequate justification for several methods components including the 12-month post-test interval, the extensive pre-post tools to be completed by Aim 2 participants, This project has the potential to impact sexual violence against MSM and the team has the ability to conduct the study.

Significance Strengths
• Sexual violence among MSM is a critical problem.
• Dating and sexual networking (DSN) apps were found to increase identification of risky behaviors for heterosexual women.• The development of an app for heterosexual women measures threat appraisal via response latency (length of time to determine risk perception) -this will be adapted for the G-Date to be developed for MSM.• The scientific premise is based on syndetic, response latency, and minority stress constructs.

Weaknesses
• The application did not adequately address if the project would strengthen the research environment of the institution.• The scientific premise for the 12-month follow-up is not well supported.Preliminary data reported for the EduDate app did not address 12-month predictive validity testing nor is there preliminary data related to the extensive number of data collection tools proposed in Aim 2.

Investigator(s) Strengths
• The PI (Angelone) and Co-I are psychologists who have previous collaborations in the area of sexual violence research and in treating victims of sexual violence.• Other Co-I's include Dr. Well who has conducted sexual assault research and has experience recruiting sexual minorities.Dr. Kurkovsky has expertise in developing mobile apps and software engineering and has worked with students on these efforts.Statistical support is provided by Dr. Fife.• Consultants provide additional support.Dr. George has experience working with minorities in the areas of alcohol, sexual risk and sexual violence.Dr. Davis has expertise in sexual-decision making.Dr. Pantalone has expertise in sexual victimization drug abuse, and sexual risk behaviors.Sr. Sitron has expertise in MSM and the use of dating apps.These consultants will be available to support and provide guidance to the investigators.• Dr. Angelone completed a sabbatical to explore the interaction of alcohol use and expectancies with sexual violence.Dr. Angelone has extensive experience working with undergraduate students in his research lab.

Weaknesses
• None noted

Innovation Strengths
• Development of an app to understand factors related to sexual violence in MSM is an innovative approach.The app allows for real time risk perception assessment.• Students will be involved in all aspects of the development and testing of the app.Including conducting focus groups and analysis of data.An innovative aspect is the context and the opportunity students will have to work with a vulnerable, minority population.

Weaknesses
• Given the extensive number of apps available, developing another app is no longer an innovative approach.

Approach Strengths
• Students will create the G-Date app with supervision from the computer science Co-I (Kurkovsky).• The development of 2 'chats' one with a 'risky' candidate, and one with a 'safe' candidate to allow assessment of risk perception and response latency.Data will be collected on number of responses in the risky vs safe conditions, and changes in ratings on desirability and interest in risky dating candidates -allowing for multiple means to assess risk perceptions.

Weaknesses
• Aim 1 participation is limited to focus groups, the rationale for not including interviews as an option for those not able or willing to participate in a group meeting was not described.• Rationale for 5 focus groups not provided, the application lacks adequate discussion related to data saturation.• Aim 1 data analysis is under-developed and lacks specifics related to student involvement.
• Neither the rational for, nor the psychometric properties of the extensive number of tools proposed in Aim 2 are discussed.• Rationale for a 12-month follow-up is not provided, nor are retention methods described.

Environment Strengths
• Rowan University has recently been designated as a research institution by the State of New Jersey.• The pool of undergraduate students will be drawn from the 16,120 student body.Efforts will be made to include diverse and underrepresented students in the project.• Dr. Angelone's lab areas are adequate to conduct all phases of the study.
• Adequate evidence was provided that student's at Rowan university are likely to pursue careers in health-related sciences Weaknesses • The environment at Central Connecticut University where the development of the app will occur was not addressed.

Study Timeline Strengths
• None noted by reviewer.

Weaknesses
• None noted by reviewer.Footnotes for 1 R15 AA028637-01; PI Name: Angelone, David Jason NIH has modified its policy regarding the receipt of resubmissions (amended applications).See Guide Notice NOT-OD-18-197 at https://grants.nih.gov/grants/guide/notice-files/NOT-OD-18-197.html.The impact/priority score is calculated after discussion of an application by averaging the overall scores (1-9) given by all voting reviewers on the committee and multiplying by 10.The criterion scores are submitted prior to the meeting by the individual reviewers assigned to an application, and are not discussed specifically at the review meeting or calculated into the overall impact score.Some applications also receive a percentile ranking.For details on the review process, see http://grants.nih.gov/grants/peer_review_process.htm#scoring.

•
The investigators have good plans in place for human subject protections.For NIH-Defined Phase III trials, Plans for valid design and analysis: Not applicable • Inclusion/Exclusion Based on Age: Distribution justified scientifically

•
The exclusion of trans-males is not justified scientifically.

Authentication of Key Biological and/or Chemical Resources
For NIH-Defined Phase III trials, Plans for valid design and analysis: Not applicable • Inclusion/Exclusion Based on Age: Distribution justified scientifically • All justified.
violence against men who have sex with men (MSM).The focus of the application is on the development and testing of a laboratory paradigm app to investigate sexual violence risk perception in MSM.The app is based on a similar product developed for heterosexual women (EduDate) and will be modified with a well-developed adaptation plan.The strong research team and the supportive environment at Rowan University lends support to the ability to accomplish the study aims.The PI • Student involvement in Aim 1 include a graduate student research coordinate and 4 paid undergraduate research assistants.Students will receive training in the content area as well as in research ethics and methods.• A strength is the assessment of feasibility, usability, and acceptability of the app with 40 additional participants in Aim 1. • Aim 2 assessment of content, concurrent, and predictive validity assures the app is well developed and has educational value.The methods are adequately described.• Aim 2 is adequately powered.

THE FOLLOWING SECTIONS WERE PREPARED BY THE SCIENTIFIC REVIEW OFFICER TO SUMMARIZE THE OUTCOME OF DISCUSSIONS OF THE REVIEW COMMITTEE, OR REVIEWERS' WRITTEN CRITIQUES, ON THE FOLLOWING ISSUES: PROTECTION OF HUMAN SUBJECTS: ACCEPTABLE INCLUSION OF WOMEN PLAN: UNACCEPTABLE
Faculty at Rowan University are on 10-month contracts, it is unclear if effort includes summer salary.Reviewers noted the exclusion of transmen or transmen in transition is not scientifically justified.The scientific rationale for the distribution of transmen in the sample should be clearly described.INCLUSION