Investigating Patient Use and Experience of Online Appointment Booking in Primary Care: a Mixed Methods Study.

Background: Online appointment booking is a commonly used tool in several industries. There is limited evidence about the benefits and challenges of using online appointment booking in healthcare settings. Potential benefits include convenience and the ability to track appointments, though some groups of patients may find it harder to engage with online appointment booking. We sought to understand how patients in England used and experienced online appointment booking. Objective: To describe and compare the characteristics of patients in relation to their use of online appointment booking in general practice and to investigate patients’ views regarding online appointment booking arrangements. Methods: This was a mixed methods study set in English general practice comprising a retrospective analysis of the general practice patient survey (GPPS) and semi-structured interviews with patients. Data used in the retrospective analysis comprised responses to the 2018 and 2019 GPPS analysed using mixed-effects logistic regression. Semi-structured interviews with purposively sampled patients from 11 general practices in England explored experiences of, and views regarding, online appointment booking. Framework analysis was used to allow comparison with the findings of the retrospective analysis. Results: The retrospective analysis included 1,327,693 GPPS responders (2018/2019 combined). Interviews were with 43 patients with a variety of experiences and awareness of online appointment booking. In the retrospective analysis, substantially more patients were aware that online appointment booking was available (45%, 581,224/1,228,341) than had experience of having used it (16%, 203,184/1,301,694). Deprivation gradients for awareness and use were evident, and there was a substantial decline in both awareness and use in patients over 75 years old. For interview participants, age and life stage were key factors


Preprint Settings
1) Would you like to publish your submitted manuscript as preprint?Please make my preprint PDF available to anyone at any time (recommended).
Please make my preprint PDF available only to logged-in users; I understand that my title and abstract will remain visible to all users.Only make the preprint title and abstract visible.
No, I do not wish to publish my submitted manuscript as a preprint.2) If accepted for publication in a JMIR journal, would you like the PDF to be visible to the public?
Yes, please make my accepted manuscript PDF available to anyone at any time (Recommended).
Yes, but please make my accepted manuscript PDF available only to logged-in users; I understand that the title and abstract will remain v Yes, but only make the title and abstract visible (see Important note, above).I understand that if I later pay to participate in <a href="http

Original Manuscript
https://preprints.jmir.org/preprint/51931[unpublished, peer-reviewed preprint] Investigating Patient Use and Experience of Online Appointment Booking in Primary Care: a Mixed Methods Study.Abstract Background Online appointment booking is a commonly used tool in several industries.There is limited evidence about the benefits and challenges of using online appointment booking in healthcare settings.Potential benefits include convenience and the ability to track appointments, though some groups of patients may find it harder to engage with online appointment booking.We sought to understand how patients in England used and experienced online appointment booking.

Objective
To describe and compare the characteristics of patients in relation to their use of online appointment booking in general practice and to investigate patients' views regarding online appointment booking arrangements.

Methods
This was a mixed methods study set in English general practice comprising a retrospective analysis of the general practice patient survey (GPPS) and semi-structured interviews with patients.Data used in the retrospective analysis comprised responses to the 2018 and 2019 GPPS analysed using mixedeffects logistic regression.Semi-structured interviews with purposively sampled patients from 11 general practices in England explored experiences of, and views regarding, online appointment booking.Framework analysis was used to allow comparison with the findings of the retrospective analysis.

Results
The retrospective analysis included 1,327,693 GPPS responders (2018/2019 combined).We conducted 43 interviews with patients with a variety of experiences and awareness of online appointment booking, of these patients 6 were from minority ethnic groups.In the retrospective analysis, more patients were aware that online appointment booking was available (45%, 581,224/1,228,341) than had experience of using it (16%, 203,184/1,301,694).There were deprivation gradients for awareness and use, and a substantial decline in both awareness and use in patients over 75 years old.For interview participants, age and life stage were factors influencing experiences and perceptions, working patients valued convenience, but older patients preferred to use the telephone.Patients with long-term conditions were more aware of (OR 1.43, 95% CI 1.41 -1.44), and more likely to use (OR 1.65, 95% CI 1.63-1.67),online appointment booking.Interview participants with long term conditions described it useful for booking routine non-urgent appointments.Patients in deprived areas were clustered in practices with low awareness and use of online appointment booking amongst GPPS patient respondents (OR for use 0.65 95%CI 0.64-0.67).
Other key findings included the influence of the availability of appointments online and differences in the registration process for accessing online booking.

Conclusions
Whether and how patients engage with online appointment booking is influenced by the practice

Introduction
Booking an appointment or service online is widespread internationally with most sectors, including travel, entertainment and hospitality, offering this facility.Healthcare has adopted online appointment booking, with varying levels of patient uptake and engagement [1][2][3].In England, most primary care is delivered by National Health Service (NHS) general practices to registered lists of patients.All NHS general practices must make appointments available for online booking [4,5], with appointments booked via web platform or app.This is part of the contract that general practices have with NHS England, the body responsible for leading the NHS in England.Despite the availability of online appointment booking being mandatory since 2012, uptake by patients has been slow and variable, with national level data showing that in March 2023 43% of patients are registered to use an online appointment booking service [6].
The use of online appointment booking is intended to lead to reduction in reception staff workload and to increased patient satisfaction.It can offer flexibility, convenience and time saving [7][8][9][10][11][12][13] and allow carers of older adults to make and track healthcare appointments [14].It may reduce the likelihood of patients missing appointments [11].It is known that people who use online appointment booking tend to be female and educated to degree level, and more frequent users of the internet [9,10,15,16]; utilisation is lower in non-white patients, in lower socioeconomic groups and in those with poorer health [9,10,15,16].Patients from medically underserved and vulnerable populations are less likely than other patient groups to access and use online health technologies [17].Specific barriers to engagement with online services for these groups include: a lack of experience, knowledge and skills when using the internet [17][18][19], lack of technical support and lower health literacy [17,20].
The routine use of online services in general practice settings, such as online triage platforms and video consultation, [21][22][23] accelerated during the Covid-19 pandemic.Understanding who benefits and who does not benefit from the use of online appointment booking and why, is important given the potential for inequality of access to primary care services to be exacerbated when services move online [19].As an online service with a defined purpose and broad geographic availability, online appointment booking potentially provides a marker for how patients may engage with the wider range of online services within modern primary care.Evidence which seeks to understand how patients interact with online healthcare services is needed to shape such services to suit patients and their needs as we adjust to a 'new normal' for primary care delivery.With levels of uptake of online appointment booking low in general practice, and a lack of evidence on why this is the case, we conducted this mixed methods study.We sought to describe and compare the characteristics of patients in relation to their use of online appointment booking, to explore how patients' use relates to their experience of care, and to investigate patients' views regarding online appointment booking arrangements in general practice.

Methods
We conducted a retrospective analysis of data about awareness and use of online appointment booking from the annual national General Practice Patient Survey conducted in English general practices.This was complemented by a semi-structured interview study with patients from a range of general practices in England with differing levels of uptake of online appointment booking.The interviews were conducted both before and during the Covid-19 pandemic.
In UK general practice patients can book an appointment online directly with the general practice they are registered with.Patients can do this using the NHS website/app, and/or via a web-platform/app provided by their general practice.General practices use a web-platform/app of their own choosing to offer online appointment booking and there are multiple suppliers of these systems in the UK market.Patients can use these services via an internet enabled device (e.g.smart phone, tablet or computer) according to their preference.These systems are not necessarily linked into the electronic health record which is also chosen by the general practice from a range of available systems.This study examines online appointment booking across all platform/app types and we did not set out to compare platforms as all offer online appointment booking in line with NHS requirements.

Data source
We conducted secondary analyses of data from the English General Practice Patient Survey (GPPS) [24].The GPPS is a postal and online survey of patients' experience of primary care in England.The survey is sent annually (January-March) to around 2.2 million adult patients with the findings published each July [25].A stratified sample of patients aged >16years is drawn from the practice list of each general practice in England.Patients from practices known from prior surveys to have low response rates are oversampled to ensure an adequate number of responses per practice.We used data from the 2018 and 2019 surveys making use of two questions, one asking about awareness of online services in the respondent's general practice and another asking about their use of online services in the past 12 months.These were used in conjunction with data from the survey on demographic characteristics and health status [26].

Analysis
Descriptive analyses and logistic regression were used to investigate the percentage of patients who reported awareness of their general practice offering online appointment booking of appointments and recent use of online appointment booking of appointments.These analyses were restricted to patients reporting trying to make an appointment at their general practice in the last 12 months.This is important because patients may not have used online appointment booking (or been aware of the option) due to not needing an appointment.
Multivariate logistic regression was used to examine associations between both awareness and use of online appointment booking in relation to age, gender, ethnicity, deprivation (based on the Index of Multiple Deprivation [27] corresponding to the respondent's postcode of residence), the presence of a long-term condition, long-term sickness and being deaf.
Comparison of models accounting for, and not accounting for, clustering by practice (using a random effect) was used to illustrate the extent to which disparities reflect the clustering of certain types of patients in practices where awareness and use of online appointment booking is high or low for all patient groups.The random effect from these models was used to quantify the variability in the odds of patients being aware of or using online appointment booking between practices.
Further models augmented those described above to include other GPPS report and evaluation items (as predictors of online appointment booking awareness and use) around how easy practice websites were to use, whether respondents had a preferred general practitioner (GP), how helpful receptionists were, ease of getting through on the phone, and use of online prescription ordering.A final set of regression models examined the extent to which awareness and use of online appointment booking was associated with patients' experiences of making an appointment, choice of appointment, ability to see their preferred GP and their overall experience.These models adjusted for the same patient factors included in the previous models as well as including a random effect for practice.
Full statistical details are provided in multimedia appendix 1: [Detailed statistical methods].Analysis was conducted using Stata version 12.

Sampling & recruitment
Interview participants were recruited from 11 general practices selected for maximum variation in levels of online booking; list size; rurality; deprivation; ethnic diversity.There were three recruitment waves: 1) a scoping survey was distributed to patients to gather data on their demographic characteristics (gender; age; education; employment status; health status; ethnicity), booking behaviours and awareness of online booking, and willingness to participate in an interview 2) interview volunteers were purposively sampled according to background and booking behaviour and invited to take part in an interview, 3) finally, aiming to fill demographic gaps within the sample, potential participants were identified based on their background via GP practices and invited directly to interview.Patients who were under 18, at end of life, lacked capacity to provide informed consent or had not recently booked a GP appointment, were excluded from participation.The Covid-19 pandemic meant recruitment was paused as the funder made the decision to halt all non-Covid related research activity, allowing practices to focus on their Covid-19 response.Recruitment was resumed in June 2020 at which time we made use of direct invites to interviews (wave 3 detailed above) to make up the remainder of the sample.

Data collection
43 interviews were carried out via the telephone, audio-recorded, transcribed, and anonymised.
During the interviews we explored in depth patients' experiences of, and views about, booking appointments online including how they did this, and for those who had not used it, whether they knew about it or not.The topic guide can be found in Multimedia appendix 2: [Interview topic guide].

Data analysis
We used theframework method to conduct a comparative thematic analysis within the interview data relevant to the key findings from the retrospective analysis of GPPS data The framework method is commonly used in health research where research questions are relevant to policy.It has five key stages: (1) Familiarisation (2) Identifying a thematic framework (3) Indexing ( 4) Charting ( 5) Mapping and Interpretation [28].A matrix was used with 'cases' arranged along one axis and 'codes' along the other.This allowed data to be organised systematically to compare differing experiences and perspectives on key areas from participants' accounts.NVivo 12 software was used to support this process.

Patient Public Involvement (PPI)
We involved members of the public at various stages of the study.A lay co-applicant joined the study team and attended meetings, provided advice throughout, and helped facilitate workshops.Public workshops were held to refine our protocol (before funding was awarded), gain input on the participant-facing documentation and discuss the implications of the study findings.We piloted the scoping survey and telephone interviews with three contributors.Two contributors also provided their interpretation of the qualitative data.

Ethics approval
We obtained National Health Service (NHS) research ethical approval and HRA approval for the semi-structured interview study from the West Midlands-Solihull Research Ethics Committee reference, study reference: 19/WM/0272.The secondary analysis of GPPS data does not require any ethical approvals or Health Research Authority (HRA) approvals.

Retrospective Analysis of General Practice Patient Survey
Of the 1,327,693 GPPS responders (2018/2019 combined) who reported attempting to make an appointment within the previous 12 months, 45% (581,224/1,288,341) were aware that their practice offered online appointment booking and 16% (203,184/1,301,694) reported using online appointment booking.We found very strong evidence (P<.001) of associations between both awareness and use of online appointment booking for all variables considered in both unadjusted and adjusted models except for the effect of rurality in the adjusted model including a random effect for practice when considering awareness (Tables 1 and 2).Percentage of responders aware of online booking.e.
Frequency numbers are weighted and rounded.Regression results are not weighted.f. p-values from joint Wald test were computed for the adjusted and unadjusted models.All p-values are <.0001, except the corresponding to rurality in the mixed effects logistic regression model.Percentage of responders aware of online booking.e.
Frequency numbers are weighted and rounded.Regression results are not weighted.f. p-values from joint Wald test were computed for the adjusted and unadjusted models.All p-values are <.0001.
There were only small differences between the unadjusted and adjusted models, with consistent direction of association except for awareness amongst patients aged 25-44, and for rural vs urban location (Tables 1 and 2).Findings from the adjusted model included a random effect for practice and so can be interpreted as differences experienced by patients registered with the same general practice.

Age
There was little variability in both awareness and use among under 65-year-olds, but a substantial drop in both for those aged over 75 (OR for awareness, 85+ years old vs. 65-74 years =0.34, 95%CI 0.33-0.37)(Tables 1 and 2, Figure 1).

Deprivation status
There was a strong deprivation gradient, with more deprived patients being less likely to use or to be aware of, online appointment booking (OR for use, most vs. least deprived =0.65 95%CI 0.64-0.67)(Tables 1 and 2, Figure 2).

Ethnicity
A more complex relationship was seen for ethnicity.All minority ethnic groups were less likely to be aware of online appointment booking than White patients.However, only patients in the Black and Other ethnic groups were less likely to use online appointment booking than White patients, and Mixed and Asian patients were somewhat more likely to use online appointment booking (Tables 1  and 2, Figure 3).

Location
Patients living in rural areas were less likely to be aware of, or to use, online appointment booking than patients living in urban areas (OR for use, rural vs. urban = 0.92, 95% CI 0.88 -0.97) (Tables 1  and 2, Figure 4).

Health status
Permanently sick/disabled patients were less likely to be aware of, or use, online appointment booking than those living in urban areas and non-sick/disabled patients (OR for awareness, sick/disabled vs. not sick/disabled = 0.80, 95% CI 0.78-0.82(Tables 1 and 2, Figure 4).
Patients with long term conditions were more likely to be aware of, and use, online appointment booking than patients without them (OR for awareness, long term condition vs. no long term condition = 1.43, 95% CI 1.41 -1.44) (Tables 1 and 2, Figure 4).Interestingly, deaf patients who use sign language were less likely to be aware but more likely to use online appointment booking than those without such an impairment (OR for use, deaf vs. not deaf = 1.21, 95% CI 1.11-1.32).

Year
Comparing

Practice
When comparing models that did and did not account for the patient's registered practice, the patient groups that were less likely to use/be aware of online appointment booking remained the same between models (except for rural patients' awareness, where the effect was small in both cases).The strongest factor in determining awareness and use of online appointment booking is the practice the patient is registered with.For all factors, apart from deprivation, the changes in OR magnitudes were small (Tables 1 and 2).The deprivation gradient in awareness and use of online appointment booking was noticeably stronger in models that did not incorporate adjustment for practice, which implies that patients living in deprived areas are more likely to be registered at practices where awareness and use is low for all patients (Tables 1 and 2).
The OR covering 95% of practices (comparing the highest with the lowest) estimated from the random effects models was 18.1 for awareness and 16.7 for use.These are considerably larger than the odds ratios associated with any patient factor suggesting that the practice a patient was registered with was the strongest predictor of awareness and use of online appointment booking when compared with a wide range of other potentially important variables.

Patient experience
Patients reporting their practice website being easy to use were far more likely to be aware of, and to use, online appointment booking (OR for use 'Very easy to use' vs. 'Not easy at all' =6.34 95%CI 6.06-6.64).(Table 3) Having a preferred GP was associated with somewhat higher awareness and use, while ease of getting through to a practice on the phone was associated with lower use and somewhat lower awareness.Patients who report finding practice receptionists helpful were more likely to be aware of online booking, but less likely to use it.Patients who have ordered repeat prescriptions online were also more likely to be aware of, and to use, online appointment booking than those who had not (OR 3.58 95%CI 3.50-3.62).
Table 3.Odds ratio for awareness and use of online booking adjusted for demographic factors, long term condition, occupation, deafness, period and for clustering of patients within practices.A final set of models (Table 4) showed that, regardless of whether patients had tried online booking, those who were aware of it were more likely than patients who were not aware to have a positive overall experience of their GP practice, to see or speak with their preferred GP, to be offered a choice of appointment or to report a positive overall experience of making an appointment.Furthermore, patients who were aware of, but who had not used online appointment booking, were more likely than unaware patients to be satisfied with their appointment type.Surprisingly, patients who were aware of and used online appointment booking were less likely than those patients who were not aware to be satisfied with their appointment type.

Semi-structured Interview Study
We conducted 43 semi-structured qualitative interviews with individuals who had recently booked general practice appointments, (see Tables 5 and 6).Participants were aged 18 to 85+ and had a range of educational and employment status.Twenty participants reported having at least one long term condition.Eighteen participants had tried online appointment booking, 16 were aware that they could access online appointment booking but had not tried it, and 9 were unaware that online appointment booking was available.The framework analysis identified three key areas where there was comprehensive qualitative evidence that matched the data identified in the quantitative element of the study.These were a) general practice mediated factors, b) impact of age and life stage and c) specific experiences of those with a long-term condition.Patients also shared information about their differing levels of confidence and experience using online services in general.
a. General practice mediated factors How the general practice organised online appointment booking systems at the general practice influenced patient experience.
Patients had varied awareness and understanding of the online appointment booking appointment system available to them.Routes to learn about online appointment booking included adverts on noticeboards, text messages, letters or being told directly by practice staff.Despite all participating practices having online appointment booking available, many participants were not aware of it and described not having seen or heard any information about it.

"The first time I went the reception, what do you call it the receptionist? She turned round and said, "Well, you can also book online." [Male, 55-64yrs, Practice 4, White British]
"I mean, you could be on hold for up to half an hour trying to call through.And the thing is, it's never been advertised to us that you could book online.Like, I've never seen it anywhere or been informed in any communication that you can do it online, so that's why that's always been my way of booking it" [Male, 25-34yrs, Practice 4, White British] The practices varied in the processes that patients described having to follow to register for online appointment booking.The most common requirement was to attend the practice in person to collect a form to complete and then return with identification before they could be registered.In contrast other participants stated their practice had not required them to show any identification.
Providing identification required the patient to be proactive in seeking registration.There were several examples of participants' describing an intention to register and then not completing the process.

"I got as far as bringing, bringing the form home and filling it in. But then because I don't go past the surgery on a daily basis I never took it, I never took it back." [Male, 55-64yrs, Practice 7, White
British] There was also variation between general practices in the appointments made available to book online, with some patients describing that their practice appeared to make very few appointments available.A lack of appointments that could be booked urgently or soon was experienced as a key barrier to use where patients needed this kind of appointment.Overall, online appointment booking was seen as fitting into a modern society, which relies heavily on online services.
"It's probably very, very good for some people, but it's not for me.I appreciate that we are in a digital age and we, you know, we're relying on these, the computer more, and online, and e-mails and stuff, but no, I'm, I really don't think it is an advantage in some cases.It's not for me."[Female,

65-74yrs, Practice 10, White Irish]
Patients, including those who did not use it, were accepting of its value in the context of it being available as one of several options for accessing an appointment in general practice.Across the sample, online appointment booking was seen as a tool that was a good fit for some patients in some circumstances.

"Well, I think in this day and age most people can do it online […]
I think a range of all three, I mean, going forward I suppose with staffing and stuff, you know, they're obviously looking to minimise the face-to-face contact, but I still think there's a, a need for that at, at some, some level for everybody."[Female, 55-64yrs, Practice 12, White British] b.Age & life stage The convenience of online appointment booking was particularly appealing for those in the workforce.Booking an appointment via telephone could be problematic due to having to call while at work or the lack of privacy in an employment setting.Online appointment booking was advantageous when childcare or caring responsibilities made it difficult for patients to call the general practice when telephone lines opened in the morning.
"When she was at primary school and my eldest was at primary school, I wouldn't, I wouldn't be able to ring at half eight because I'd be in the car, or I'd be stood in the playground.You know, it, it doesn't work."[Female, 45-54yrs, Practice 11, White British] Some patients regarded their age as leading to a reluctance to use online appointment booking.Online appointment booking was regarded as something new to learn that was unfamiliar.Some participants did not use the internet regularly, did not use smartphones to access such services, and saw switching on a computer as inconvenient.
"People that are younger than me and just do this automatically online, you know, whereas I don't do it automatically" [Female, 75-84yrs, Practice 12, White British] Requiring help to use online appointment booking was seen as disempowering.Older patients were accustomed to booking an appointment using the analogue telephone and did not explore the possibility of using another route.Sometimes, using online appointment booking meant relying on family members or a partner to do the 'internet admin' In whatever format this might take.

c. Differing experiences of those with a long-term condition
Patients in the sample who had experience of using online appointment booking and had long term conditions found the service was a good fit for them.Participants with long term conditions that used online appointment booking said they had only become aware of, and subsequently used, online appointment booking when accessing the online system to obtain their repeat prescriptions for their long-term condition."I actually first used the app to get a repeat prescription […] Then I noticed that you could also book appointments on there" [Female, 25-34yrs, Practice 4, White British] Accessing repeat prescriptions provided a gateway to use of other online services including online appointment booking and this was also demonstrated in the quantitative findings.
Patients with long term conditions were used to booking pre-planned routine appointments for the management of their condition.Online appointment booking allowed them to do this at a time that was convenient and in a way that was convenient, be this via smartphone or computer.
"It was for a regular, you know, prescription update thing, so it wasn't like I needed it on the day, it was something I could plan."[Female, 35-44yrs, Practice 1, White British] This reflected the non-urgent nature of the appointments they were booking which fitted with how online appointment booking was being offered at their practice, where appointments could be more readily booked some time ahead than on the day.

Principal results
Awareness of online appointment booking is much higher than use, though it is still under half of patients are aware of online appointment booking.Patients, including those who did not use it, were accepting of its value as a convenient option for accessing an appointment.
There was little variability in awareness and use amongst under 65-year-olds but a substantial decline in those over 75 years.Age and life stage were key factors influencing experiences and perceptions.
Working patients and those with caring responsibilities found it particularly convenient relative to the telephone, but older patients in the interview sample preferred to use the telephone, being familiar and not requiring the help of others.
There was a strong deprivation gradient with deprived patients being less likely to use or to be aware of online appointment booking.Findings relating to ethnicity were more complex with all minority groups less likely to be aware of online appointment booking than white ethnic groups.While patients in 'black' or 'other' ethnic groups were less likely to use online appointment booking than those in white ethnic groups, and those in 'mixed' or 'Asian' ethnic groups were more likely to have.Location was influential with patients in rural areas less likely to be aware of or use online appointment booking.Whilst the interview study sampled patients from practices with a range of Index of Multiple Deprivation scores (55% of practices were below average IMD), from different ethnic groups and from a range of rural and urban areas we did not observe any findings relating directly to these factors.
Health status was a key factor.Those permanently sick or disabled were less likely to be aware of or use online appointment booking.Patients with long term conditions found that it was a good fit for them, being useful for booking routine non-urgent appointments for their condition.The only group where there was disparity between awareness and use was in deaf patients who use sign language, where they were less likely to be aware but more likely to use online appointment booking.This was the only group where there was such disparity between awareness and use (compared with the wider population), suggesting that although deaf people may find online appointment booking particularly useful, they are unlikely to be aware that it is an option.
The strongest factor in determining awareness and use of online appointment booking across the study was the general practice the patient was registered with.The influence of general practice mediated factors on patient experience was a key finding in the interview study.There was variation in how general practices organised their appointments and the registration process required for online appointment booking, both of which impacted on experience.
Patient experience was a key focus of this study and impacted on awareness and use of online appointment booking.Patients who found the practice website easy to use, patients with a preferred GP and patients who have ordered repeat prescriptions online were more likely to be aware of and use online appointment booking.Patients reported that accessing repeat prescriptions was their gateway to use of other online services including online appointment booking.Patients reporting it was easier to get through to the general practice on the phone had lower use and somewhat lower awareness and patients reporting receptionists as helpful were more likely to be aware of but less likely to use online appointment booking.Regardless of whether they had tried it, patients who were aware of online appointment booking were more likely to have had a positive experience with their general practice when arranging and obtaining an appointment than those who were not aware.

Limitations
Our study was conducted on data from the 2018/2019 GPPS and much has changed in healthcare delivery since.However, the 2022 general practice patient survey showed that the numbers of patients booking appointments online had only risen to 21% representing a change of just 5% over the period since 2018/2019.The 2022 GPPS also asked how patients chose to book the appointment and just 16% of patients booked online.
The GPPS is a large national survey and includes responses from patients registered with every general practice in England and provides a high generalisability of findings.Like similar surveys the GPPS has a relatively modest response rate (34% in 2018 and 33% in 2019).Whilst low response rates may lead to substantial biases in absolute percentages, they are less likely to result in biased estimates of associations [29].
The interview study was disrupted by the Covid-19 pandemic.Recruitment was paused for 3 months when the funder made the decision to halt all non-Covid 19 related research activity, allowing general practices to focus on their Covid-19 response.Interviews were conducted via the telephone and therefore we could interview participants already recruited prior to mid-March 2020 and adhere to social distancing restrictions.Recruitment was resumed in June 2020, and we revised our approach (as outlined in the methods section), using a purposive approach to ensure that our sample was as diverse as possible.
Online appointment booking was unavailable to patients from the onset of the Covid-19 pandemic and general practices used telephone and online triage to control the availability of face-to-face appointments.Online appointment booking resumed later in 2020 and has been available since.This may have influenced the responses to the interviews that were conducted after the onset of the Covid-19 pandemic though we were not able to identify anything specific relating to this.We aimed for a maximally variable sample in the interview study.This allowed us to provide qualitative findings contextualising the quantitative results according to age and presence of longterm conditions.However, the proportion of participants in minority ethnic groups was slightly lower than for the population in England & Wales in the 2021 census (14% in our sample versus 18.3% in England & Wales) and was not reflective of the populations of the included practices in all cases.Under-recruitment to health research studies of people in ethnic minority groups is a longstanding concern within the research community and can be attributed to researchers not facilitating access to research studies, with rapport with participants being a key factor in increasing recruitment [30,31].This was a challenge during the Covid-19 pandemic.Further exploration of the views and experience of patients in ethnic minority groups is needed.Our interview data did not provide insight into the impact of deprivation as identified in the retrospective analysis.This may be due to our sample, with a smaller number of interview participants having lower levels of education (a proxy for deprivation levels).The interview participants were not sampled for location (rural/urban) as we did not realise the significance of this distinction ahead of conducting the study.Future studies should consider these factors.
Our study was conducted in the UK where there is a national health service, and healthcare is free at the point of use.Findings may differ in countries where health systems differ, particularly those where primary care is accessed on a fee for service basis.

Comparison with Prior Work
Our study included online appointment booking via any online interface for a general practice appointment.This included the NHS App where appointments can be booked.The NHS App had 18 million registrations during the Covid-19 pandemic due to its use for vaccination passports [32] A study examined use of the NHS App between January 2019 and May 2021.Those researchers examined total numbers of downloads, registrations, prescriptions ordered, records access and appointment booking.Using time series analysis they calculated that 12 months after the first UK Covid-19 lockdown there were 21, 606 fewer appointments booked online than might have been expected if the Covid-19 lockdown had not occurred [33] indicating that the Covid-19 pandemic had a detrimental effect on use of online appointment booking.
A US observational study examined a patient portal [34] offering online appointment booking found that use of the portal was low, and use was lowest in persons of lower socioeconomic status and without broadband internet.Another observational study in the US examined the use of patient portal functions by patients, including appointment booking [35].As in our study, the study demonstrated that use of the appointment booking function decreased with age with the sharpest decrease over age 65.A cross-sectional survey of general practice patients in the West Midlands, England, looked at use of online appointment booking services pre-pandemic [36].The study observed similar associations to our study between awareness and use and deprivation level, and between awareness and use and long-term conditions.Our study reinforces these findings as well as adding explanations behind some on the patterns identified within the qualitative findings.Patterns relating to age and socio-economic status, as well as ethnicity, which we could say less about, have been identified as structural barriers to the equitable use of digital health by those examining the political economy of digital health equity [37].This has implications for those designing and devising online services in healthcare.

Future Research
Experience with the general practice was a key factor in awareness and use of online appointment booking and this warrants further research into exactly how practices can best enable patients to access and use online services, and the different types of support that may be needed for different patient groups.General practices in England offer online services through portals which are provided by various third-party companies.Alongside these portals there is the NHS App.Future research should explore how the different interfaces impact on experience and use of online services and what that means for uptake.A survey exploring patient views on what supports uptake and use of web-based services in primary care found that there were varied factors impacting on their use of these services including poor design of online interfaces and lack of support from the general practice along with personal factors such as preference for human interaction [38].
Older patients use online appointment booking less, and in some cases found it difficult to use online services or chose not to when the telephone was available and allowed them to book an appointment.
Services may wish to focus on supporting those older people who do want to use online services, whilst maintaining a telephone service for those who cannot or do not wish to use online routes.
Research has demonstrated that understanding the characteristics of general practitioners, including their personalities, can influence the digital maturity levels in a given general practice, and so examining the general practitioners in a practice may contribute to a deeper understanding of how and why patients choose to use, or not, online services [39].
Our qualitative interviews showed that using an online repeat prescription service was a gateway to use of other online services.Researchers, and those designing services may wish to explore why this service is relatively highly accessed when compared to other online services to determine what makes an online service usable.In the case of online appointment booking, accessing repeat prescriptions raised awareness of online appointment booking services and we have demonstrated that most patients are not aware of this service.Our quantitative findings identified that deaf patients who use sign language may find online appointment booking particularly useful, but they are unlikely to be aware that it is an option.We did not recruit any deaf patients to the interview study and their views are likely to be of importance in future research.We identified a clear deprivation gradient, and this may be exacerbating inequitable access for patients.The risk of excluding patients who cannot access online services should be a key consideration in the design of the wider access systems within general practice and a key factor examined in future research.To ensure equitable access, it is important to understand how particular groups (patients who are older, more deprived, from minority ethnic groups) who were shown to be less likely to be aware of and use online booking, can be supported should they wish to use these services and not disadvantaged should they choose not to.
Patient awareness of online appointment booking may be influenced by the actions of the general practice and how they promote these services.A systematic review examining how primary care supports patients to become aware of and access online services highlighted the lack of evidence in this area but concluded that existing evidence indicated that there was the potential for such support to increase awareness and use of services, with more research needed to understand what this support would look like [40].

Conclusions
Awareness and use of online appointment booking by patients in general practice is associated with characteristics of the practice they attend, their age, the level of deprivation associated with their general practice and whether they have a long-term condition.There are clear contextual explanations behind why online booking is more appealing for those with long term conditions and those aged under 75.Other factors, such as why deprivation and/or ethnicity mediates use of online booking remains less understood.The findings have implications for patient engagement with the wider range of online services offered in general practice, how these are delivered, and for future research in this area.

Figure 1 .
Figure 1.Age and awareness, and use, of online appointment booking.

Figure 2 .
Figure 2. Deprivation and awareness, and use, of online appointment booking.

Figure 3 .
Figure 3. Ethnicity and awareness, and use, of online appointment booking.

Figure 4 .
Figure 4. Location and Long term conditions and awareness, and use, of online appointment booking.

Figures
Figures

Table 1 .
Awareness of online booking of general practice appointments.

Table 2 .
Variation in use of online booking of GP appointments.

easy is to use your GP practice website?
a P values from joint Wald tests.

Table 4 .
Odds ratio for different outcome measures, as explain below, adjusted for demographic factors, long term condition, occupation, deafness, period and for clustering of patients within practices.
a. P values from joint Wald tests b.Overall, how would you describe your experience of your GP practice?c.How often do you see or speak to your preferred GP when you would like to? d.On this occasion, were you offered a choice of appointment?e. Were you satisfied with the type of appointment you were offered?f.Overall, how would you describe your experience of making an appointment?

Table 6 .
Demographics details on participating general practices for the interview study.