overview

Mobile Calendars:
Apple vs. Google

Comparative Usability Analysis

As a student, I found even the most popular calendar apps painful to use. I tried to make Google Calendar work but ended up compiling a list of 50+ usability issues that plagued my experience. So, for UX271 (Research in User Experience Design), I formally analyzed the usability of Google Calendar and compared it with its primary competitor: Apple's Calendar.

‍The goal of the study was to determine which calendar was more user-friendly by using a mix of UX research methods to identify usability issues.

Nature

University project

Role

UX Research,
Statistical Analysis

Team

Solo

Time

4 months

Process & Tools

Limited to a maximum of 5 participants, I recruited 4 iPhone users, 1 of which dropped off after Part #1. Experiments were conducted remotely using Discord mobile screen share and Google Sheets was used for data collection, analysis, visualization.

Hypothesis

Despite the fact that Apple's Calendar should have an advantage on its native OS and my perception of Apple's dominance in UX, I predicted that Google's iOS Calendar would be the more user-friendly application due to its higher monthly usage.

Part #1

Basic Scheduling

Experiment 1

Scheduling events

Perhaps the most common task required by a calendar is scheduling an event. So, first, I wanted to see if users would have any issues in doing so. The goal of this experiment was to test the primary functionality of scheduling events in both applications and locate points of friction in each system.

Each participant was given the following task:
"Create a calendar event titled “Psychology 101” and schedule it for September 6th, 2021 between 10:00 and 11:00 AM. Set the event to repeat every Monday, Wednesday, and Friday until December 3rd, 2021. Set a reminder 30 minutes prior to each instance of the event."

Hypothesis A

Google Calendar will have a higher task success rate compared to Apple’s Calendar.

Hypothesis B

Google Calendar will require less time on task to complete compared to Apple’s Calendar.

Apple

Google

Result A

A chi-square test was run comparing the task success data between the two sample groups for each calendar application. The test produced a result of α = 0.65 indicating that there was no significant difference in task success between the two calendar applications.

Result B

A paired samples t-test was run comparing time on task between calendar applications. The test produced a result of α = 0.41 indicating that there was no significant difference in time on task between the two calendar applications.

Usability Issues

With no significant difference between either calendar application, we can conclude that either application is mostly similar in their performance. Based on task completion time, I would predict that this is because both applications are actually equally poor in their usability for this particular type of task. Participants found it quite difficult to figure out how to set a calendar event that repeats on multiple days of the week in both calendar applications and this is the point at which every single participant got stuck and began having to think quite hard about how to find what they were looking for.

The main problem with both applications seemed to be that a user had to know that to be able to select the days of the week that they wanted an event to repeat on, they first had to select that the event would repeat weekly rather than daily, which did not appear to be very intuitive. When a user navigates to the event repeating options, both systems could simply display a row of the seven days of the week that can all be selected or deselected as buttons. This would be presented alongside the existing drop-down menu that allows users to select the repeat frequency. This would allow users with either mental model (one beginning with weekly repeating followed by day selection, and one beginning with day selection followed by weekly repeating) to use the event repeat feature seamlessly and without so much friction.

Part #2

Multi-user Scheduling

Experiment 2

Scheduling virtual meetings between users

Although a more complex task than the first, scheduling a virtual meeting should be a primary feature of any modern calendar application. Therefore, we should expect a highly optimized and user-friendly process. The goal of this experiment was to gauge the quality of between-user functionality and uncover any usability issues relating to time zone differences, inviting, and digital meeting coordination via popular video call software.

Each participant was given the following task:
"Create a calendar event titled ‘Meeting’ and schedule it for Monday, March 29, at 3:00pm EST. Invite me to the meeting by email (hello@andrewmullins.ca) and indicate that you’d like to meet remotely via Zoom."

Hypothesis A

Apple Calendar will have greater unique usability issues than Google Calendar.

Hypothesis B

Apple Calendar's self-reported user satisfaction will be lower than Google Calendar's.

Self-reported Metrics

Result A

A paired samples t-test was run comparing total unique usability issues between calendar applications. The test produced a result of α = 0.25 indicating that there was no significant difference between the two calendars in terms of usability.

Result B

Two chi-square tests were run comparing user satisfaction using self-reported metrics. Users filled out a three-item semantic differential scale measuring perceived predictability, friction, and effectiveness as well as the After-Scenario Questionnaire (ASQ). The chi-square test for the Semantic Differential Scale produced a result of α = 0.92, while the chi-square test for the After-Scenario Questionnaire produced a result of α = 0.90, indicating that there was no a significant difference in self-reported user satisfaction between the two calendar applications for either scale of measurement.

Usability Issues

  1. Users had a visual preference for Google Calendar despite reporting it as the inferior calendar in terms of overall user satisfaction. Users said that if they were to use a personal calendar application they would rather learn to use Google Calendar even if it was the harder one to pick up.
  2. Users found that Apple Calendar had less functionality for accomplishing their task (but they still seemed more satisfied with it due to its simplicity).
  3. Users searched their meeting guest by name when the system expected an email address and they didn’t know whether the people popping up in the results were coming from their Google account or iPhone contacts.
  4. The timezone features in both applications proved to have significant flaws resulting in meetings scheduled at the wrong time. Neither featured an option to simply select a timezone by its name (i.e. EST, PST, etc.) so every user attempted to mentally calculate the timezone difference which consistently resulted in error.
  5. The user lacking the Invitee function in Apple Calendar also found that there was no indication within the system explaining how to gain the functionality to add someone to their meeting or why it was missing (despite both iOS and the app being updated to their latest versions)

Part #3

Calendar Views

Experiment 3

Creating and navigating between sub-calendars

Another critical, standardized calendar feature is sub-calendars, commonly used to categorize particular dates and events like birthdays and holidays. The creation of and navigation between these sub-calendars views is another core UX flow of any calendar app. The goal of this experiment was to test the overall organization of each calendar application and the ease of navigation between varying calendars and time-related views.

Each participant was given the following task:
“Create two calendars and title one ‘Work’ and the other ‘Play.’ Then, enter a monthly view of only these two calendars for the month of December 2021.”

Hypothesis A

Apple Calendar's ratio of positive to negative comments will be higher than for Google Calendar.

Result A

Apple Calendar's ratio of positive to negative comments was 2:0, while Google's was 1:5.

Usability Issues

  1. Users struggled with creating a new calendar in Google Calendar -> there is no way to create one on iOS.
  2. Navigating through longer spans of time and “zooming in/out” by date metrics (day/week/month/year) is not so obvious or intuitive in Google's Calendar.

Conclusion

Results & Suggested Improvements

Findings

As you'll see from the combined metrics scores, Google's Calendar was the winner. However, neither calendar scored particularly well. And although this result supports my initial hypothesis, I'd actually argue that Apple's Calendar was superior due to the fact that its usability flaws mostly resulted from missing features whereas Google's had issues with existing features that weren't properly executed.

Addressing bias

It is possible (if not likely, due to my personal calendar usage being predominantly desktop-based) that the experiments I designed overestimated the functionality of mobile calendars, which could explain the similarly low scores. Though, I don't believe, with mobile outpacing desktop, that it is unfair to expect a calendar to allow users to simply and easily schedule events, virtual meetings, and create and navigate between calendar views on mobile.

39.4%

Combined usability score

43.7%

Combined usability score
hello@andrewmullins.ca
Designed and developed by Andrew Mullins in Waterloo, ON.