---
title: "Pan and Zoom Timeline Analysis"
author: "Michail Schwab, Sicheng Hao, Olga Vitek, James Tompkin, Jeff Huang, Michelle A. Borkin"
#author: "Anonymous" #Michail Schwab, Sicheng Hao, James Tompkin, Jeff Huang, Michelle Borkin"
date: "March 21, 2018"
output:
pdf_document: default
html_document: default
---
# Introduction
This experiment evaluates timeline navigation on computers and mobile phones and compares visualization and interaction choices. We want to answer the questions:
1. **Q1 - Representation**: "Do timelines need visual guidance of year, month, day, hour and minute, or do users navigate numbers-based timelines just as fast?"
2. **Q2 - Orientation**: "Do people navigate pan and zoom timelines faster when they are horizontal or vertical?", and
3. **Q3 - Method**: "Which methods are faster or slower at navigating timelines compared to the standard methods on computers and mobile phones?".
# Experimental Design
This experiment was done with four sets of participants (Group 1-4), each split into half the number for each platform. With the first group, we evaluate timeline representations, such as dates. With the second group we evaluate both timeline orientations, such as horizontal and vertical, as well as one interaction method, as compared to the standard method on each platform. Groups three and four are only used to evaluate interaction techniques.
```{r, out.width = "400px", echo=FALSE}
#knitr::include_graphics("img/design.png")
```
The study design is within-subject. Half of the experiment was done on mobile, the other on desktop. No comparisons are done between platforms or between studies, and we fit separate models for each platform:
```{r, out.width = "480px", echo=FALSE}
#knitr::include_graphics("img/data-per-study.png")
knitr::include_graphics("img/studygroups-table.png")
```
Figure 1: Overview of the four groups of study participants, the factor that was varied during the study, and the fixed parameters. Group 1 data was used in for Q1 (numbers versus dates). Group 2 data was used for Q2 (horizontal versus vertical) and for Q3 (methods). Data from groups 3 and 4 was used for Q3. All four groups were split somewhat evenly into groups of about forty participants per platform (mobile phone and PC).
About 40 participants were part of each study per platform. With 4 studies and two platforms, we have $316 \approx 320 = 4 \cdot 2 \cdot 40$ participants. All study parameters were presented in a randomized order.
In the first study, we evaluate whether people are faster navigating a timeline with explicit years, months, days, hours and seconds, or whether they can navigate a line simply with number annotations, where numbers range from zero to 20,000,000. For this, we keep the other parameters, such as pan and zoom method, fixed. Since we only vary one binary variable, we only measure two sets of data per participant.
In the second study, we evaluate whether people are faster navigating a horizontal or vertical timeline while keeping orientation fixed as vertical. In addition, we show participants two different interaction techniques that we expect to be more dependant on orientation to see whether any possible difference between horizontal and vertical timelines is stronger with the different interaction technique. In total we collect four sets of data per participants, since orientation is binary and we switch between two interaction techniques.
In studies 3 and 4, we only use vertical dates-based visualizations, and we vary the interaction technique. Each participant sees the standard interaction technique for their respective platform (mobile or desktop), and is additionally exposed to one to three other techniques. In total, we measure data for five interaction techniques on vertical dates-based timelines per platform, in addition to the standard method.
Below, we show in which study we collected the data for different interaction methods to answer the question of the best method, Q_MET.
```{r, out.width = "150px", echo=FALSE}
knitr::include_graphics("img/interaction-data.png")
```
Figure 3: Overview of which data was collected in which part of the study. Each ``o'' symbolizes that we measured task times for all ~40 participants of that study on the visualization and method of the respective column. The standard methods for each platform (scrollwheel zoom for desktop, and pinch zoom for mobile phones) are highlighted in green. Each row is the same set of 40 people that all underwent the same set of conditions.
Participants were explained the methods they were interacting with via text, and had a test run to practice the method before performing the tasks twice. Each set of tasks was comprised of 8 tasks, where participants first had to zoom in from a fully zoomed out timeline, and then zoom to targets on the timeline of 10, 20, 40, 100, 1,000, 100,000, and 5,000,000 minutes / units distance away from the last target. The order in which these distances appeared as targets is randomized.
```{r, out.width = "400px", echo=FALSE}
#knitr::include_graphics("img/study.png")
```
# Data Preparation: Loading and Parsing
```{r libraries, include=FALSE, results='hide', message=FALSE, warning=FALSE}
library(dplyr)
library(tidyr)
library(ggplot2)
library(lme4)
library(car)
library(MASS)
#library(lsmeans)
library(emmeans)
library(gridExtra)
library(interplot)
library(visreg)
library(moments)
knitr::opts_chunk$set(echo = TRUE)
```
```{r, warning=FALSE, echo=FALSE}
setwd('data')
data = read.csv("7290data_wide.csv")
data = data[data$run != 'testrun-1',]
data = data[data$dist!='dist-0',]
data$study = substr(data$id, 1, 4)
# hold zoom method of sz02 should not be used from mobile because of implementation problems.
data = data[data$method != "vertical-dates-normalpan-holdzoomin-holdzoomout"
| data$mobile == TRUE | data$study != "SZ02",]
```
```{r, warning=FALSE, echo=FALSE}
# Some distances were slightly altered during the study to
# to be less repetitive and predictable.
data[data$dist == 'dist-21',]$dist = 'dist-20'
data[data$dist == 'dist-39',]$dist = 'dist-40'
data[data$dist == 'dist-98',]$dist = 'dist-100'
data[data$dist == 'dist-997',]$dist = 'dist-1000'
data[data$dist == 'dist-98972',]$dist = 'dist-100000'
data[data$dist == 'dist-98970',]$dist = 'dist-100000'
data[data$dist == 'dist-4987901',]$dist = 'dist-5000000'
data[data$dist == 'dist-4987900',]$dist = 'dist-5000000'
```
## Preprocessing: Distances and Difficulties
Here, we define the task difficulties, and calculate the log-transformed values we use in the analysis.
```{r, warning=FALSE}
distances = c(0, 10, 20, 40, 100, 1000, 100000, 5000000)
difficulties = sapply(distances, function(d) log2(d+1))
data$distNumber = as.numeric(substr(data$dist, 6, 100))
data$difficulty = difficulties[match(data$distNumber, distances)]
data$logDifficulty = log(data$difficulty+1)
data$isInitialZoomIn = data$dist == 'dist-0'
data$isInitialZoomInNumeric = as.numeric(data$dist == 'dist-0')
data$logy = log(data$taskTime + 1)
```
# Exploration
## Outliers
Extreme Outliers: Over 100 tasks in which participants idled for over a minute at once, including up to one hour.
```{r}
nrow(data[data$max.idle > 60000,])
```
In terms of outlier rejection, task times were only removed on grounds of users not paying attention: If a user was ever not interacting with the website for more than 10 seconds during a task, then we removed that participant's data for that task (task durations typically ranged from 5-50 seconds). This is not aggregating total idle time, but only removing task data when a participant continuously did not interact for then seconds at some point during the task. In total, this removed about 7 \% of the data.
```{r}
data = data[data$max.idle <= 10000,]
data.p = data[data$mobile == FALSE,]
data.m = data[data$mobile == TRUE,]
```
# Motivation of Log Transform
As an example, we will look at distance 100.000 of the first study on computers.
```{r}
data.s0p<-data%>%filter(study=="SZ00" & mobile == FALSE)
unique(data.s0p$distNumber)
data.s0p.example = data.s0p%>%filter(distNumber==100000)
hist(data.s0p.example$taskTime / 1000, breaks=30)
```
The task times are clearly very skewed. The median is at 31 seconds:
```{r}
median(data.s0p.example$taskTime / 1000)
```
However, individual people performed much worse, and took up to 96 seconds:
```{r}
max(data.s0p.example$taskTime / 1000)
```
This example task is not an outlier, and is randomly chosen, as this is what the data looks like for most tasks. Remember that this is after outlier removal based on maximum idle time. We have to accept that this is what the data looks like when it comes from this realistic and uncontrolled environment.
However, careful modeling has to be done to not make wrong assumptions about the data. A linear regression would assume normal distribution, which is not available here. One indicator that this is not a normal distribution is the data's skewness:
```{r}
skewness(data.s0p.example$taskTime)
```
A skewness of 1.6 is a clear indication that the data is heavily right-skewed. An optimal value for linear modeling would be near zero.
Because of this, we try a log transformation of the data, and get:
```{r}
hist(data.s0p.example$logy, breaks=30)
```
Here, the distribution looks much more normal.
```{r}
skewness(data.s0p.example$logy)
```
A skewness of 0.4 is still not ideal, but we consider this good enough to be able to deal with it. As an example, we show below what a linear model would look like for Q1, including a non-ideal Q-Q plot, but otherwise use the log-transformed data for analysis.
## Fitts Law
Fitts' Law from Human-Computer Interaction predicts that the relationship between distance between pointer and target and the time it takes users to cover this distance is logarithmic. For this reason, in HCI, researchers refer to the "Index of Difficulty": The index of difficulty is the logarithm of the ratio between target distance and target size. Using the index of difficulty, rather than the distance to a target, we can assume a linear relationship between task time and index of difficulty.
```{r, warning=FALSE, echo=FALSE, fig.width=4,fig.height=2.5}
#pdf("fitts.pdf", width=4, height=3)
meansPC = sapply(distances[distances!=0], function(distance)
mean(data.p[data.p$distNumber == distance,]$taskTime) / 1000)
meansMobile = sapply(distances[distances!=0], function(distance)
mean(data.m[data.m$distNumber == distance,]$taskTime) / 1000)
plotData = data.frame(data = c(meansMobile, meansPC),
x = c(difficulties[difficulties!=0], difficulties[difficulties!=0]),
Platform = c(rep("Mobile", 7), rep("Computer", 7)))
qplot(plotData$x, plotData$data, data = plotData, group=Platform, color=Platform, main = "Aggregate Task Times Across All Studies and Methods", ylab = "Task Time in seconds", xlab = "Index of Difficulty", size=I(3)) + theme_bw() + theme(legend.position = c(0.85, 0.24)) + theme(plot.title = element_text(size=10))
#dev.off()
```
```{r, warning=FALSE, echo=FALSE, fig.width=4,fig.height=2.5}
#pdf("fitts-fitted.pdf", width=4, height=3)
meansPC = sapply(distances[distances!=0], function(distance)
mean(data.p[data.p$distNumber == distance,]$taskTime) / 1000)
meansMobile = sapply(distances[distances!=0], function(distance)
mean(data.m[data.m$distNumber == distance,]$taskTime) / 1000)
plotData = data.frame(data = c(meansMobile, meansPC),
x = c(difficulties[difficulties!=0], difficulties[difficulties!=0]),
Platform = c(rep("Mobile", 7), rep("Computer", 7)))
linearFitPc <- lm(data ~ x, data=plotData[plotData$Platform == 'Computer',])
summary(linearFitPc)
linearFitMobile <- lm(data ~ x, data=plotData[plotData$Platform == 'Mobile',])
summary(linearFitMobile)
qplot(plotData$x, plotData$data, data = plotData, group=Platform, color=Platform, main = "Aggregate Task Times Across All Studies and Methods", ylab = "Task Time in seconds", xlab = "Index of Difficulty", size=I(3), geom=c("point", "smooth"), method="lm", formula=y~x, level=0) + theme_bw() + theme(legend.position = c(0.85, 0.24)) + theme(plot.title = element_text(size=10))
#dev.off()
```
While there is a reasonable amount of variation in the data, it appears to be in accordance with Fitts' law. In our analysis, we assume Fitts' law to hold, and model the task times accordingly.
## Dates Vs Numbers
```{r, warning=FALSE, echo=FALSE, fig.width=6,fig.height=3.5}
#pdf("dates-numbers-raw.pdf", width=7, height=4)
s0md = sapply(distances[distances!=0], function(distance) mean(data[data$mobile == TRUE & data$study == "SZ00" & data$method == "horizontal-dates-normalpan-pinchzoom" & data$dist == sprintf("dist-%d", distance),]$taskTime) / 1000)
s0mn = sapply(distances[distances!=0], function(distance) mean(data[data$mobile == TRUE & data$study == "SZ00" & data$method == "horizontal-numbers-normalpan-pinchzoom" & data$dist == sprintf("dist-%d", distance),]$taskTime) / 1000)
s0pd = sapply(distances[distances!=0], function(distance) mean(data[data$mobile == FALSE & data$study == "SZ00" & data$method == "horizontal-dates-normalpan-wheelzoom" & data$dist == sprintf("dist-%d", distance),]$taskTime) / 1000)
s0pn = sapply(distances[distances!=0], function(distance) mean(data[data$mobile == FALSE & data$study == "SZ00" & data$method == "horizontal-numbers-normalpan-wheelzoom" & data$dist == sprintf("dist-%d", distance),]$taskTime) / 1000)
plotData = data.frame(
data = c(s0md, s0mn, s0pd, s0pn), x = c(difficulties[distances!=0], difficulties[distances!=0], difficulties[distances!=0], difficulties[distances!=0]),
method = c(rep("mobile dates", 7), rep("mobile numbers", 7), rep("computer dates", 7), rep("computer numbers", 7)),
Representation = c(rep("Dates", 7), rep("Numbers", 7), rep("Dates", 7), rep("Numbers", 7)),
Platform = c(rep("Mobile", 14), rep("Computer", 14))
)
qplot(plotData$x, plotData$data, data = plotData, group=method, color = Platform,shape=Platform, main = "Q1: Dates and Numbers Comparison",
ylab = "Task Time in seconds", xlab = "Index of Difficulty", size=I(2)) +
theme(legend.position = c(0.85, 0.2)) +
geom_line(aes(colour=Platform,linetype=Representation)) +
#geom_line(aes(linetype=linetypevar)) +
#scale_colour_manual(values = c("red", "red", "blue", "blue")) +
theme_bw() +
#theme(legend.position="bottom")
theme(legend.position = c(0.7, 0.18), legend.box="horizontal", legend.box.background = element_rect())
# theme(legend.key.width = unit(0.08, "cm")) +
# theme(legend.key.size = unit(0.4, "cm"))
#gplot(data = plotData, aes(x=dist, y=val)) + geom_line(aes(colour=variable))
#matplot(plotData, type = c("b"), col = 2:3) #plot
#dev.off()
```
## Vertical Vs Horizontal
```{r, warning=FALSE, echo=FALSE, fig.width=9,fig.height=4}
#pdf("vertical-horizontal-raw.pdf", width=9, height=3.5)
s1mph = sapply(distances[distances!=0], function(distance) mean(data[data$mobile == TRUE & data$study == "SZ01" & data$method == "horizontal-dates-normalpan-pinchzoom" & data$dist == sprintf("dist-%d", distance),]$taskTime) / 1000)
s1mpv = sapply(distances[distances!=0], function(distance) mean(data[data$mobile == TRUE & data$study == "SZ01" & data$method == "vertical-dates-normalpan-pinchzoom" & data$dist == sprintf("dist-%d", distance),]$taskTime) / 1000)
s1mdv = sapply(distances[distances!=0], function(distance) mean(data[data$mobile == TRUE & data$study == "SZ01" & data$method == "vertical-dates-normalpan-dynamiczoomxadjustable" & data$dist == sprintf("dist-%d", distance),]$taskTime) / 1000)
s1mdh = sapply(distances[distances!=0], function(distance) mean(data[data$mobile == TRUE & data$study == "SZ01" & data$method == "horizontal-dates-normalpan-dynamiczoomxadjustable" & data$dist == sprintf("dist-%d", distance),]$taskTime) / 1000)
s1pwh = sapply(distances[distances!=0], function(distance) mean(data[data$mobile == FALSE & data$study == "SZ01" & data$method == "horizontal-dates-normalpan-wheelzoom" & data$dist == sprintf("dist-%d", distance),]$taskTime) / 1000)
s1pwv = sapply(distances[distances!=0], function(distance) mean(data[data$mobile == FALSE & data$study == "SZ01" & data$method == "vertical-dates-normalpan-wheelzoom" & data$dist == sprintf("dist-%d", distance),]$taskTime) / 1000)
s1pbv = sapply(distances[distances!=0], function(distance) mean(data[data$mobile == FALSE & data$study == "SZ01" & data$method == "vertical-dates-wheelpan-brushzoomin-dblclickzoomout" & data$dist == sprintf("dist-%d", distance),]$taskTime) / 1000)
s1pbh = sapply(distances[distances!=0], function(distance) mean(data[data$mobile == FALSE & data$study == "SZ01" & data$method == "horizontal-dates-wheelpan-brushzoomin-dblclickzoomout" & data$dist == sprintf("dist-%d", distance),]$taskTime) / 1000)
colors = c("purple","orange")
#plotData = data.frame(dist = difficulties, horizontalDynamic = s1mdh, verticalDynamic = s1mdv)
plotDataMobile = data.frame(data = c(s1mpv, s1mph, s1mdv, s1mdh),
x = rep(difficulties[distances!=0], 2.5),
method = c(rep("pinch vertical", 7), rep("pinch horizontal", 7), rep("dynamic vertical", 7), rep("dynamic horizontal", 7)),
Technique = c(rep("Pinch", 14), rep("Dynamic", 14)),
Orientation = c(rep("Vertical", 7), rep("Horizontal", 7), rep("Vertical", 7), rep("Horizontal", 7))
)
plotDataPC = data.frame(data = c(s1pwh, s1pwv, s1pbv, s1pbh),
x = rep(difficulties[distances!=0], 4),
method = c(rep("wheel vertical", 7), rep("wheel horizontal", 7), rep("brush vertical", 7), rep("brush horizontal", 7)),
Technique = c(rep("Wheel", 14), rep("Brush", 14)),
Orientation = c(rep("Vertical", 7), rep("Horizontal", 7), rep("Vertical", 7), rep("Horizontal", 7))
)
plot1 <- qplot(plotDataMobile$x, plotDataMobile$data, data = plotDataMobile, group=method, color=Technique, shape=Technique,main = "Mobile Q2: Vertical Vs. Horizontal",
ylab = "Task Time in seconds", xlab = "Index of Difficulty", size=I(2.5)) +
geom_line(aes(colour=Technique,linetype=Orientation)) +
theme_bw() +
scale_colour_manual(values = colors, name="Technique") +
theme(legend.position = c(0.7, 0.18), legend.box="horizontal", legend.box.background = element_rect())
plot2 <- qplot(plotDataPC$x, plotDataPC$data, data = plotDataPC, group=method, color=Technique, shape=Technique,main = "Computer Q2: Vertical Vs. Horizontal",
ylab = "Task Time in seconds", xlab = "Index of Difficulty", size=I(2.5)) +
geom_line(aes(colour=Technique, linetype=Orientation)) +
theme_bw() +
scale_colour_manual(values = colors, name="Technique") +
theme(legend.position = c(0.7, 0.18), legend.box="horizontal", legend.box.background = element_rect())
grid.arrange(plot1, plot2, ncol=2)
#dev.off()
```
# Analysis
## General Approach
Motivated by the exploration above, we model the task times on a logarithmic scale. We want to conform to Fitts' law:
$$
\bar{Y} \propto \mathrm{difficulty} \propto \log_2(\mathrm{target distance})
$$
But, as we showed above, and as shown again for Q1 below, a logarithmic transformation is necessary. Hence we are modeling $\log(\bar{Y})$, and have to use $\log(\mathrm{difficulty})$:
$$
\log(\bar{Y}) \propto \log(\mathrm{difficulty}) \propto \log(\log_2(\mathrm{target distance}))
$$
With this, we denote:
\begin{itemize}
\item $Y_{ijr}$ is the time to completion for a representation/orientation/method, a task type (initial zoom in, or normal navigation), a person and a replicate.
\item $\beta_i, i=1..I$ is the varied fixed intercept parameter, such as representation, orientation, and method. $\beta_1$ is always used as baseline, with $\beta_1 = 0$.
\item $\gamma_i, i=1..I$ is the varied fixed slope parameter, such as representation, orientation, and method. $gamma_1$ is always used as baseline, with $\gamma_1 = 0$.
\item $id_j, j = 1..40$ is the participant-specific variation, which is assumed to be normally distributed.
\item $r = 1, 2$ are the replicates, since each participant did each task twice.
\item $\epsilon_{ijr}$ is the non-systematic error.
\end{itemize}
as follows:
\begin{eqnarray*}
&log(Y)_{ijr} = \mathrm{Intercept} &+ \alpha \cdot \log(\mathrm{difficulty})\\
&&+ \mathrm{\beta}_i + \mathrm{\gamma}_i \cdot \log(\mathrm{difficulty})\\
&&+ \mathrm{id}_{j}+\epsilon_{ijr},\ \mbox{where}\\
& \mathrm{\gamma}_{1} = 0, &\mathrm{\beta}_{1} = 0 \text{~as baseline level}, \\
&id_{j} \stackrel{iid}{\sim} \mathcal{N}(0,\tau^2), & \epsilon_{ijr} \stackrel{iid}{\sim} \mathcal{N}(0,\sigma^2)
\end{eqnarray*}
and all $\mathrm{id}_{j}$ and $\epsilon_{ijr}$ are independent.
For hypothesis testing, we build a confidence band for each value $\mathrm{\gamma}_i$ of the fixed variable (such as representation, orientation and method). Then, the contribution of this level of the fixed effect on the total task time is compared against zero across all distances:
\begin{eqnarray*}
H_{0}: \mathrm{\beta}_i + \mathrm{\gamma}_i \cdot \log(\mathrm{difficulty}) =& 0 & \ ~~\forall \mathrm{difficulty}\\
H_{a}: \mathrm{\beta}_i + \mathrm{\gamma}_i \cdot \log(\mathrm{difficulty}) \neq& 0 & \ ~~\mathrm{for~some~} \mathrm{difficulty}
\end{eqnarray*}
Being equal to zero would mean to be equivalent to the the level of the baseline representation/orientation/method, $\mathrm{\beta}_1,\mathrm{\gamma}_1$ This baseline changes as we ask different questions below.
For Question 1, we do one test per platform: we compare dates with numbers, hence we have a total of 2 tests for question 1. For question 2, we compare the orientation of two methods per platform, hence we have a total of 4 tests for question 2. For question 3, we compare 5 methods with the standard method per platform, hence we have a total of 10 tests for this platform. Altogether, we do 16 tests. Being defensive with our claims, we use Bonferoni's method for multiple testing. We use a family-wise error rate of XXXXXX, with an individual test error rate of XXXX / 16.
```{r}
alphaTotal = 0.05
bonferoniFactor = 16
alpha = alphaTotal / bonferoniFactor
ci = 1 - alpha
```
## Question 1: Numbers or Dates?
In this case, the fixed parameter that is varied is the representation of the targets, $\mathrm{\beta}_i = \mathrm{representation}_i$. Since there is no very obvious standard representation, we arbitrarily set the ``dates'' representation as baseline, $\mathrm{\beta}_1 = \mathrm{representation}_1 = 0$. We then investigate the numbers-based representation relative to this baseline.
```{r, warning=FALSE, include=FALSE}
data.s0p<-data%>%filter(study=="SZ00" & mobile == FALSE)
count<-length(unlist(strsplit(as.character(data.s0p$method),split = "-")))
data.s0p$method<-unlist(strsplit(as.character(data.s0p$method),split = "-"))[seq(2,count-2,by = 4)]
data.s0m<-data%>%filter(study=="SZ00" & mobile == TRUE)
count<-length(unlist(strsplit(as.character(data.s0m$method),split = "-")))
data.s0m$method<-unlist(strsplit(as.character(data.s0m$method),split = "-"))[seq(2,count-2,by = 4)]
ggplot(data = rbind(data.s0p,data.s0m))+geom_boxplot(aes(x = as.factor(difficulty),y = logy,fill = method))+facet_wrap(~mobile)+theme(axis.text.x = element_text(angle = 90, hjust = 1))
```
Here, we want to find out whether participants were faster at navigating to targets when they were visualized as dates or as numbers.
### Computer
```{r, warning=FALSE}
fit.s0p<-lmer(data = data.s0p,logy ~ (1|id) +method+logDifficulty+
method * logDifficulty, REML = FALSE)
summ.s0p<-summary(fit.s0p)
qqnorm(summ.s0p$residuals)
summ.s0p
```
```{r, warning=FALSE, fig.height=3}
visreg(fit.s0p, "logDifficulty", band = TRUE, by="method", overlay=T, jitter=TRUE, ci=ci)
```
```{r, fig.width=4,fig.height=3, echo=FALSE}
#pdf("q1p.pdf", width=2, height=3)
interplot(fit.s0p, var1 = "method", var2 = "logDifficulty", steps = 100, xmin=1.4, ci=ci) +
theme_bw() + geom_hline(yintercept = 0, linetype = "dashed") +
ylab("Est. log Task Time Relative to Dates") +
xlab("log(Difficulty)") +
ggtitle("Representation Comparison for Computer")
#dev.off()
```
Here, for small distances (until the logDifficulty is 1.7, referring to mostly panning navigation), we have to accept the null hypothesis, meaning that we have not found significant difference between numbers and dates for navigation on computers that primarily involves panning. However, for larger distances, we accept the alternative hypothesis, and conclude that dates are significantly faster than numbers when navigating on computers when zooming is important.
### Impact of Log transformation
After having seen the non-normality of the data in the exploration and motivating the logarithmic transformation, we can here further show the utility of the log transformation by comparing model results with and without. In the Q-Q Plot above, we can see that this model fits the data nicely. We can further check normality by checking the skewness of the residuals:
```{r}
skewness(summ.s0p$residuals)
```
A skewness of 0.4 is relatively close to 0, which would be a perfect normal distribution of the error. To motivate the log transformation, we show the Q-Q plot and skewness of residuals below without transformation:
```{r}
fit.s0p.linear<-lmer(data = data.s0p,taskTime ~ (1|id) +method+difficulty+
method * difficulty, REML = FALSE)
summ.s0p.linear<-summary(fit.s0p.linear)
qqnorm(summ.s0p.linear$residuals)
summ.s0p.linear
```
From the Q-Q Plot, it is clear that the model does not fit the data well. The AIC scores of 24000 and 1000 for linear and logarithmic model, respectively, also suggest a model selection of the logarithmic model. We further observe the skewness of the residuals in the linear model:
```{r}
skewness(summ.s0p.linear$residuals)
```
The skewness is 2, which is too high to make a normality assumption. Hence we use the logarithmic model here and in the remaining questions.
### Mobile
```{r, warning=FALSE, echo=FALSE}
fit.s0m<-lmer(data = data.s0m,logy ~ (1|id)+method+logDifficulty+
method * logDifficulty, REML = FALSE)
summ.s0m<-summary(fit.s0m)
#boxplot(summ.s0m$residuals~as.character(data.s0m$method))
#qqnorm(summ.s0m$residuals)
#summ.s0m
```
```{r, warning=FALSE, echo=FALSE, fig.height=3}
visreg(fit.s0m, "logDifficulty", band = TRUE, by="method", overlay=T, jitter=TRUE, ylab="log(taskTime)", yaxt="n")
```
```{r, fig.width=4,fig.height=3, echo=FALSE}
#pdf("q1m.pdf", width=2, height=3)
interplot(fit.s0m, var1 = "method", var2 = "logDifficulty", steps = 100, xmin=1.4, ci=ci) +
theme_bw() + geom_hline(yintercept = 0, linetype = "dashed") +
ylab("Est. log Task Time Relative to Dates") +
xlab("log(Difficulty)") +
ggtitle("Representation Comparison for Mobile")
#dev.off()
```
On mobile phones, we accept the null hypothesis for the middle range of distance values, meaning that for tasks that involve some panning and some zooming, we did not find a significant difference between navigating dates and numbers. For navigation primarily involving panning, numbers were significantly faster than dates, and for navigation primarily involving zooming, dates were significantly faster than numbers.
## Question 2: Vertical or Horizontal?
```{r, include=FALSE}
data.s1p<-data%>%filter(study=="SZ01" & mobile == FALSE)
data.s1m<-data%>%filter(study=="SZ01" & mobile == TRUE)
# ggplot(data = rbind(data.s1p,data.s1m))+geom_boxplot(aes(x = as.factor(difficulty),y = logy,fill = method))+facet_wrap(~mobile)+theme(axis.text.x = element_text(angle = 90, hjust = 1))
```
### Computer
Here, we set horizontal wheel zoom as base line to be able to compare against the vertical wheel zoom. We ignore results for the other interaction method in this part, since here, we are only comparing orientation. Including them in the model still aids estimate participant variance, so their data is included.
```{r, echo=FALSE}
data.s1p$m = as.character(data.s1p$method)
data.s1p$m[data.s1p$m == "horizontal-dates-normalpan-wheelzoom"] = "horizontal wheel"
data.s1p$m[data.s1p$m == "vertical-dates-normalpan-wheelzoom"] = "vertical wheel"
data.s1p$m[data.s1p$m == "horizontal-dates-wheelpan-brushzoomin-dblclickzoomout"] = "horizontal brush"
data.s1p$m[data.s1p$m == "vertical-dates-wheelpan-brushzoomin-dblclickzoomout"] = "vertical brush"
data.s1p$m_ = factor(data.s1p$m)
data.s1p$m_ = relevel(data.s1p$m_, ref="horizontal wheel")
fit.s1p<-lmer(data = data.s1p,logy ~ (1|id)+ m_+logDifficulty
+ m_ * logDifficulty,REML = FALSE)
summ.s1p<-summary(fit.s1p)
#boxplot(summ.s1p$residuals~as.character(data.s1p$method))
#qqnorm(summ.s1p$residuals)
#summ.s1p
data.s1p.1<-data.s1p
data.s1p.1$m_ = relevel(data.s1p.1$m_, ref="horizontal brush")
fit.s1p.1<-lmer(data = data.s1p.1,logy ~ (1|id)+ m_+logDifficulty
+ m_ * logDifficulty,REML = FALSE)
summ.s1p.1 <- summary(fit.s1p.1)
```
```{r, warning=FALSE, echo=FALSE, fig.height=3}
visreg(fit.s1p, "logDifficulty", band = TRUE, by="m_", overlay=T, jitter=TRUE, ylab="log(taskTime)", yaxt="n", gg=TRUE)
```
```{r, echo=FALSE,fig.height=3}
#pdf("q2pw.pdf", width=3, height=3)
interplot(fit.s1p, var1 = "m_", var2 = "logDifficulty", xmin=1.4, ci=ci) + theme_bw() +
ggtitle("Computer Comparison. Baseline: Wheel Zoom Horizontal") +
ylab("Est. log Task Time Relative to Horizontal") +
xlab("log(Difficulty)") +
geom_hline(yintercept = 0, linetype = "dashed")
#dev.off()
```
We focus on the comparison with wheel zoom vertical (right). Across all difficulties, we can not reject the null hypothesis, and conclude that for the computer, we could not find a significant difference between horizontal and vertical navigation using scroll wheel zoom with normal pan. This does not mean that no difference exists, only that we can not be sure either way with this data.
Next, we investigate the difference between horizontal and vertical orientation using brush zoom and scroll wheel panning. For a simple comparison, we swap the baseline to be the horizontal brush zoom method, and re-draw the confidence bands of the remaining methods:
```{r, echo=FALSE, fig.height=3}
visreg(fit.s1p, "logDifficulty", band = TRUE, by="m_", overlay=T, jitter=TRUE, gg=TRUE)
```
```{r, echo=FALSE,fig.height=3}
#pdf("q2pb.pdf", width=3, height=3)
interplot(fit.s1p.1, var1 = "m_", var2 = "logDifficulty", xmin=1.4, ci=ci) + theme_bw() +
ggtitle("Computer Comparison. Baseline: Brush Zoom Horizontal") +
ylab("Est. log Task Time Relative to Horizontal") +
xlab("log(Difficulty)") +
geom_hline(yintercept = 0, linetype = "dashed")
#dev.off()
```
Here, we focus on comparing brush zoom (center). We can see that the confidence band is quite consistently below zero, but starts to include zero for the very highest differences. Hence, we accept the alternative hypothesis for the most part, meaning that on computers, participants performed significantly faster using vertical orientation of brush zoom with scroll wheel pan except for targets very far, where we have not been able to measure a significant difference.
```{r, include=FALSE}
difficulties = c(0, 3.3, 4.3, 5.3, 6.6, 10, 16.6, 22.3)
brushzoomvertical.inter = summ.s1p.1$coefficients[4]
brushzoomvertical.slope = summ.s1p.1$coefficients[11]
brushzoomvertical.p<-difficulties*brushzoomvertical.slope+brushzoomvertical.inter
brushzoomhorizontal.p<-difficulties*0
res.df<-data.frame(brushzoomhorizontal.p,brushzoomvertical.p, difficulties)
res.df<-res.df%>%gather("method","y",1:2)
ggplot(data = res.df)+geom_line(aes(x = difficulties,y = y,colour = method))
```
#### Mobile
Equivalent to the analysis for the computer data, we first set the baseline method to be horizontal pinch zoom. Then, we find the following confidence bands:
```{r, warning=FALSE, echo=FALSE}
data.s1m$m = as.character(data.s1m$method)
data.s1m$m[data.s1m$m == "horizontal-dates-normalpan-pinchzoom"] = "horizontal pinch"
data.s1m$m[data.s1m$m == "vertical-dates-normalpan-pinchzoom"] = "vertical pinch"
data.s1m$m[data.s1m$m == "horizontal-dates-normalpan-dynamiczoomxadjustable"] = "horizontal dynamic"
data.s1m$m[data.s1m$m == "vertical-dates-normalpan-dynamiczoomxadjustable"] = "vertical dynamic"
data.s1m$m_ = factor(data.s1m$m)
data.s1m$m_ = relevel(data.s1m$m_, ref="horizontal pinch")
fit.s1m<-lmer(data = data.s1m,logy ~ (1|id) + m_ + logDifficulty
+ m_ * logDifficulty, REML = FALSE)
summ.s1m<-summary(fit.s1m)
#boxplot(summ.s1m$residuals~as.character(data.s1m$m_))
#qqnorm(summ.s1m$residuals)
#summ.s1m
data.s1m.1<-data.s1m
data.s1m.1$m_ = relevel(data.s1m.1$m_, ref="horizontal dynamic")
fit.s1m.1<-lmer(data = data.s1m.1,logy ~ (1|id)+ m_+logDifficulty
+ m_ * logDifficulty, REML = FALSE)
#summary(fit.s1m.1)
```
```{r, echo=FALSE, fig.height=3}
visreg(fit.s1m, "logDifficulty", band = TRUE, by="m_", overlay=T, jitter=TRUE, gg=TRUE)
```
```{r, echo=FALSE,fig.height=3}
#pdf("q2mp.pdf", width=3, height=3)
interplot(fit.s1m, var1 = "m_", var2 = "logDifficulty", xmin=1.4, ci=ci) +
theme_bw() +
ylab("Est. log Task Time Relative to Horizontal") +
xlab("log(Difficulty)") +
ggtitle("Mobile Comparison. Baseline: Pinch Zoom Horizontal") +
geom_hline(yintercept = 0, linetype = "dashed")
#dev.off()
```
We focus on comparing pinch zoom (right). For small to medium distances, the fit of the vertical orientation is significantly below zero, before the confidence interval contains zero. This means that we have to reject the null hypothesis for small to medium distances, meaning that for pinch zoom on mobile, navigation to nearby targets on a vertical timeline is significantly faster than to the same distance targets on a horizontal timeline. Next, we set the baseline to be horizontal dynamic zoom:
```{r, echo=FALSE,fig.height=3}
#pdf("q2md.pdf", width=3, height=3)
interplot(fit.s1m.1, var1 = "m_", var2 = "logDifficulty", xmin=1.4, ci=ci) +
theme_bw() +
ylab("Est. log Task Time Relative to Horizontal") +
xlab("log(Difficulty)") +
ggtitle("Mobile Comparison. Baseline: Dynamic Zoom Horizontal") +
geom_hline(yintercept = 0, linetype = "dashed")
#dev.off()
```
We focus on comparing dynamic zoom (center). Consistently, the vertical orientation is faster - we accept the alternative hypothesis.
## Question 3: Techniques Comparison
Because in most cases, dates and vertical timelines were faster than numbers and horizontal timelines, we chose to do the comparison of interaction techniques using vertical dates-based timelines. In this part of the analysis, the method is the fixed parameter, $\mathrm{fixed}_i = \mathrm{method}_i$. For both platforms, we choose the respective standard method as baseline: pinch zoom for mobile phones, scroll wheel zoom for computer.
```{r, echo=FALSE}
methods.p<-c("vertical-dates-normalpan-wheelzoom","vertical-dates-wheelpan-brushzoomin-dblclickzoomout","vertical-dates-normalpan-holdzoomin-holdzoomout","vertical-dates-normalpan-dblclickzoomin-holdzoomout","vertical-dates-normalpan-dynamiczoomxadjustable","vertical-dates-normalpan-rubzoominx-rubzoomouty")
methods.m<-c("vertical-dates-normalpan-pinchzoom","vertical-dates-pinchpan-brushzoomin-dblclickzoomout","vertical-dates-normalpan-holdzoomin-holdzoomout","vertical-dates-normalpan-dblclickzoomin-holdzoomout","vertical-dates-normalpan-dynamiczoomxadjustable","vertical-dates-normalpan-rubzoominx-rubzoomouty")
data.s2p<-data%>%filter(mobile == FALSE,method %in% methods.p)
data.s2m<-data%>%filter(mobile == TRUE,method %in% methods.m)
data.s2p$method<-as.character(data.s2p$method)
data.s2m$method<-as.character(data.s2m$method)
```
```{r, include=FALSE}
ggplot(data = rbind(data.s2p,data.s2m))+geom_boxplot(aes(x = as.factor(difficulty),y = logy))+facet_grid(mobile~method)+theme(axis.text.x = element_text(angle = 90, hjust = 1))
```
### Computer
```{r, echo=FALSE}
#data.methods.p$D<-as.factor(round(data.methods.p$difficulty,digits = 1))
data.s2p$m = substr(data.s2p$method, 16, 100)
data.s2p$m[data.s2p$m == "normalpan-wheelzoom"]<-"Standard"
data.s2p$m[data.s2p$m == "normalpan-dblclickzoomin-holdzoomout"] = "2x click"
data.s2p$m[data.s2p$m == "normalpan-dynamiczoomxadjustable"] = "dynamic"
data.s2p$m[data.s2p$m == "normalpan-holdzoomin-holdzoomout"] = "hold"
data.s2p$m[data.s2p$m == "normalpan-rubzoominx-rubzoomouty"] = "rub"
data.s2p$m[data.s2p$m == "wheelpan-brushzoomin-dblclickzoomout"] = "brush"
data.s2p$m_ = factor(data.s2p$m)
data.s2p$m_ = relevel(data.s2p$m_, ref="Standard")
fit.s2p<-lmer(data = data.s2p,logy ~ (1|id) + logDifficulty+m_+
m_ * logDifficulty, REML = FALSE)
summ.s2p<-summary(fit.s2p)
#boxplot(summ.s2p$residuals~as.character(data.s2p$method))
#qqnorm(summ.s2p$residuals)
#summ.s2p
# anova(fit.s2p)
```
```{r, echo=FALSE,fig.height=2.5}
v = visreg(fit.s2p, "logDifficulty", by="m_", jitter=TRUE, gg=TRUE, overlay=TRUE) + theme_bw()
plot(v)
```
```{r, echo=FALSE,fig.height=3}
#pdf("q3p.pdf", width=5, height=3)
interplot(fit.s2p, var1 = "m_", var2 = "logDifficulty", xmin=1.4, ci=ci) +
theme_bw() +
ylab("Est. log Task Time Relative to Standard") +
xlab("log(Difficulty)") +
ggtitle("Methods Comparison for Computer") +
geom_hline(yintercept = 0, linetype = "dashed")
#dev.off()
```
On computers, our results are:
\begin{itemize}
\item Rub Zoom performs significantly slower than the standard method across the board, especially for high distances.
\item Dynamic Zoom performs significantly slower for easier tasks, and we do not know either way about higher difficulties.
\item Hold Zoom and Double Click Zoom are borderline significantly slower for higher difficulties.
\item For Brush zooming with scroll pan, results are promising for low distances because of lower expected task time, but the result is borderline insignificant. In general, no significant results found for brush zoom in with scroll pan and double click zoom out.
\end{itemize}
### Mobile
```{r, echo=FALSE}
#data.methods.m$D<-as.factor(round(data.methods.m$difficulty,digits = 1))
data.s2m$m = substr(data.s2m$method, 16, 100)
data.s2m$m[data.s2m$m == "normalpan-pinchzoom"]<-"Standard"
data.s2m$m[data.s2m$m == "normalpan-dblclickzoomin-holdzoomout"] = "2x click"
data.s2m$m[data.s2m$m == "normalpan-dynamiczoomxadjustable"] = "dynamic"
data.s2m$m[data.s2m$m == "normalpan-holdzoomin-holdzoomout"] = "hold"
data.s2m$m[data.s2m$m == "normalpan-rubzoominx-rubzoomouty"] = "rub"
data.s2m$m[data.s2m$m == "pinchpan-brushzoomin-dblclickzoomout"] = "brush"
data.s2m$m_ = factor(data.s2m$m)
data.s2m$m_ = relevel(data.s2m$m_, ref="Standard")
#data.s2m = within(data.s2m, shortmethod <- relevel(shortmethod, ref = "Standard"))
fit.s2m<-lmer(data = data.s2m,logy ~ (1|id) + logDifficulty+m_
+ m_ * logDifficulty,REML = FALSE)
summ.s2m<-summary(fit.s2m)
#boxplot(summ.s2m$residuals~as.character(data.s2m$shortmethodF))
#qqnorm(summ.s2m$residuals)
#summ.s2m
```
```{r, echo=FALSE,fig.height=2.5}
v = visreg(fit.s2m, "logDifficulty", by="m_", jitter=TRUE, gg=TRUE, overlay=TRUE) + theme_bw()
plot(v)
```
```{r, echo=FALSE,fig.height=3}
#pdf("q3m.pdf", width=5, height=3)
interplot(fit.s2m, var1 = "m_", var2 = "logDifficulty", xmin=1.4, ci=ci) +
theme_bw() +
ylab("Est. log Task Time Relative to Standard") +
xlab("log(Difficulty)") +
ggtitle("Methods Comparison for Mobile") +
geom_hline(yintercept = 0, linetype = "dashed")
#dev.off()
```
On mobile phones, our results are:
\begin{itemize}
\item Rub Zoom performs significantly slower than the standard method across the board.
\item Dynamic Zoom performs significantly slower for low difficulty tasks.
\item Hold Zoom and Double Click Zoom are significantly slower for high difficulty tasks.
\item Brush Zoom with two finger pan is significantly slower than the standard method for navigation primarily involving pan.
\end{itemize}