Tutorial

::

Linguistic Experiments with WebExp2

 
[Overview Local setup WE2 Support Design Issues Creating WE2 Experiments Advertising Results & Analysis]
 

Analysis

[TOP]

Random vs. fixed factors

Controlling unbalanced factors

Comparing models: Goodness-of-fit

Simple (and very misleading) measures of such quantitative comparisons of theories are the classification accuracy (in percent) or the amount of variation in the dependent variable that each theory accounts for. This measures can also be used to assess "how good" your hypothesis is (i.e. does it explain a lot of what's going on?). For continuous dependent variables, the r- square, which you will often see in psycholinguistic publications, is a direct measure of the variation accounted for by a hypothesis. NB: Even this measure is highly fragile. This has to do with the fact that (as stated above) you are measuring behavioral correlates/operationalizations of the variables you are really interested in. These correlates often don't come with intrinsically defined scales and any transformation of the scales you measure your variables on, will affect the goodness-of-fit of your model (which is the operationalization of your hypothesis/theory).

For example, you may hypothesize that the overall frequency of the main predicate of a simple transitive sentence like "Tom reads books" may be a good predictor of its acceptability. But maybe the log-frequency would be an (even) better predictor, etc. In sum, be aware that the quality of a theory (operationalized as the goodness-of-fit of the model) depends on the scales you have used to determine the values of the variables/factors involved in the model.

[TOP]

Details

[TOP]

 
[Overview Local setup WE2 Support Design Issues Creating WE2 Experiments Advertising Results & Analysis]