Method and Weighting Procedures
Methodology Overview, Weighting Variables, Weighted Results: an Example, Data Collection via Online Survey
Methodology Overview
The Research Ethics Office (Institutional Review Board) at the Université du Québec à Montréal (previously at Denver University) has determined that the project meets the standards to protect respondents and their confidentiality.
Since 2013, the BRC Census team has collected data in two phases: a random sample during the event and a more comprehensive online survey after the event. During Burning Man, Census volunteers conduct the random sample by administering a short socio-demographic survey to randomly selected Burners who agree to participate.
After the event, announcements about the online survey are sent out via email lists, social media, the Burning Man website, etc. Some Burners participate but others do not, which introduces a self-selection bias in the survey results. To correct for this bias, the online survey results are adjusted (or “weighted”) based on the results of the random sample.
Weighting Variables
Eight variables are collected during the random sample. These are used later to weight the results of the online survey:
- Day of arrival
- Age
- Virgin Burner or not
- English as a first language or not
- Gender
- US resident or not
- Voting behavior (if eligible to vote in the US)
- US party affiliation (if eligible to vote in the US)
It is the goal of the BRC Census team to randomly sample 5-10% of the incoming population each year.
Weighted Results: an Example
In 2018, 69.6% of survey respondents were non-virgins (that is, 2018 was not their first time in BRC). This was higher than the percentage of non-virgins observed in the random sample (68.3%). This means that non-virgins were "over-represented” (and virgins "under-represented”) in the raw results of the online survey.
In other words, the random sample results show that the raw survey results are not an accurate representation of the overall BRC population:
Burning Man Virgin in 2018?
Error bars indicate the upper and lower bounds of the 95% confidence intervals. (See the next page for more information about error bars.)
| Virgin | Non-Virgin | ||||
|---|---|---|---|---|---|
| Percent | MoE +/- % | Percent | MoE +/- % | ||
| Un-Weighted | 30.4 | 1.0 | 69.6 | 1.0 | |
| Weighted | 31.7 | 1.1 | 68.3 | 1.1 | |
Data Collection via Online Survey
From 2013 to 2018, the number of submitted surveys (and the associated survey sampling rate) were:
| 2013 | 2014 | 2015 | 2016 | 2017 | 2018 | |
|---|---|---|---|---|---|---|
| Total Number of Surveys | 11,919 | 11,676 | 9,585 | 7,137 | 9,168 | 8,745 |
| Survey Sampling Rate | 17% | 18% | 14% | 11% | 13% | 12% |
This "survey sampling rate" indicates the percentage of the BRC population who completed the online survey in a given year based on the population count reported in each annual Burning Man report.
While the survey sampling rate has fluctuated over the years, the sampling rate is consistently high enough to allow for year-to-year comparisons using the methodology described here. As a comparison, the American Community Survey (the largest population survey in the USA) has a sampling rate of approximately 2.5% (National Research Council, 2007).
The actual “response rate” for the online survey (i.e., the number of participants divided by the number of participants who were aware of the survey) cannot be calculated, as there is no way to know how many BRC citizens were reached during our promotion of the online survey.