4000 survey responses – geographical distribution and the need for translation

Last week we silently passed the 4,000 responses mark on our survey. With the summer season waning it seems a good moment to look at where we stand. The survey has been running for 15 weeks, with another 23 weeks to go. We’re glad to have 4,000 responses, but they are not nearly enough to allow for detailed analyses, e.g. by field and country. We would like to see that number double or triple before the survey ends on February 10, 2016. And what is perhaps more important: we would like to see a more or less even global distribution.

A self-selected non-probability sample as the one we work with is bound to have a lot of biases in the response, due to uneven distribution and uptake across groups and countries. The levels of survey uptake in countries is probably affected by:

• (Effect of) distribution and promotion actions
• Propensity of people in a certain country to take surveys
• Degree to which a survey on research tools is considered relevant or interesting
• Ability of targets groups to understand the survey, largely due to differences in (foreign) language proficiency

Response levels per 100 billion US$ GDP at August 22, 2015, weighted average = 5,1

This map shows the geographical variation in uptake of our survey. To make things comparable we need to use relative numbers of responses. Ideally we’d have them relative to the number of researchers in each country. However those figures are not available for most countries. Instead we use GDP of 2013/2014 (Worldbank data) as a proxy as we expect countries to have more active researchers if their economy is larger.

The map shows response levels at or above average (green) in many countries in Europe, Oceania and Canada. Uptake in Russia, Latin America and South Asia is below average (orange/yellow). Despite many responses from the US, that country is also still slightly below average with 4.53. Levels in many countries in East Asia, the Arab World and Africa are very low (red) or even zero (white).

As said, many factors come into play, but it seems obvious that to increase levels outside Europe and Anglo-Saxon countries, translation into a few world languages would help. To find out which languages are the most important for us, we calculated for all language areas the number of responses needed to get below average country levels to the average, relative to their GDP:

language	responses needed to get to average
Chinese, simplified	484
Japanese	170
Arabic	124
Spanish	104
Portuguese	82
French	68
Korean	59
Russian	56
Bahasa Indonesia	43

This means that we are now working towards having the survey and some other texts translated into …

• simplified Chinese
• Japanese
• Arabic
• Spanish
• French
• Russian

whereas we hope to increase uptake in Brazil, Korea and Indonesia by partnering with local institutions to distribute the English version of the survey.

We are looking for support in reviewing, testing and distributing the translations in these six languages. If you have any ideas or contacts that might be helpful for that, please let us know!