Essay prompt: In no more than 500 words, propose a brief research plan to address a question of interest within the social sciences.
As chatbots increasingly replace traditional search engines, their single responses limit exposure to multiple perspectives, thereby heightening their influence on public opinion. Authoritarian regimes like Russia and China, which censor media and search engines, now face challenges controlling chatbots trained on vast datasets. This research asks: to what extent do Russian and Chinese LLMs reflect state-aligned biases on politically sensitive issues specific to these countries, compared to Western counterparts? Moreover, it aims to investigate how developers’ approaches to state-imposed guidelines may manifest in chatbots’ responses, revealing potential tensions between compliance and resistance.
Digital sovereignty and internet censorship are well-studied in the context of authoritarian states (Chander & Sun, 2023). Research has explored cultural and political biases in ChatGPT and similar models (Rutinowski et al., 2024). ChatGPT often promotes American values, as shown by the Hofstede Culture Survey (Arora et al., 2023; Cao et al., 2023). A political compass analysis of 14 LLMs revealed biases from left-libertarian to right-authoritarian (Feng et al., 2023). Makhortykh et al. (2024) found that over a quarter of chatbots spread misinformation about the war in Ukraine. Most studies focus on ChatGPT or Copilot, leaving Russian and Chinese chatbots understudied.
The research examines three chatbots: GigaChat, by Russia’s state-owned bank Sber, ERNIE by China’s Baidu, and ChatGPT, which serves as a control to compare content generation across different political contexts. State-influenced LLMs often avoid sensitive topics like Tiananmen or the Russian-Ukrainian conflict. To assess biases, open-ended questions and a custom questionnaire in each chatbot’s native language will be used. Since the Political Compass primarily reflects Western political systems, this study will adapt it to Russian and Chinese contexts, emphasising dimensions like Authoritarianism—Libertarianism and Nationalism—Globalism. Closed-ended questions will be reformulated as mask-filling tasks. Open-ended questions will cover topics like digital privacy, AI, cryptocurrency, sanctions, and the war in Ukraine. Furthermore, the research will explore GigaChat developers’ motivations and the restrictions they navigate, through anonymous interviews where ethically and practically feasible.
LLM responses will be collected via APIs and parsing libraries. Closed-ended questions will be scored to measure each chatbot’s political alignment. Open-ended responses will be compared against state-backed media and ChatGPT outputs, utilising spaCy for keyword parsing and Gensim for Latent Semantic Analysis to assess semantic similarity. This will identify state-approved narratives and overlaps with Western biases. The study will also examine how chatbots handle restricted questions, including refusal to answer, deflections, and generic responses. Qualitative analysis of interviews with developers will further illuminate the mechanisms of state control.
This research focuses on three major chatbots for feasibility. With more computational resources, it could include additional chatbots like China’s Spark, Qwen, and Russia’s YandexGPT for a broader study. Investigating whether restricted questions differ between chatbots in the same country would reveal how control is exercised across various organisations. Analysing these chatbots’ alignment with government views will contribute to the discourse on state control and public opinion manipulation, potentially guiding policy and AI development standards.
