This contribution is based on a presentation given at The Digital Orientalist’s Virtual Conference 2025 (AI and the Digital Humanities) by Hong-Yu Hsien. The recording of the presentation can be found here.
Introduction
How can “sycophantic” AI help with interpretations of literary Chinese?
Wang Huiluan 汪噦鸞, courtesy name Shucheng 書城 (birth and death dates unknown), was my great-grandfather. He received the rank of jieyuan 解元, first place candidate of provincial imperial examination (keju 科舉), in 1902 in Hubei. Over the course of his life, he transformed from a loyal Qing dynasty official into a revolutionary alongside Sun Yat-sen 孫中山 (1866–1925), the founder of Republic China. In this paper, I will compare my reading of his essay with AI’s analysis, concentrating on the issue of the interpretation of the specific grammatical structureheyi 何以 (“by what means”).
Research question
Wang Shucheng wrote eight essays discussing many government policy topics for the examination. His essay, “The Grand Minister uses the method of land accounting to categorize the living things of the five types of land” 大司徒以土會之法辨五地之物生論 discusses how land in China should be categorized for agricultural production. I have analyzed his essay in order to see if the revolutionary ideas he developed later in life can be traced to his early works. My primary goal was to answer the following questions: 1) Does Wang Shucheng express his revolutionary ideas in his essay through heyi 何以? 2) How can AI intervene productively in my interpretations of heyi 何以?
In the process of my investigation, I found an approach that answered these questions about my great-grandfather and AI which I will call Bias Cancelling.
Method
The methodology I call Bias Cancelling emerged from my experimental process. It involved the following steps:
First, I conducted research myself by transcribing and translating the text. With my interpretation, I concluded that Wang was, in fact, questioning the traditional land categories listed in the Rites of Zhou (Zhouli 周禮), which was a work on institution and politics that was a part of the Confucian canon. However, a conclusion that a Qing dynasty scholar would oppose a Confucian classic in the examination is radical and needs caution to be made. If I had accepted this conclusion, it would overturn how we think about late-Qing provincial examinations in Hubei. Since Wang Huiluan successfully passed, we could infer that the examiner approved of or was tolerant towards revolutionary thought and progressive thinking if Wang was confident enough to risk this opportunity.
Next, to validate my assumption, I treated my conclusions as hypotheses that I could double-check with AI, by prompting GPT-4-Turbo to analyze the essay text the same way I did. The AI produced ambivalent results, with different prompts giving different interpretations. Generative AI never gives the same response twice, so, among the responses, I received some that questioned my conclusions; the doubt prompted me to reassess. While the unpredictability of AI was beneficial as it pushed me to reconsider my conclusions, it also acted as a hard warning on the use of AI in research. The AI canceled my bias about Wang’s dissent, and I used my judgement to override the sycophancy of AI. This method could be called Bias Cancelling. Bias Cancelling utilizes the unpredictable results of AI to show new perspectives of a problem.
Finally, Bias Cancelling assisted in my process of understanding Wang Shucheng’s essay, pushing me to look for more evidence in the essay and secondary sources, which led to my new, more well-considered conclusion.
Research
When I was reading Wang’s essay, a particular passage that makes “generalizing statements” about agriculture (marked by the particle fu 夫) caught my attention. These statements are based on the five types of classified land that were defined earlier in the text. “Thus when farm working, if it is a forest, if it is a swamp, if it is a mountain, if it is a flatland, if it is a wetland, those are the five types of land…” (則在稼穡，若夫山林也川澤也邱陵也墳衍也原隰也之五地也). Wang uses almost the same phrasing to that found in the Rites of Zhou, which has a paragraph that begins: “Using the method of land accounting to distinguish the living things of the five types of land “以土會之法，辨五地之物生。” ( Diguan Situ 地官‧司徒 chapter of the Rites of Zhou). Both the Rites of Zhou and Wang’s essay list animals and plants corresponding to each type of land. Here is how they compare, according to my original interpretation of the essay:
Wang Shucheng discusses all the categories using the following sentence structure: “I am going to make a generalizing statement that I distinguish…animals by what means…plants by what means” (辨夫…動物何以…植物何以…也). For example:
“I am going to make a generalizing statement that I distinguish, in the land of swamps, the animals by what means are considered to have scales, and the plants by what means are considered to be good for oil.”
辨夫川澤之地。動物何以宜麟。植物何以宜膏也。
Although Wang is using the Rites of Zhou categories, he appears to question this framework with the term heyi 何以 (interpreted as ‘by what means’). Furthermore, he uses the same parallel structure to make similar statements questioning the other types of land. Later in the text, the scholar possibly doubts the Rites of Zhou using a second sentence structure, as follows:
“I am going to make a generalizing statement: If I distinguish that the animals and plants of flatlands near water by what means are not appropriate for swamps, then we know the animals generally are suited to have exoskeletons, and the plants are suited to have pods.”
辨夫墳衍之動植物何以不宜於川澤。則凡宜介宜莢者可知也。
In my original interpretation, the essay states that the animals and plant characteristics of the flatlands near water are more appropriate for the swamp. He also makes the same statements for the other lands. As mentioned before, the characteristics the author uses are all from the Rites of Zhou, as well as the categorization he presumably challenges. And if this reading is correct, not only did he directly challenge the Confucian classic, but he was also awarded first place at the traditionally conservative state examination.
In my attempt to validate my conclusion, I queried GPT to see how AI approaches this analysis. This is how AI responded:
As you can see, AI thinks that Wang is agreeing with the Rites of Zhou. That made me realize that rather than expressing doubt, heyi refers to the characteristics that make one set of plants and animals not suited to another ecosystem. Initially, I interpreted heyi as an interrogative, implying a question such as, “By what means are the plants appropriate?” However, after my exchange with ChatGPT, I now read heyi as a part of a subordinate clause introduced by ze 則, as in “If we know by what means the plants are suited, then…” After careful reconsideration, AI’s suggestion that Wang was confirming the Rites of Zhou made more sense, especially given that further evidence in the essay itself and secondary sources support this reading.
A section later in the essay gives us clues to Wang’s stand on the Rites of Zhou:
Refined due to following established principles, and skillful due to innovation. They use skill to make it meticulous, and when they are skillful they have a lot of work. This comes from the Rites of Zhou. But my China does not know how to preserve it. France steals China’s methods. And the country’s growing of vegetables allow farmers to receive a hundred gold in a farm year. The American model used this method. The country’s growing of wood can allow a year’s harvest (of American timber) to be among multiple thousand and ten-thousands. Alas! The Rites of Zhou have one place of opportunity.
因故精、創故巧。巧以精巧之業多。出於周官。而我中國莫知省。法竊其術。而其國之種蔬也。一瞻歲可獲百金。美師其法。而其國之材木也。歲率數千、萬。嘻乎。周官有一隅之地。
In this section of text, Wang describes how the methods in the Rites of Zhou are not preserved in China, while other countries are using them to prosper. The descriptions of prosperity in foreign countries give us insight on Wang’s opinion on the Rites’ teachings. Wang thinks the Rites’ methods are, in fact, useful for land cultivation and a shame to be lost. Furthermore, this reading aligns with the historical context and the reputation of the imperial examinations for requiring conformity with tradition:
As this quote shows, imperial examination essays rarely contained the author’s own opinion. Rather, they were written to conform to accepted thoughts.
Conclusion
My whole research process demonstrates what I call Bias Cancelling. Humans tend to have biases when making decisions and interpreting texts. Since I had Wang Shucheng’s revolutionary future in mind, I read his essay as revolutionary too, which is a case of confirmation bias. Similarly, AI models are shaped by the data they are trained on, which can introduce bias into their responses as well. But balancing biases from both sides can aid us in making more circumspect conclusions. That said, AI agreed with my original conclusion when I asked the question another way. Why is that? Like I stated above, AI is biased too. A lot of generative AIs are sycophants; they tend to agree with the opinions contained in a prompt. In Bias Cancelling, we are trying to mitigate sycophancy while taking advantage of AI’s capacity to offer alternative analyses. A careful prompt crafting that avoids injecting one’s own opinion then can show different perspectives of the question that one may not have previously considered, which could make or break the original interpretation. Since AI can introduce alternative opinions, while humans are better equipped to verify the information, there is potential in using LLMs as a collaboration tool to cancel biases. In my research, I countered my own starting bias with the bias in AI LLM models.
In conclusion, AI assisted us to find that Wang Shucheng confirms the Rites of Zhou in his essay “The Grand Minister uses the method of land accounting to categorize the living things of the five types of land” 大司徒以土會之法辨五地之物生論. My new conclusion was reached by disproving a false hypothesis using AI tools, seeking quotations within the essay, and referencing research literature. AI’s unpredictable interpretations, which could be both sycophantic and alternative to mine, helped me avoid a false conclusion in this case. AI therefore can be useful in cases such as mine. Opposed to the view of AI as an unbridled tool used as a shortcut, this experience also suggests that AI can be further applied in research and education as a dialogue tool that contains a lot of unrealized potential for students and researchers.
Acknowledgments
I sincerely thank Michael Berson, J Phillips, Taiwo Togun, Sofia Gans and Mariana Zorkina for their constructive feedback and valuable guidance during my research and writing processes.
Sources
Ctext.org. 2025. “The Rites of Zhou: Diguan situ.” Chinese Text Project. https://ctext.org/rites-of-zhou/di-guan-si-tu.
O’Sullivan, Barry, and Liying Cheng. “Lessons from the Chinese imperial examination system.” Language Testing in Asia 12, 52 (2022). https://doi.org/10.1186/s40468-022-00201-5
ChatGPT, Open AI. https://chat.openai.com/chat.
Zhisheng chunwei 直省闈墨. 1904?
