Leveraging Large Language Models and In-Context Learning for Construct Identification in Computational Social Science: A Case Study on Wearable Devices

Large Language Models (LLMs) have opened new frontiers in Computational Social Science (CSS) by enabling the extraction, classification, and analysis of large-scale unstructured text data. This study aims to leverage LLMs to systematically encode theoretical constructs from user-generated content. We propose an LLM-powered construct identification framework that employs LLMs for automated encoding, validated against human-coded benchmarks. The framework was evaluated as a case study in the domain of wearable devices. Two experiments for binary and ternary encoding were tested. For both experiments, the LLM demonstrated high accuracy, precision, and recall in encoding theoretical constructs of user-generated content. The findings emphasize that LLMs can complement traditional methods in CSS, enabling scalable, efficient, and effective analysis of social phenomena across diverse domains.