Understanding and Analyzing Appraisal Systems in the Underground Marketplaces

—An appraisal system is a feedback mechanism that has gained popularity in underground marketplaces. This system allows appraisers, who receive free samples from vendors, to provide assessments (i.e., appraisal reviews) for products in underground marketplaces. In this paper, we present the first measurement study on the appraisal system within underground marketplaces. Specifically, from 17M communication traces from eight marketplaces spanning from Feb 2006 to Mar 2023, we discover 56,229 appraisal reviews posted by 18,701 unique ap-praisers. We look into the appraisal review ecosystem, revealing five commonly used requirements and merits in the appraiser selection process. These findings indicate that the appraisal system is a well-established and structured process within the underground marketplace ecosystem. Furthermore, we reveal the presence of high-quality and unique cyber threat intelligence (CTI) in appraisal reviews. For example, we identify the ge-olocations of followers for a social booster and programming languages used for malware. Leveraging our extraction model, which integrates 41 distinct types of CTI, we capture 23,978 artifacts associated with 16,668 (50.2%) appraisal reviews. In contrast, artifacts are found in only 8.9% of listings and 2.7% of non-appraisal reviews. Our study provides valuable insights into this under-explored source of CTI, complementing existing research on threat intelligence gathering.