With nearly 2.5m users, onion services have become the prominent part of the darkweb. Over the last five years alone, the number of onion domains has increased 20x, reaching more than 700k unique domains in January 2022. As onion services host various types of illicit content, they have become a valuable resource for darkweb research and an integral part of e-crime investigation and threat intelligence. However, this content is largely un-indexed by today's search engines and researchers have to rely on outdated or manually-collected datasets that are limited in scale, scope, or both. To tackle this problem, we built Dizzy: An open-source crawling and analysis system for onion services. Dizzy implements novel techniques to explore, update, check, and classify hidden services at scale, without overwhelming the Tor network. We deployed Dizzy in April 2021 and used it to analyze more than 63.3m crawled onion webpages, focusing on domain operations, web content, cryptocurrency usage, and web graph. Our main findings show that onion services are unreliable due to their high churn rate, have a relatively small number of reachable domains that are often similar and illicit, enjoy a growing underground cryptocurrency economy, and have a topologically different graph structure than the regular web.
翻译:洋葱服务近2.5米用户,洋葱服务已成为暗网的突出部分。仅在过去5年中,洋葱域数就增加了20x20x,在2022年1月达到700公里的独特域。洋葱服务拥有各种非法内容,因此成为暗网研究的宝贵资源,成为电子犯罪调查和威胁情报的一个组成部分。然而,这一内容基本上没有被今天的搜索引擎和研究人员所依赖的过时或人工收集的数据集作为索引,这些数据集在规模、范围或两者之间都有限。为了解决这个问题,我们建立了 Dizzy:一个面向洋葱服务的开放源爬行和分析系统。Dizy使用新技术,探索、更新、检查和分类隐藏服务的规模,而没有压倒Tor网络。我们于2021年4月部署了Dizy,并用它来分析超过63.3米的在线网页,重点是域业务、网络内容、加密货币使用和网络图。我们的主要调查结果显示,由于它们的高螺旋速度,因此洋葱服务不可靠:一个开放来源的爬行和分析系统;一个相对小的地下和不相近乎常规的货币结构的地层,其最接近于地下的地形。