commoncrawl/commoncrawl
109 TB
AI & ML interests
Crawled data and metadata
Recent Activity
Organization Card
Common Crawl
Welcome to the Common Crawl Foundation's Hugging Face page!
We aim to provide metadata and experimental versions of our latest data products here.
Useful Links
- Common Crawl's official website
- Our existing statistics webpages (GitHub repo)
- AWS infrastructure status page
Datasets
Explore our datasets hosted on Hugging Face:
- Common Crawl Citations
- Common Crawl Citations, Annotated
- Common Crawl Statistics
- EOT 2024 Host-Level Logs (only available to EOT collaborators)
We look forward to supporting the research and development community with these resources.
models 0
None public yet
datasets 7
commoncrawl/statistics
Viewer • Updated • 626k • 364 • 26
commoncrawl/citations
Viewer • Updated • 9.18k • 80 • 2
commoncrawl/CommonLID
Viewer • Updated • 373k • 162 • 52
commoncrawl/gneissweb-annotation-host-testing-v1
Viewer • Updated • 617M • 106
commoncrawl/gneissweb-annotation-url-testing-v1
Viewer • Updated • 11.5B • 92
commoncrawl/host-index-testing-v2
Preview • Updated • 1.45k
commoncrawl/eot2024_hostlevel_logs
Viewer • Updated • 271k • 5 • 1