smicallef / spiderfoot

查看原文 / View Original 编辑 / Edit

smicallef / spiderfoot

SpiderFoot is an open source intelligence (OSINT) automation tool. It integrates with just about every data source available and utilises a range of methods for data analysis, making that data easy to navigate. SpiderFoot has an embedded web-server for providing a clean and intuitive web-based interface but can also be used completely via the command-line. It’s written in Python 3 and MIT-licensed.

SpiderFoot 是一款开源情报 (OSINT) 自动化工具。它几乎集成了所有可用的数据源，并利用多种方法进行数据分析，使数据易于浏览。SpiderFoot 内置了一个 Web 服务器，提供简洁直观的网页界面，同时也完全支持通过命令行使用。它基于 Python 3 编写，并采用 MIT 许可证。

FEATURES

功能特点

  • Web based UI or CLI
  • 基于 Web 的 UI 或命令行界面 (CLI)
  • Over 200 modules (see below)
  • 超过 200 个模块（见下文）
  • Python 3.7+
  • Python 3.7+
  • YAML-configurable correlation engine with 37 pre-defined rules
  • 可通过 YAML 配置的关联引擎，内置 37 条预定义规则
  • CSV/JSON/GEXF export
  • 支持 CSV/JSON/GEXF 导出
  • API key export/import
  • API 密钥导出/导入
  • SQLite back-end for custom querying
  • 用于自定义查询的 SQLite 后端
  • Highly configurable
  • 高度可配置
  • Fully documented
  • 文档齐全
  • Visualisations
  • 可视化功能
  • TOR integration for dark web searching
  • 集成 TOR 以进行暗网搜索
  • Dockerfile for Docker-based deployments
  • 提供用于 Docker 部署的 Dockerfile
  • Can call other tools like DNSTwist, Whatweb, Nmap and CMSeeK
  • 可调用 DNSTwist、Whatweb、Nmap 和 CMSeeK 等其他工具
  • Actively developed since 2012!
  • 自 2012 年起持续活跃开发！

WANT MORE?

想要更多功能？

Need more from SpiderFoot? Check out SpiderFoot HX for: 需要 SpiderFoot 的更多功能？请查看 SpiderFoot HX，它提供：

  • 100% Cloud-based and managed for you
  • 100% 基于云端并为您托管
  • Attack Surface Monitoring with change notifications by email, REST and Slack
  • 攻击面监控，支持通过电子邮件、REST 和 Slack 发送变更通知
  • Multiple targets per scan
  • 每次扫描支持多个目标
  • Multi-user collaboration
  • 多用户协作
  • Authenticated and 2FA Investigations
  • 经过身份验证和双重认证 (2FA) 的调查
  • Customer support
  • 客户支持
  • Third party tools pre-installed & configured
  • 预装并配置好的第三方工具
  • Drive it with a fully RESTful API
  • 通过完整的 RESTful API 进行驱动
  • TOR integration built-in
  • 内置 TOR 集成
  • Screenshotting
  • 屏幕截图功能
  • Bring your own Python SpiderFoot modules
  • 支持自定义 Python SpiderFoot 模块
  • Feed scan data to Splunk, ElasticSearch and REST endpoints
  • 将扫描数据馈送到 Splunk、ElasticSearch 和 REST 端点

See the full set of differences between SpiderFoot HX and the open source version here. 查看 SpiderFoot HX 与开源版本之间的完整差异，请点击此处。

USES

用途

SpiderFoot can be used offensively (e.g. in a red team exercise or penetration test) for reconnaissance of your target or defensively to gather information about what you or your organisation might have exposed over the Internet. SpiderFoot 可用于进攻性场景（例如红队演习或渗透测试）以侦察目标，也可用于防御性场景，收集您或您的组织在互联网上可能暴露的信息。

You can target the following entities in a SpiderFoot scan: 您可以在 SpiderFoot 扫描中针对以下实体：

  • IP address
  • IP 地址
  • Domain/sub-domain name
  • 域名/子域名
  • Hostname
  • 主机名
  • Network subnet (CIDR)
  • 网络子网 (CIDR)
  • ASN
  • 自治系统号 (ASN)
  • E-mail address
  • 电子邮件地址
  • Phone number
  • 电话号码
  • Username
  • 用户名
  • Person’s name
  • 人名
  • Bitcoin address
  • 比特币地址

SpiderFoot’s 200+ modules feed each other in a publisher/subscriber model to ensure maximum data extraction to do things like: SpiderFoot 的 200 多个模块通过发布者/订阅者模型相互协作，以确保最大程度的数据提取，从而实现以下功能：

  • Host/sub-domain/TLD enumeration/extraction
  • 主机/子域名/顶级域名枚举与提取
  • Email address, phone number and human name extraction
  • 电子邮件地址、电话号码和人名提取
  • Bitcoin and Ethereum address extraction
  • 比特币和以太坊地址提取
  • Check for susceptibility to sub-domain hijacking
  • 检查子域名劫持风险
  • DNS zone transfers
  • DNS 区域传输
  • Threat intelligence and Blacklist queries
  • 威胁情报和黑名单查询
  • API integration with SHODAN, HaveIBeenPwned, GreyNoise, AlienVault, SecurityTrails, etc.
  • 与 SHODAN、HaveIBeenPwned、GreyNoise、AlienVault、SecurityTrails 等平台的 API 集成
  • Social media account enumeration
  • 社交媒体账号枚举
  • S3/Azure/Digitalocean bucket enumeration/scraping
  • S3/Azure/Digitalocean 存储桶枚举/抓取
  • IP geo-location
  • IP 地理位置定位
  • Web scraping, web content analysis
  • 网页抓取、网页内容分析
  • Image, document and binary file meta data analysis
  • 图像、文档和二进制文件元数据分析
  • Dark web searches
  • 暗网搜索
  • Port scanning and banner grabbing
  • 端口扫描和横幅抓取
  • Data breach searches
  • 数据泄露搜索
  • So much more…
  • 以及更多…

INSTALLING & RUNNING

安装与运行

To install and run SpiderFoot, you need at least Python 3.7 and a number of Python libraries which you can install with pip. We recommend you install a packaged release since master will often have bleeding edge features and modules that aren’t fully tested. 要安装和运行 SpiderFoot，您至少需要 Python 3.7 以及一些可以通过 pip 安装的 Python 库。我们建议您安装打包好的发布版本，因为 master 分支通常包含未经充分测试的前沿功能和模块。

Stable build (packaged release): 稳定版本（打包发布）：

wget https://github.com/smicallef/spiderfoot/archive/v4.0.tar.gz
tar zxvf v4.0.tar.gz
cd spiderfoot-4.0
pip3 install -r requirements.txt
python3 ./sf.py -l 127.0.0.1:5001

Development build (cloning git master branch): 开发版本（克隆 git master 分支）：

git clone https://github.com/smicallef/spiderfoot.git
cd spiderfoot
pip3 install -r requirements.txt
python3 ./sf.py -l 127.0.0.1:5001

Check out the documentation and our asciinema videos for more tutorials. 查看文档和我们的 asciinema 视频以获取更多教程。

COMMUNITY

社区

Whether you’re a contributor, user or just curious about SpiderFoot and OSINT in general, we’d love to have you join our community! SpiderFoot now has a Discord server for seeking help from the community, requesting features or just general OSINT chit-chat. 无论您是贡献者、用户，还是仅仅对 SpiderFoot 和 OSINT 感兴趣，我们都欢迎您加入我们的社区！SpiderFoot 现在拥有一个 Discord 服务器，用于寻求社区帮助、请求功能或进行一般的 OSINT 闲聊。

WRITING CORRELATION RULES

编写关联规则

We have a comprehensive write-up and reference of the correlation rule-set introduced in SpiderFoot 4.0 here. Also take a look at the template.yaml file for a walk through. The existing 37 rules are also quite readable and good as starting points for additional rules. 我们在此处提供了 SpiderFoot 4.0 中引入的关联规则集的全面说明和参考。此外，请查看 template.yaml 文件以获取指南。现有的 37 条规则也非常易读，适合作为编写额外规则的起点。

MODULES / INTEGRATIONS

模块/集成

SpiderFoot has over 200 modules, most of which don’t require API keys, and many of those that do require API keys have a free tier. SpiderFoot 拥有超过 200 个模块，其中大多数不需要 API 密钥，而许多需要 API 密钥的模块也提供免费层级。

NameDescriptionType
AbstractAPILook up domain, phone and IP address information from AbstractAPI.Tiered API
Abuse.chCheck if a host/domain, IP address or netblock is malicious according to Abuse.ch.Free API
AbuseIPDBCheck if an IP address is malicious according to AbuseIPDB.com blacklist.Tiered API
Abusix Mail IntelligenceCheck if a netblock or IP address is in the Abusix Mail Intelligence blacklist.Tiered API
Account FinderLook for possible associated accounts on over 500 social and other websites such as Instagram, Reddit, etc.Internal
AdBlock CheckCheck if linked pages would be blocked by AdBlock Plus.Tiered API
AdGuard DNSCheck if a host would be blocked by AdGuard DNS.Free API
AhmiaSearch Tor ‘Ahmia’ search engine for mentions of the target.Free API
AlienVault IP ReputationCheck if an IP or netblock is malicious according to the AlienVault IP Reputation database.Free API
AlienVault OTXObtain information from AlienVault Open Threat Exchange (OTX)Tiered API
Amazon S3 Bucket FinderSearch for potential Amazon S3 buckets associated with the target and attempt to list their contents.Free API
Apple iTunesSearch Apple iTunes for mobile apps.Free API
Archive.orgIdentifies historic versions of interesting files/pages from the Wayback Machine.Free API
ARINQueries ARIN registry for contact information.Free API
Azure Blob FinderSearch for potential Azure blobs associated with the target and attempt to list their contents.Free API
Base64 DecoderIdentify Base64-encoded strings in URLs, often revealing interesting hidden information.Internal
BGPViewObtain network information from BGPView API.Free API
Binary String ExtractorAttempt to identify strings in binary content.Internal
BinaryEdgeObtain information from BinaryEdge.io Internet scanning systems, including breaches, vulnerabilities, torrents and passive DNS.Tiered API
Bing (Shared IPs)Search Bing for hosts sharing the same IP.Tiered API
← 返回新闻列表 / Back to news list