magma

# OSINT sources

The following is a list of free, Open Source Intelligence (OSINT) resources that can be used to quickly and easily search IPs, domains, file hashes, and URLs. For user convenience, the sources are grouped by category (where appropriate, an explanatory parenthetical is provided to detail what the source can be used to accomplish).

This collection of information is gathered from public or open sources and can help you assess an activity and find background information related to networks, ISPs, IP addresses, domain names, network tools from various vantage points (such as traceroute, ping, and other connectivity tests), information about URLs, BGP, Whois and other services primarily related to Internet services.

# BGP

  • BGPView - Allows users to debug and investigate information pertaining to IP addresses, ASN, IXs, BGP, ISPs, prefixes, and domain names.

# Domains

  • Alexa Siteinfo - A site overview and analysis tool by Alexa.
  • BlueCoat Domain - BlueCoat's web filtering domain name categorization.
  • Censys - Get Censys data for a domain name such as open ports, DNS configuration, TLS certificate information.
  • FortiGuard Labs - FortiGuard Labs OSINT domain report.
  • MX Toolbox - MX statistics, IP/host reputation, DNS, blacklist and SMTP diagnostics.
  • Onyphe - A search engine for OSINT collected by crawling various sources available on the Internet.
  • Pulsedive - Analyze a domain for threat intelligence sources and other OSINT data.
  • SecurityTrails - Historical DNS data.
  • Shodan - OSINT search engine of Internet connected devices.
  • Talos
    • IP and domain reputation lookup.
  • ThreatCrowd - OSINT data aggregation by AlienVault OTX, VirusTotal, Malwr and others.
  • ThreatMiner - OSINT thread intelligence portal.
  • Tor Metrics: Relay Search - The relay search tool displays data about single relays and bridges in the Tor network.
  • VirusTotal - URL/domain blacklisting OSINT data.
  • IBM X-Force Exchange - Threat intelligence sharing platform of IPs, domains, URLs and applications.

# DNS

  • DNSdumpster - Allows users to conduct DNS recon and research, and also find and lookup DNS records.
  • DNS Propagation Checker - Allows users to check DNS of a domain name from multiple DNS nameservers and resolvers around the world.
  • ViewDNS - Allows users to gather a large amount of data about a particular website or IP address.
  • DNS Census 2013 - A public dataset of registered domains and DNS records that contains approximately 2.5 billion DNS records gathered from 2012-2013.
  • Domains Index datasets - A list of zone files available for free download.
  • DNSApe - A collection of (mainly) DNS and other network tools.

# IPs

# Looking glasses

A looking glass is a publicly accessible (route) server that allows you to perform various networking tests (such as ping, traceroute and BGP route view) to a specific destination or network. This information can help to troubleshoot routing problems and detect network connectivity issues that may be a result of network blocking.

  • Looking.house - Looking Glass points from various companies and countries.
  • LookingGlass.Org - A collection of looking glass services around the world from different countries, ISPs and IXs.

# Network diagnostic tools

  • Ping.pe - Ping, MTR, dig and TCP port check from multiple locations.
  • CyberChef - Multiple networking and other tools.

# Web archives

  • Archive.today - Allows users to take a "snapshot" of a webpage that will always be online even if the original page disappears or is altered. Information on which parts of a webpage will be saved and other technical details are available here.
  • Internet Archive Wayback Machine - Allows users to capture a webpage as it currently appears for later use as a trusted citation. A newer version of the program supports saving error pages (HTTP Status=4xx, 5xx) and outlinks. Wayback Machine's subscription service, Archive-It, also allows institutions to build and preserve collections of born digital content. Technical details are available here.

# Whois services

# URLs

  • Any.Run - Interactive online malware analysis sandbox.
  • HackerTarget - This tool will parse the HTML of a website and extract links from the page.
  • TrendMicro Site Safety Center URL/Domain reputation database.
  • urlscan.io - A service to scan and analyse websites. An automated process will browse to the URL like a regular user and record the activity that this page navigation creates. This includes the domains and IPs contacted, the resources (JavaScript, CSS, etc) requested from those domains, as well as additional information about the page itself. urlscan.io will take a screenshot of the page, record the DOM content, JavaScript global variables, cookies created by the page, and a myriad of other observations.
  • Zulu - URL Risk Analyser - A dynamic risk scoring engine for web based content.

# Other

  • CloudFlare Watch - Find IPs of CloudFlare hosted websites and services.
  • Digital Attack Map - A map of daily and historical DDoS attacks worldwide.
  • The IP Observatory - A platform that remotely observes, in real time, the quality of the Internet at any location on the globe and then publishes the results of these observations in clear, accessible visualizations.
  • Internet-Wide Scan Data Repository - A public archive of research datasets that describe the hosts and sites on the Internet.
  • Common Crawl - A corpus of web crawl data composed of over 25 billion web pages.
  • GDELT - The Global Database of Events, Language and Tone, a project that "monitors the world's broadcast, print, and web news from nearly every corner of every country in over 100 languages and identifies the people, locations, organizations, counts, themes, sources, emotions, quotes, images and events driving our global society every second of every day".
  • GDELT Global Material Conflict 48-Hour Trend Report - Every day the GDELT Project produces a PDF global trend report detailing changes in Material Conflict across the globe, instantly summarizing the latest developments.

← Improve the magma guide →