Skip to content

Add TweetFeed Lookup expansion/hover module - #898

Merged
adulau merged 1 commit into
MISP:mainfrom
0xDanielLopez:expansion-tweetfeed
Sep 8, 2026
Merged

Add TweetFeed Lookup expansion/hover module#898
adulau merged 1 commit into
MISP:mainfrom
0xDanielLopez:expansion-tweetfeed

Conversation

@0xDanielLopez

Copy link
Copy Markdown
Contributor

What

New expansion + hover module TweetFeed Lookup (misp_modules/modules/expansion/tweetfeed.py).

TweetFeed is a free, CC0 feed of URLs, domains, IPs and MD5/SHA-256 hashes that security researchers share on X/Twitter, extracted every 15 minutes since 2021. MISP already consumes it as a bulk feed; this module adds the per-attribute half: given an ip-src, ip-dst, domain, hostname, url, md5 or sha256 attribute it calls GET https://api.tweetfeed.live/v1/ioc?value=<v> (no API key, 365-day window plus an older archive) and returns misp_standard results.

Mapping

  • each source tweet -> a microblog object (type Twitter, url, username parsed from the tweet URL, one hashtag per tag, creation-date when unambiguous), referenced to the queried attribute with mentions;
  • IOCs co-reported in the same tweet -> attributes (to_ids false) referenced from the microblog object with mentions;
  • domain registration / hosting (reg) -> a domain-ip object (apex, IPs, registration-date, registrar / NS / ASN / age in text);
  • IP network metadata, cross-feed corroboration (URLhaus, ThreatFox, MalwareBazaar, USOM, IPsum), AI-generated context and campaign membership -> text / link attributes with disable_correlation;
  • reports older than 365 days -> microblog objects built from the archived tweet id.

Deliberately no tag and no confidence score are set from this data: it is community reported and not independently verified. The AI summary is labelled as LLM-generated in the attribute comment.

Tests and docs

  • tests/test_tweetfeed.py: 13 pytest cases that mock requests.get with verbatim captures of the live API (domain, url, ip, sha256, md5, corroborated, archive-only, miss, error paths, hover call without uuid, introspection/version). Same style as tests/test_xposedornot.py.
  • flake8 clean at the project's 120-column limit.
  • README.md bullet and the documentation/README.md / documentation/mkdocs/expansion.md blocks were added by hand following the generator layout; happy to regenerate if you prefer.

Note for reviewers

While writing the tests I found that adding an object with misp_event.add_object(**obj) drops the references added earlier with obj.add_reference(...): a standalone MISPObject does not serialise ObjectReference, so the unpacked dict never carries them. Passing the instance (misp_event.add_object(obj)) keeps them. This module uses the instance form; other modules that use the **obj form after add_reference may be affected.

Author: Daniel López (TweetFeed maintainer).

Queries https://api.tweetfeed.live/v1/ioc (no API key) for an ip-src/ip-dst/
domain/hostname/url/md5/sha256 attribute and returns misp_standard results:
microblog objects for the source tweets (reporter, hashtags, first/last
seen), co-reported IOCs as attributes referenced with 'mentions', a
domain-ip object for registration/hosting, and text/link attributes for AI
context, IP network metadata, cross-feed corroboration and campaign
membership. Nothing is tagged and no confidence score is asserted: the data
is community reported and unverified.

Tests mock requests.get with verbatim API captures (13 cases). Docs blocks
added by hand following the generator layout.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@adulau
adulau merged commit f32eedc into MISP:main Sep 8, 2026
13 of 14 checks passed
@adulau

adulau commented Sep 8, 2026

Copy link
Copy Markdown
Member

Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants