Conversation
|
Thanks @tusharjamunkar, solid work so far. Before merge, please in this PR:
Once those land and CI is green, good to merge I think, will review again before merge <3 |
|
Hi @rosspeili, thanks for the review! I have addressed both items in commit \7cee580:
Verification:
Ready for your review and merge! |
7cee580 to
4a863c7
Compare
|
Hi @rosspeili, I have addressed both string hygiene tweaks (ASCII ->\ in telemetry and -\ in docstring, trailing newline in \confusables.json) and rebased cleanly onto \main\ resolving the \CHANGELOG.md\ conflict. All local tests, black, and flake8 are passing. Ready for merge! |
|
This LGTM now @tusharjamunkar and addresses #273 cleanly, will merge after a short rebase and ensure your changelog lines are under the unreleased. Tiny optional follow-up: |
) - Add evasion detection: leetspeak deobfuscation, ROT13 and token-reversal decoding, typoglycemia scrambled-token detection, mixed-script homoglyphs, and markdown/HTML image exfiltration channels - Refine false-positive control: mention-vs-use detection preserves benign security research and advisory prose - Add operator policy telemetry: policy_action (allow/flag/block), removed_span_count, sanitized_length_delta, decode_chain, decoded_preview - Enforce DoS soft limits: text length (100k chars) and candidate caps with safe fail-closed semantics - Add benign/adversarial test fixtures and 36 unit tests covering evasion vectors and telemetry - Update manifest, instructions, documentation catalog, and demo script
… trailing newline
4a863c7 to
3b23d64
Compare
|
Hi @rosspeili, thanks for the review! I've rebased cleanly onto the latest main, ensured all v0.2.0 changelog entries are properly placed under [Unreleased] (following the v0.5.6 release), and fixed the |
|
LGTM @tusharjamunkar, merging. Thanks for this <3 Closes #273 |
2 participants
Summary
Resolves #273
Upgrades
security/prompt_injection_firewallto v0.2.0, introducing an evasion detection engine, enhanced mention-vs-use false-positive controls for security research, prescriptive operator policy telemetry, and DoS resource caps—while strictly adhering to the Skillware constitution of zero network calls, zero cloud dependencies, zero external libraries (requirements: []), and zero auditing models.Key Capabilities Added
Evasion Detection Engine (
firewall.py):_detect_leetspeak): Unfolds numeric/symbolic character substitutions (0->o,1->i/l,3->e,4->a,5->s,7->t,8->b,@->a,$->s,!->i) and matches normalized tokens against high-signal override lexicons._detect_encoded_payload): Added ROT13 decoding and token-reversal heuristics (snoitcurtsni suoiverp erongi) across word sequences._detect_typoglycemia): Detects interior-scrambled anagrams of critical keywords (ignroe,sysetm,disreagrd,overrdie,exflitrate,passwrod,propmt,secert) while verifying exact boundary characters._detect_mixed_script): Detects tokens mixing Latin characters with Cyrillic or Greek lookalikes (e.g.iгnore). Added Cyrillic small/capital Ghe (\u0433,\u0413) tokb/confusables.json._detect_exfiltration_channels): Flags markdown image syntax () and HTML<img>tags carrying exfiltration query parameters (?q=,?leak=,?data=,?prompt=,?token=).False-Positive Control (Mention-vs-Use):
DISCOURSE_MARKERS_REto recognize academic papers, security blog posts, CVE advisories, tutorials, and red team analyses (tutorial,advisory,cwe,owasp,mitigation,proof of concept,payload,red team,research,academic).balancedmode, quoted or fenced critical phrases embedded within legitimate research contexts remain safe (is_safe=True,policy_action="flag") without breaking workflows, while still blocking understrict.Operator Telemetry & Policy Actions:
policy_action: Prescriptive directive (allow,flag,block) enabling downstream orchestrators to automate gating.removed_span_countandsanitized_length_delta.decode_chain(tracking transformation steps) anddecoded_preview.is_safe,risk_level, andsanitized_textcontinue unaffected.DoS Soft Limits (Resource Hygiene):
MAX_SOURCE_TEXT_CHARS = 100_000), decode candidate cap (MAX_DECODE_CANDIDATES = 50), and decode byte limits (MAX_DECODE_BYTES = 8192).policy_action="block"andrisk_level="high"if DoS limits are exceeded.Test Fixtures & Expanded Assurance:
fixtures/benign/(security_research_blog.txt,prompt_injection_advisory.md,academic_jailbreak_paper.txt) andfixtures/adversarial/(leetspeak_override.txt,rot13_jailbreak.txt,typoglycemia_attack.txt,markdown_image_exfil.txt,mixed_script_injection.txt).test_skill.py) from 19 to 36 tests covering all new evasion detectors, mention-vs-use rules, DoS limits, and telemetry fields.Documentation & Demo:
manifest.yamlversion to0.2.0with updated output properties schema.instructions.mdwith Layer-1 deployment architecture, evasion coverage, and DoS limits.docs/skills/prompt_injection_firewall.mdanddocs/skills/README.md).examples/prompt_injection_firewall_demo.pywith leetspeak evasion scenario andpolicy_actiondisplay.Verification & Test Coverage
skills/security/prompt_injection_firewall/test_skill.py.tests/skills/security/test_prompt_injection_firewall.py.skills/security/,tests/skills/security/, andtests/test_registry_docs.py.tests/test_examples_smoke.py.python scripts/sync_extras.py --checkOK.examples/prompt_injection_firewall_demo.py.requirements: []), 100% offline, deterministic stdlib-based execution.