Releases: mandiant/capa
v7.0.1
This release fixes a circular import error when using capa as a library.
Bug Fixes
- fix potentially circular import errors #1969 @williballenthin
Raw diffs
v7.0.0
This is the v7.0.0 release of capa which was mainly worked on during the Google Summer of Code (GSoC) 2023. A huge
shoutout to our GSoC contributors @colton-gabertan and @yelhamer for their amazing work. See our blog posts for more details:
Also, a big thanks to the other contributors: @aaronatp, @Aayush-Goel-04, @bkojusner, @doomedraven, @ruppde, @larchchen, @JCoonradt, and @xusheng6.
New Features
- add Ghidra backend #1770 #1767 @colton-gabertan @mike-hunhoff
- add Ghidra UI integration #1734 @colton-gabertan @mike-hunhoff
- add dynamic analysis via CAPE sandbox reports #48 #1535 @yelhamer
- binja: add support for forwarded exports #1646 @xusheng6
- binja: add support for symtab names #1504 @xusheng6
- add com class/interface features #322 @Aayush-Goel-04
- dotnet: emit enclosing class information for nested classes #1780 #1913 @bkojusner @mike-hunhoff
Breaking Changes
- remove the
SCOPE_*
constants in favor of theScope
enum #1764 @williballenthin - protobuf: deprecate
RuleMetadata.scope
in favor ofRuleMetadata.scopes
@williballenthin - protobuf: deprecate
Metadata.analysis
in favor ofMetadata.analysis2
that is dynamic analysis aware @williballenthin - update freeze format to v3, adding support for dynamic analysis @williballenthin
- extractor: ignore DLL name for api features #1815 @mr-tz
- main: introduce wrapping routines within main for working with CLI args #1813 @williballenthin
- move functions from
capa.main
to newcapa.loader
namespace #1821 @williballenthin - proto: add
package
declaration #1960 @larchchen
New Rules (41)
- nursery/get-ntoskrnl-base-address @mr-tz
- host-interaction/network/connectivity/set-tcp-connection-state @johnk3r
- nursery/capture-process-snapshot-data @mr-tz
- collection/network/capture-packets-using-sharppcap jakub.jozwiak@mandiant.com
- nursery/communicate-with-kernel-module-via-netlink-socket-on-linux michael.hunhoff@mandiant.com
- nursery/get-current-pid-on-linux michael.hunhoff@mandiant.com
- nursery/get-file-system-information-on-linux michael.hunhoff@mandiant.com
- nursery/get-password-database-entry-on-linux michael.hunhoff@mandiant.com
- nursery/mark-thread-detached-on-linux michael.hunhoff@mandiant.com
- nursery/persist-via-gnome-autostart-on-linux michael.hunhoff@mandiant.com
- nursery/set-thread-name-on-linux michael.hunhoff@mandiant.com
- load-code/dotnet/load-windows-common-language-runtime michael.hunhoff@mandiant.com blas.kojusner@mandiant.com jakub.jozwiak@mandiant.com
- nursery/log-keystrokes-via-input-method-manager @mr-tz
- nursery/encrypt-data-using-rc4-via-systemfunction032 richard.weiss@mandiant.com
- nursery/add-value-to-global-atom-table @mr-tz
- nursery/enumerate-processes-that-use-resource @Ana06
- host-interaction/process/inject/allocate-or-change-rwx-memory @mr-tz
- lib/allocate-or-change-rw-memory 0x534a@mailbox.org @mr-tz
- lib/change-memory-protection @mr-tz
- anti-analysis/anti-av/patch-antimalware-scan-interface-function jakub.jozwiak@mandiant.com
- executable/dotnet-singlefile/bundled-with-dotnet-single-file-deployment sara.rincon@mandiant.com
- internal/limitation/file/internal-dotnet-single-file-deployment-limitation sara.rincon@mandiant.com
- data-manipulation/encoding/encode-data-using-add-xor-sub-operations jakub.jozwiak@mandiant.com
- nursery/access-camera-in-dotnet-on-android michael.hunhoff@mandiant.com
- nursery/capture-microphone-audio-in-dotnet-on-android michael.hunhoff@mandiant.com
- nursery/capture-screenshot-in-dotnet-on-android michael.hunhoff@mandiant.com
- nursery/check-for-incoming-call-in-dotnet-on-android michael.hunhoff@mandiant.com
- nursery/check-for-outgoing-call-in-dotnet-on-android michael.hunhoff@mandiant.com
- nursery/compiled-with-xamarin michael.hunhoff@mandiant.com
- nursery/get-os-version-in-dotnet-on-android michael.hunhoff@mandiant.com
- data-manipulation/compression/create-cabinet-on-windows michael.hunhoff@mandiant.com jakub.jozwiak@mandiant.com
- data-manipulation/compression/extract-cabinet-on-windows jakub.jozwiak@mandiant.com
- lib/create-file-decompression-interface-context-on-windows jakub.jozwiak@mandiant.com
- nursery/enumerate-files-in-dotnet moritz.raabe@mandiant.com anushka.virgaonkar@mandiant.com
- nursery/get-mac-address-in-dotnet moritz.raabe@mandiant.com michael.hunhoff@mandiant.com echernofsky@google.com
- nursery/get-current-process-command-line william.ballenthin@mandiant.com
- nursery/get-current-process-file-path william.ballenthin@mandiant.com
- nursery/hook-routines-via-dlsym-rtld_next william.ballenthin@mandiant.com
- nursery/linked-against-hp-socket still@teamt5.org
- host-interaction/process/inject/process-ghostly-hollowing sara.rincon@mandiant.com
Bug Fixes
- ghidra: fix
ints_to_bytes
performance #1761 @mike-hunhoff - binja: improve function call site detection @xusheng6
- binja: use
binaryninja.load
to open files @xusheng6 - binja: bump binja version to 3.5 #1789 @xusheng6
- elf: better detect ELF OS via GCC .ident directives #1928 @williballenthin
- elf: better detect ELF OS via Android dependencies #1947 @williballenthin
- fix setuptools package discovery #1886 @gmacon @mr-tz
- remove unnecessary scripts/vivisect-py2-vs-py3.sh file #1949 @JCoonradt
capa explorer IDA Pro plugin
- various integration updates and minor bug fixes
Development
Developer Notes
With this new release, many classes and concepts have been split up into static (mostly identical to the
prior implementations) and dynamic ones. For example, the legacy FeatureExtractor class has been renamed to
StaticFeatureExtractor and the DynamicFeatureExtractor has been added.
Starting from version 7.0, we have moved the component responsible for feature extractor from main to a new
capabilities' module. Now, users wishing to utilize capa’s feature extraction abilities should use that module instead
of importing the relevant logic from the main file.
For sandbox-based feature extractors, we are using Pydantic models. Contributions of more models for other sandboxes
are very welcome!
With this release we've reorganized the logic found in main()
to localize logic and ease readability and ease changes
and integrations. The new "main routines" are expected to be used only within main functions, either capa main or
related scripts. These functions should not be invoked from library code.
Beyond copying code around, we've refined the handling of the input file/format/backend. The logic for picking the
format and backend is more consistent. We've documented that the input file is not necessarily the sample itself
(cape/freeze/etc.) inputs are not actually the sample.
Raw diffs
v7.0.0-beta
This is the beta release of capa v7.0 which was mainly worked on during the Google Summer of Code (GSoC) 2023. A huge
shoutout to @colton-gabertan and @yelhamer for their amazing work.
Also a big thanks to the other contributors: @aaronatp, @Aayush-Goel-04, @bkojusner, @doomedraven, @ruppde, and @xusheng6.
New Features
- add Ghidra backend #1770 #1767 @colton-gabertan @mike-hunhoff
- add dynamic analysis via CAPE sandbox reports #48 #1535 @yelhamer
- binja: add support for forwarded exports #1646 @xusheng6
- binja: add support for symtab names #1504 @xusheng6
- add com class/interface features #322 @Aayush-Goel-04
- dotnet: emit enclosing class information for nested classes #1780 #1913 @bkojusner @mike-hunhoff
Breaking Changes
- remove the
SCOPE_*
constants in favor of theScope
enum #1764 @williballenthin - protobuf: deprecate
RuleMetadata.scope
in favor ofRuleMetadata.scopes
@williballenthin - protobuf: deprecate
Metadata.analysis
in favor ofMetadata.analysis2
that is dynamic analysis aware @williballenthin - update freeze format to v3, adding support for dynamic analysis @williballenthin
- extractor: ignore DLL name for api features #1815 @mr-tz
New Rules (41)
- nursery/get-ntoskrnl-base-address @mr-tz
- host-interaction/network/connectivity/set-tcp-connection-state @johnk3r
- nursery/capture-process-snapshot-data @mr-tz
- collection/network/capture-packets-using-sharppcap jakub.jozwiak@mandiant.com
- nursery/communicate-with-kernel-module-via-netlink-socket-on-linux michael.hunhoff@mandiant.com
- nursery/get-current-pid-on-linux michael.hunhoff@mandiant.com
- nursery/get-file-system-information-on-linux michael.hunhoff@mandiant.com
- nursery/get-password-database-entry-on-linux michael.hunhoff@mandiant.com
- nursery/mark-thread-detached-on-linux michael.hunhoff@mandiant.com
- nursery/persist-via-gnome-autostart-on-linux michael.hunhoff@mandiant.com
- nursery/set-thread-name-on-linux michael.hunhoff@mandiant.com
- load-code/dotnet/load-windows-common-language-runtime michael.hunhoff@mandiant.com blas.kojusner@mandiant.com jakub.jozwiak@mandiant.com
- nursery/log-keystrokes-via-input-method-manager @mr-tz
- nursery/encrypt-data-using-rc4-via-systemfunction032 richard.weiss@mandiant.com
- nursery/add-value-to-global-atom-table @mr-tz
- nursery/enumerate-processes-that-use-resource @Ana06
- host-interaction/process/inject/allocate-or-change-rwx-memory @mr-tz
- lib/allocate-or-change-rw-memory 0x534a@mailbox.org @mr-tz
- lib/change-memory-protection @mr-tz
- anti-analysis/anti-av/patch-antimalware-scan-interface-function jakub.jozwiak@mandiant.com
- executable/dotnet-singlefile/bundled-with-dotnet-single-file-deployment sara.rincon@mandiant.com
- internal/limitation/file/internal-dotnet-single-file-deployment-limitation sara.rincon@mandiant.com
- data-manipulation/encoding/encode-data-using-add-xor-sub-operations jakub.jozwiak@mandiant.com
- nursery/access-camera-in-dotnet-on-android michael.hunhoff@mandiant.com
- nursery/capture-microphone-audio-in-dotnet-on-android michael.hunhoff@mandiant.com
- nursery/capture-screenshot-in-dotnet-on-android michael.hunhoff@mandiant.com
- nursery/check-for-incoming-call-in-dotnet-on-android michael.hunhoff@mandiant.com
- nursery/check-for-outgoing-call-in-dotnet-on-android michael.hunhoff@mandiant.com
- nursery/compiled-with-xamarin michael.hunhoff@mandiant.com
- nursery/get-os-version-in-dotnet-on-android michael.hunhoff@mandiant.com
- data-manipulation/compression/create-cabinet-on-windows michael.hunhoff@mandiant.com jakub.jozwiak@mandiant.com
- data-manipulation/compression/extract-cabinet-on-windows jakub.jozwiak@mandiant.com
- lib/create-file-decompression-interface-context-on-windows jakub.jozwiak@mandiant.com
- nursery/enumerate-files-in-dotnet moritz.raabe@mandiant.com anushka.virgaonkar@mandiant.com
- nursery/get-mac-address-in-dotnet moritz.raabe@mandiant.com michael.hunhoff@mandiant.com echernofsky@google.com
- nursery/get-current-process-command-line william.ballenthin@mandiant.com
- nursery/get-current-process-file-path william.ballenthin@mandiant.com
- nursery/hook-routines-via-dlsym-rtld_next william.ballenthin@mandiant.com
- nursery/linked-against-hp-socket still@teamt5.org
- host-interaction/process/inject/process-ghostly-hollowing sara.rincon@mandiant.com
Bug Fixes
- ghidra: fix
ints_to_bytes
performance #1761 @mike-hunhoff - binja: improve function call site detection @xusheng6
- binja: use
binaryninja.load
to open files @xusheng6 - binja: bump binja version to 3.5 #1789 @xusheng6
- elf: better detect ELF OS via GCC .ident directives #1928 @williballenthin
- fix setuptools package discovery #1886 @gmacon @mr-tz
Development
Developer Notes
With this new release, many classes and concepts have been split up into static (mostly identical to the
prior implementations) and dynamic ones. For example, the legacy FeatureExtractor class has been renamed to
StaticFeatureExtractor and the DynamicFeatureExtractor has been added.
Starting from version 7.0, we have moved the component responsible for feature extractor from main to a new
capabilities' module. Now, users wishing to utilize capa’s feature extraction abilities should use that module instead
of importing the relevant logic from the main file.
For sandbox-based feature extractors, we are using Pydantic models. Contributions of more models for other sandboxes
are very welcome!
Raw diffs
v6.1.0
capa v6.1.0 is a bug fix release, most notably fixing unhandled exceptions in the capa explorer IDA Pro plugin. @Aayush-Goel-04 put a lot of effort into improving code quality and adding a script for rule authors. The script shows which features are present in a sample but not referenced by any existing rule. You could use this script to find opportunities for new rules.
Speaking of new rules, we have eight additions, coming from Ronnie, Jakub, Moritz, Ervin, and still@teamt5.org!
New Features
- ELF: implement import and export name extractor #1607 #1608 @Aayush-Goel-04
- bump pydantic from 1.10.9 to 2.1.1 #1582 @Aayush-Goel-04
- develop script to highlight features not used during matching #331 @Aayush-Goel-04
New Rules (8)
- executable/pe/export/forwarded-export ronnie.salomonsen@mandiant.com
- host-interaction/bootloader/get-uefi-variable jakub.jozwiak@mandiant.com
- host-interaction/bootloader/set-uefi-variable jakub.jozwiak@mandiant.com
- nursery/enumerate-device-drivers-on-linux @mr-tz
- anti-analysis/anti-vm/vm-detection/check-for-foreground-window-switch ervin.ocampo@mandiant.com
- linking/static/sqlite3/linked-against-cppsqlite3 still@teamt5.org
- linking/static/sqlite3/linked-against-sqlite3 still@teamt5.org
Modified rules (9)
- anti-analysis/anti-forensic/self-deletion/self-delete.yml
- collection/browser/gather-chrome-based-browser-login-information.yml
- collection/browser/gather-firefox-profile-information.yml
- data-manipulation/encoding/base64/decode-data-using-base64-via-dword-translation-table.yml
- host-interaction/process/inject/free-user-process-memory.yml
- lib/get-os-version.yml
- nursery/deserialize-json-in-dotnet.yml
- nursery/serialize-json-in-dotnet.yml
- persistence/authentication-process/act-as-credential-manager-dll.yml
Renamed rules (1)
Bug Fixes
- rules: fix forwarded export characteristic #1656 @RonnieSalomonsen
- Binary Ninja: Fix stack string detection #1473 @xusheng6
- linter: skip native API check for NtProtectVirtualMemory #1675 @williballenthin
- OS: detect Android ELF files #1705 @williballenthin
- ELF: fix parsing of symtab #1704 @williballenthin
- result document: don't use deprecated pydantic functions #1718 @williballenthin
- pytest: don't mark IDA tests as pytest tests #1719 @williballenthin
capa explorer IDA Pro plugin
- fix unhandled exception when resolving rule path #1693 @mike-hunhoff
EDIT: a standalone binary created using Python 3.11 is now available.
Raw diffs
v6.0.0
v6.0.0
capa v6.0 brings many bug fixes and quality improvements, including 64 rule updates and 26 new rules. We're now publishing to PyPI via Trusted Publishing and have migrated to using a pyproject.toml
file. @Aayush-Goel-04 contributed a lot of new code across many files, so please welcome them to the project, along with @anders-v @crowface28 @dkelly2e @RonnieSalomonsen and @ejfocampo as first-time rule contributors!
For those that use capa as a library, we've introduced some limited breaking changes that better represent data types (versus less-structured data like dictionaries and strings). With the recent deprecation, we've also dropped support for Python 3.7.
New Features
- add script to detect feature overlap between new and existing capa rules #1451 @Aayush-Goel-04
- extract forwarded exports from PE files #1624 @williballenthin
- extract function and API names from ELF symtab entries @yelhamer mandiant/capa-rules#736
- use fancy box drawing characters for default output #1586 @williballenthin
Breaking Changes
- use a class to represent Metadata (not dict) #1411 @Aayush-Goel-04 @manasghandat
- use pathlib.Path to represent file paths #1534 @Aayush-Goel-04
- Python 3.8 is now the minimum supported Python version #1578 @williballenthin
- Require a Contributor License Agreement (CLA) for PRs going forward #1642 @williballenthin
New Rules (26)
- load-code/shellcode/execute-shellcode-via-windows-callback-function ervin.ocampo@mandiant.com jakub.jozwiak@mandiant.com
- nursery/execute-shellcode-via-indirect-call ronnie.salomonsen@mandiant.com
- data-manipulation/encryption/aes/encrypt-data-using-aes-mixcolumns-step @mr-tz
- linking/static/aplib/linked-against-aplib still@teamt5.org
- communication/mailslot/read-from-mailslot nick.simonian@mandiant.com
- nursery/hash-data-using-sha512managed-in-dotnet jonathanlepore@google.com
- nursery/compiled-with-exescript jonathanlepore@google.com
- nursery/check-for-sandbox-via-mac-address-ouis-in-dotnet jonathanlepore@google.com
- host-interaction/hardware/enumerate-devices-by-category @mr-tz
- host-interaction/service/continue-service @mr-tz
- host-interaction/service/pause-service @mr-tz
- persistence/exchange/act-as-exchange-transport-agent jakub.jozwiak@mandiant.com
- host-interaction/file-system/create-virtual-file-system-in-dotnet jakub.jozwiak@mandiant.com
- compiler/cx_freeze/compiled-with-cx_freeze @mr-tz jakub.jozwiak@mandiant.com
- communication/socket/create-vmci-socket jakub.jozwiak@mandiant.com
- persistence/office/act-as-excel-xll-add-in jakub.jozwiak@mandiant.com
- persistence/office/act-as-office-com-add-in jakub.jozwiak@mandiant.com
- persistence/office/act-as-word-wll-add-in jakub.jozwiak@mandiant.com
- anti-analysis/anti-debugging/debugger-evasion/hide-thread-from-debugger michael.hunhoff@mandiant.com jakub.jozwiak@mandiant.com
- host-interaction/memory/create-new-application-domain-in-dotnet jakub.jozwiak@mandiant.com
- host-interaction/gui/switch-active-desktop jakub.jozwiak@mandiant.com
- host-interaction/service/query-service-configuration @mr-tz
- anti-analysis/anti-av/patch-event-tracing-for-windows-function jakub.jozwiak@mandiant.com
- data-manipulation/encoding/xor/covertly-decode-and-write-data-to-windows-directory-using-indirect-calls dan.kelly@mandiant.com
- linking/runtime-linking/resolve-function-by-brute-ratel-badger-hash jakub.jozwiak@mandiant.com
Bug Fixes
- extractor: add a Binary Ninja test that asserts its version #1487 @xusheng6
- extractor: update Binary Ninja stack string detection after the new constant outlining feature #1473 @xusheng6
- extractor: update vivisect Arch extraction #1334 @mr-tz
- extractor: avoid Binary Ninja exception when analyzing certain files #1441 @xusheng6
- symtab: fix struct.unpack() format for 64-bit ELF files @yelhamer
- symtab: safeguard against ZeroDivisionError for files containing a symtab with a null entry size @yelhamer
- improve ELF strtab and needed parsing @mr-tz
- better handle exceptional cases when parsing ELF files #1458 @Aayush-Goel-04
- improved testing coverage for Binary Ninja backend #1446 @Aayush-Goel-04
- add logging and print redirect to tqdm for capa main #749 @Aayush-Goel-04
- extractor: fix binja installation path detection does not work with Python 3.11
- tests: refine the IDA test runner script #1513 @williballenthin
- output: don't leave behind traces of progress bar @williballenthin
- import-to-ida: fix bug introduced with JSON report changes in v5 #1584 @williballenthin
- main: don't show spinner when emitting debug messages #1636 @williballenthin
capa explorer IDA Pro plugin
Development
- update ATT&CK/MBC data for linting #1568 @mr-tz
- log time taken to analyze each function #1290 @williballenthin
- tests: make fixture available via conftest.py #1592 @williballenthin
- publish via PyPI trusted publishing #1491 @williballenthin
- migrate to pyproject.toml #1301 @williballenthin
- use pre-commit to invoke linters #1579 @williballenthin
Raw diffs
v5.1.0
capa version 5.1.0 adds a Protocol Buffers (protobuf) format for result documents. Additionally, the Vector35 team contributed a new feature extractor using Binary Ninja. Other new features are a new CLI flag to override the detected operating system, functionality to read and render existing result documents, and a output color format that's easier to read.
Over 25 capa rules have been added and improved.
Thanks for all the support, especially to @xusheng6, @captainGeech42, @ggold7046, @manasghandat, @ooprathamm, @linpeiyu164, @yelhamer, @HongThatCong, @naikordian, @stevemk14ebr, @emtuls, @raymondlleong, @bkojusner, @joren485, and everyone else who submitted bugs and provided feedback!
New Features
- add protobuf format for result documents #1219 @williballenthin @mr-tz
- extractor: add Binary Ninja feature extractor @xusheng6
- new cli flag
--os
to override auto-detected operating system for a sample @captainGeech42 - change colour/highlight to "cyan" instead of "blue" for better readability #1384 @ggold7046
- add new format to parse output json back to capa #1396 @ooprathamm
- parse ELF symbols' names to guess OS #1403 @yelhamer
New Rules (26)
- persistence/scheduled-tasks/schedule-task-via-at joren485
- data-manipulation/prng/generate-random-numbers-via-rtlgenrandom william.ballenthin@mandiant.com
- communication/ip/convert-ip-address-from-string @mr-tz
- data-manipulation/compression/compress-data-via-zlib-inflate-or-deflate blas.kojusner@mandiant.com
- executable/installer/dotnet/packaged-as-single-file-dotnet-application michael.hunhoff@mandiant.com
- communication/socket/create-raw-socket blas.kojusner@mandiant.com
- communication/http/reference-http-user-agent-string @mr-tz
- communication/http/get-http-content-length william.ballenthin@mandiant.com
- nursery/move-directory michael.hunhoff@mandiant.com
- nursery/get-http-request-uri william.ballenthin@mandiant.com
- nursery/create-zip-archive-in-dotnet michael.hunhoff@mandiant.com
- nursery/extract-zip-archive-in-dotnet anushka.virgaonkar@mandiant.com michael.hunhoff@mandiant.com
- data-manipulation/encryption/tea/decrypt-data-using-tea william.ballenthin@mandiant.com raymond.leong@mandiant.com
- data-manipulation/encryption/tea/encrypt-data-using-tea william.ballenthin@mandiant.com raymond.leong@mandiant.com
- data-manipulation/encryption/xtea/encrypt-data-using-xtea raymond.leong@mandiant.com
- data-manipulation/encryption/xxtea/encrypt-data-using-xxtea raymond.leong@mandiant.com
- nursery/hash-data-using-ripemd128 raymond.leong@mandiant.com
- nursery/hash-data-using-ripemd256 raymond.leong@mandiant.com
- nursery/hash-data-using-ripemd320 raymond.leong@mandiant.com
- nursery/set-web-proxy-in-dotnet michael.hunhoff@mandiant.com
- nursery/check-for-windows-sandbox-via-subdirectory echernofsky@google.com
- nursery/enumerate-pe-sections-in-dotnet @mr-tz
- nursery/destroy-software-breakpoint-capability echernofsky@google.com
- nursery/send-data-to-internet michael.hunhoff@mandiant.com
- nursery/compiled-with-cx_freeze @mr-tz
- nursery/contain-a-thread-local-storage-tls-section-in-dotnet michael.hunhoff@mandiant.com
Bug Fixes
- extractor: removed '.dynsym' as the library name for ELF imports #1318 @stevemk14ebr
- extractor: fix vivisect loop detection corner case #1310 @mr-tz
- match: extend OS characteristic to match OS_ANY to all supported OSes #1324 @mike-hunhoff
- extractor: fix IDA and vivisect string and bytes features overlap and tests #1327 #1336 @xusheng6
capa explorer IDA Pro plugin
- fix exception when plugin loaded in IDA hosted under idat #1341 @mike-hunhoff
- improve embedded PE detection performance and reduce FP potential #1344 @mike-hunhoff
Raw diffs
v5.0.0
This capa version comes with major improvements and additions to better handle .NET binaries. To showcase this we've updated and added over 30 .NET rules.
Additionally, capa now caches its rule set for better performance. The capa explorer also caches its analysis results, so that multiple IDA Pro or plugin invocations don't need to repeat the same analysis.
We have removed the SMDA backend and changed the program return codes to be positive numbers.
Other improvements to highlight include better ELF OS detection, various rendering bug fixes, and enhancements to the feature extraction. We've also added support for Python 3.11.
Thanks for all the support, especially to @jsoref, @bkojusner, @edeca, @richardweiss80, @joren485, @ryantxu1, @mwilliams31, @anushkavirgaonkar, @MalwareMechanic, @Still34, @dzbeck, @johnk3r, and everyone else who submitted bugs and provided feedback!
New Features
- verify rule metadata format on load #1160 @mr-tz
- dotnet: emit property features #1168 @anushkavirgaonkar
- dotnet: emit API features for objects created via the newobj instruction #1186 @mike-hunhoff
- dotnet: emit API features for generic methods #1231 @mike-hunhoff
- Python 3.11 support #1192 @williballenthin
- dotnet: emit calls to/from MethodDef methods #1236 @mike-hunhoff
- dotnet: emit namespace/class features for ldvirtftn/ldftn instructions #1241 @mike-hunhoff
- dotnet: emit namespace/class features for type references #1242 @mike-hunhoff
- dotnet: extract dotnet and pe format #1187 @mr-tz
- don't render all library rule matches in vverbose output #1174 @mr-tz
- cache the rule set across invocations for better performance #1212 @williballenthin
- update ATT&CK/MBC data for linting #1297 @mr-tz
Breaking Changes
- remove SMDA backend #1062 @williballenthin
- error return codes are now positive numbers #1269 @mr-tz
New Rules (77)
- collection/use-dotnet-library-sharpclipboard @johnk3r
- data-manipulation/encryption/aes/use-dotnet-library-encryptdecryptutils @johnk3r
- data-manipulation/json/use-dotnet-library-newtonsoftjson @johnk3r
- data-manipulation/svg/use-dotnet-library-sharpvectors @johnk3r
- executable/resource/embed-dependencies-as-resources-using-fodycostura @johnk3r @mr-tz
- communication/ftp/send/send-file-using-ftp michael.hunhof@mandiant.com anushka.virgaonkar@mandiant.com
- nursery/extract-zip-archive anushka.virgaonkar@mandiant.com
- nursery/allocate-unmanaged-memory-in-dotnet michael.hunhoff@mandiant.com
- nursery/check-file-extension-in-dotnet michael.hunhoff@mandiant.com
- nursery/decode-data-using-base64-in-dotnet michael.hunhoff@mandiant.com
- nursery/deserialize-json-in-dotnet michael.hunhoff@mandiant.com
- nursery/find-data-using-regex-in-dotnet michael.hunhoff@mandiant.com
- nursery/generate-random-filename-in-dotnet michael.hunhoff@mandiant.com
- nursery/get-os-version-in-dotnet michael.hunhoff@mandiant.com
- nursery/load-xml-in-dotnet michael.hunhoff@mandiant.com
- nursery/manipulate-unmanaged-memory-in-dotnet michael.hunhoff@mandiant.com
- nursery/save-image-in-dotnet michael.hunhoff@mandiant.com
- nursery/send-email-in-dotnet michael.hunhoff@mandiant.com
- nursery/serialize-json-in-dotnet michael.hunhoff@mandiant.com
- nursery/set-http-user-agent-in-dotnet michael.hunhoff@mandiant.com
- nursery/compile-csharp-in-dotnet michael.hunhoff@mandiant.com
- nursery/compile-visual-basic-in-dotnet michael.hunhoff@mandiant.com
- nursery/compress-data-using-gzip-in-dotnet michael.hunhoff@mandiant.com
- nursery/execute-sqlite-statement-in-dotnet michael.hunhoff@mandiant.com
- nursery/execute-via-asynchronous-task-in-dotnet michael.hunhoff@mandiant.com
- nursery/execute-via-timer-in-dotnet michael.hunhoff@mandiant.com
- nursery/execute-wmi-query-in-dotnet michael.hunhoff@mandiant.com
- nursery/manipulate-network-credentials-in-dotnet michael.hunhoff@mandiant.com
- nursery/encrypt-data-using-aes william.ballenthin@mandiant.com Ivan Kwiatkowski (@JusticeRage)
- host-interaction/uac/bypass/bypass-uac-via-rpc david.cannings@pwc.com david@edeca.net
- nursery/check-for-vm-using-instruction-vpcext richard.weiss@mandiant.com
- nursery/get-windows-directory-from-kuser_shared_data david.cannings@pwc.com
- nursery/encrypt-data-using-openssl-dsa Ana06
- nursery/encrypt-data-using-openssl-ecdsa Ana06
- nursery/encrypt-data-using-openssl-rsa Ana06
- runtime/dotnet/execute-via-dotnet-startup-hook william.ballenthin@mandiant.com
- host-interaction/console/manipulate-console-buffer william.ballenthin@mandiant.com michael.hunhoff@mandiant.com
- nursery/access-wmi-data-in-dotnet michael.hunhoff@mandiant.com
- nursery/allocate-unmanaged-memory-via-dotnet michael.hunhoff@mandiant.com
- nursery/generate-random-bytes-in-dotnet michael.hunhoff@mandiant.com
- nursery/manipulate-console-window michael.hunhoff@mandiant.com
- nursery/obfuscated-with-koivm michael.hunhoff@mandiant.com
- nursery/implement-com-dll moritz.raabe@mandiant.com
- nursery/linked-against-libsodium @mr-tz
- compiler/nuitka/compiled-with-nuitka @williballenthin
- nursery/authenticate-data-with-md5-mac william.ballenthin@mandiant.com
- nursery/resolve-function-by-djb2-hash still@teamt5.org
- host-interaction/mutex/create-semaphore-on-linux @ramen0x3f
- host-interaction/mutex/lock-semaphore-on-linux @ramen0x3f
- host-interaction/mutex/unlock-semaphore-on-linux @ramen0x3f
- data-manipulation/hashing/sha384/hash-data-using-sha384 william.ballenthin@mandiant.com
- data-manipulation/hashing/sha512/hash-data-using-sha512 william.ballenthin@mandiant.com
- nursery/decode-data-using-url-encoding michael.hunhoff@mandiant.com
- nursery/manipulate-user-privileges michael.hunhoff@mandiant.com
- lib/get-os-version @mr-tz
- nursery/decrypt-data-using-tea william.ballenthin@mandiant.com
- nursery/encrypt-data-using-tea william.ballenthin@mandiant.com
- nursery/hash-data-using-whirlpool william.ballenthin@mandiant.com
- nursery/reference-base58-string william.ballenthin@mandiant.com
- communication/mailslot/create-mailslot william.ballenthin@mandiant.com
- executable/resource/access-dotnet-resource @mr-tz
- linking/static/linked-against-cpp-standard-library @mr-tz
- data-manipulation/compression/compress-data-using-lzo david@edeca.net david.cannings@pwc.com
- data-manipulation/compression/decompress-data-using-lzo david@edeca.net david.cannings@pwc.com
- communication/socket/tcp/create-tcp-socket-via-raw-afd-driver william.ballenthin@mandiant.com
- host-interaction/process/map-section-object william.ballenthin@mandiant.com
- lib/create-or-open-section-object william.ballenthin@mandiant.com
- load-code/dotnet/execute-dotnet-assembly-via-clr-host blas.kojusner@mandiant.com
- load-code/execute-vbscript-javascript-or-jscript-in-memory blas.kojusner@mandiant.com
- host-interaction/file-system/reference-absolute-stream-path-on-windows blas.kojusner@mandiant.com
- nursery/generate-method-via-reflection-in-dotnet michael.hunhoff@mandiant.com
- nursery/unmanaged-call-via-dynamic-pinvoke-in-dotnet michael.hunhoff@mandiant.com
Bug Fixes
- render: convert feature attributes to aliased dictionary for vverbose #1152 @mike-hunhoff
- decouple Token dependency / extractor and features #1139 @mr-tz
- update pydantic model to guarantee type coercion #1176 @mike-hunhoff
- do not overwrite version in version.py during PyInstaller build #1169 @mr-tz
- render: fix vverbose rendering of offsets #1215 @williballenthin
- elf: better detect OS via GLIBC ABI version needed and dependencies #1221 @williballenthin
- dotnet: address unhandled exceptions with improved type checking #1230 @mike-hunhoff
- fix import-to-ida script formatting #1208 @williballenthin
- render: fix verbose rendering of scopes #1263 @williballenthin
- rules: better detect invalid rules #1282 @williballenthin
- show-features: better render strings with embedded whitespace #1267 @williballenthin
- handle vivisect bug around strings at instruction level, use min length 4 #1271 @williballenthin @mr-tz
- extractor: guard against invalid "calls from" features #1177 @mr-tz
- extractor: add format to global features #1258 @mr-tz
- extractor: discover all strings with length >= 4 #1280 @mr-tz
- extractor: don't extract byte features for strings #1293 @mr-tz
capa explorer IDA Pro plugin
- fix: display instruction items #1154 @mr-tz
- fix: accept only plaintext pasted content #1194 @williballenthin
- fix: UnboundLocalError #1217 @williballenthin
- extractor: add support for COFF files and extern functions #1223 @mike-hunhoff
- doc: improve error messaging and documentation related to capa rule set #1249 @mike-hunhoff
- fix: assume 32-bit displacement for offsets #1250 @mike-hunhoff
- generator: refactor caching and matching #1251 @mike-hunhoff
- fix: improve exception handling to prevent IDA from locking up when errors occur #1262 @mike-hunhoff
- verify rule metadata using Pydantic #1167 @mr-tz
- extractor: make read consistent with file object behavior #1254 @mr-tz
- fix: UnboundLocalError x2 #1302 @mike-hunhoff
- cache capa results across IDA sessions #1279 @mr-tz
Raw diffs
v4.0.1
Some rules contained invalid metadata fields that caused an error when rendering rule hits. We've updated all rules and enhanced the rule linter to catch such issues.
New Rules (1)
- anti-analysis/obfuscation/obfuscated-with-vs-obfuscation jakub.jozwiak@mandiant.com
Bug Fixes
- linter: use pydantic to validate rule metadata #1141 @mike-hunhoff
- build binaries using PyInstaller no longer overwrites functions in version.py #1136 @mr-tz
Raw diffs
v4.0.0
Version 4 adds support for analyzing .NET executables. capa will autodetect .NET modules, or you can explicitly invoke the new feature extractor via --format dotnet
. We've also extended the rule syntax for .NET features including namespace
and class
.
Additionally, new instruction
scope and operand
features enable users to create more explicit rules. These features are not backwards compatible. We removed the previously used /x32
and /x64
flavors of number and operand features.
We updated 49 existing rules and added 22 new rules leveraging these new features and characteristics to detect capabilities seen in .NET malware.
More breaking changes include updates to the JSON results document, freeze file format schema (now format version v2), and the internal handling of addresses.
Thanks for all the support, especially to @htnhan, @jtothej, @sara-rn, @anushkavirgaonkar, and @_re_fox!
Deprecation warning: v4.0 will be the last capa version to support the SMDA backend.
New Features
- add new scope "instruction" for matching mnemonics and operands #767 @williballenthin
- add new feature "operand[{0, 1, 2}].number" for matching instruction operand immediate values #767 @williballenthin
- add new feature "operand[{0, 1, 2}].offset" for matching instruction operand offsets #767 @williballenthin
- extract additional offset/number features in certain circumstances #320 @williballenthin
- add detection and basic feature extraction for dotnet #987 @mr-tz, @mike-hunhoff, @williballenthin
- add file string extraction for dotnet files #1012 @mike-hunhoff
- add file function-name extraction for dotnet files #1015 @mike-hunhoff
- add unmanaged call characteristic for dotnet files #1023 @mike-hunhoff
- add mixed mode characteristic feature extraction for dotnet files #1024 @mike-hunhoff
- emit class and namespace features for dotnet files #1030 @mike-hunhoff
- render: support Addresses that aren't simple integers, like .NET token+offset #981 @williballenthin
- document rule tags and branches #1006 @williballenthin, @mr-tz
Breaking Changes
- instruction scope and operand feature are new and are not backwards compatible with older versions of capa
- Python 3.7 is now the minimum supported Python version #866 @williballenthin
- remove /x32 and /x64 flavors of number and operand features #932 @williballenthin
- the tool now accepts multiple paths to rules, and JSON doc updated accordingly @williballenthin
- extractors must use handles to identify functions/basic blocks/instructions #981 @williballenthin
- the freeze file format schema was updated, including format version bump to v2 #986 @williballenthin
Deprecation notice: as described in #937, we plan to remove the SMDA backend for v5. If you rely on this backend, please reach out so we can discuss extending the support for SMDA or transitioning your workflow to use vivisect.
New Rules (30)
- data-manipulation/encryption/aes/manually-build-aes-constants huynh.t.nhan@gmail.com
- nursery/get-process-image-filename michael.hunhoff@mandiant.com
- compiler/v/compiled-with-v jakub.jozwiak@mandiant.com
- compiler/zig/compiled-with-zig jakub.jozwiak@mandiant.com
- anti-analysis/packer/huan/packed-with-huan jakub.jozwiak@mandiant.com
- internal/limitation/file/internal-dotnet-file-limitation william.ballenthin@mandiant.com
- nursery/get-os-information-via-kuser_shared_data @mr-tz
- load-code/pe/resolve-function-by-parsing-PE-exports @sara-rn
- anti-analysis/packer/huan/packed-with-huan jakub.jozwiak@mandiant.com
- nursery/execute-dotnet-assembly anushka.virgaonkar@mandiant.com
- nursery/invoke-dotnet-assembly-method anushka.virgaonkar@mandiant.com
- collection/screenshot/capture-screenshot-via-keybd-event @_re_fox
- collection/browser/gather-chrome-based-browser-login-information @_re_fox
- nursery/power-down-monitor michael.hunhoff@mandiant.com
- nursery/hash-data-using-aphash @_re_fox
- nursery/hash-data-using-jshash @_re_fox
- host-interaction/file-system/files/list/enumerate-files-on-windows moritz.raabe@mandiant.com anushka.virgaonkar@mandiant.com
- nursery/check-clipboard-data anushka.virgaonkar@mandiant.com
- nursery/clear-clipboard-data anushka.virgaonkar@mandiant.com
- nursery/compile-dotnet-assembly anushka.virgaonkar@mandiant.com
- nursery/create-process-via-wmi anushka.virgaonkar@mandiant.com
- nursery/display-service-notification-message-box anushka.virgaonkar@mandiant.com
- nursery/find-process-by-name anushka.virgaonkar@mandiant.com
- nursery/generate-random-numbers-in-dotnet anushka.virgaonkar@mandiant.com
- nursery/send-keystrokes anushka.virgaonkar@mandiant.com
- nursery/send-request-in-dotnet anushka.virgaonakr@mandiant.com
- nursery/terminate-process-by-name-in-dotnet anushka.virgaonkar@mandiant.com
- nursery/hash-data-using-rshash @_re_fox
- persistence/authentication-process/act-as-credential-manager-dll jakub.jozwiak@mandiant.com
- persistence/authentication-process/act-as-password-filter-dll jakub.jozwiak@mandiant.com
Bug Fixes
- improve handling _ prefix compile/link artifact #924 @mike-hunhoff
- better detect OS in ELF samples #988 @williballenthin
- display number feature zero in vverbose #1097 @mike-hunhoff
capa explorer IDA Pro plugin
- improve file format extraction #918 @mike-hunhoff
- remove decorators added by IDA to ELF imports #919 @mike-hunhoff
- bug fixes for Address abstraction #1091 @mike-hunhoff
Raw diffs
v3.2.1
This release bumps the SMDA dependency version to enable installation on Python 3.10.
Bug Fixes
- update SMDA dependency @mike-hunhoff #922