{"payload":{"feedbackUrl":"https://github.com/orgs/community/discussions/53140","repo":{"id":24228422,"defaultBranch":"master","name":"bearded-avenger","ownerLogin":"csirtgadgets","currentUserCanPush":false,"isFork":false,"isEmpty":false,"createdAt":"2014-09-19T12:07:04.000Z","ownerAvatar":"https://avatars.githubusercontent.com/u/5203786?v=4","public":true,"private":false,"isOrgOwned":true},"refInfo":{"name":"","listCacheKey":"v0:1672176382.710321","currentOid":""},"activityList":{"items":[{"before":"731d4e45f1f46e45ac1d9d9fd0dc79cadd4d14ae","after":"418794dbeedb14d195b955d728032b4f412cddef","ref":"refs/heads/master","pushedAt":"2023-04-20T22:54:05.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"wesyoung","name":"wes","path":"/wesyoung","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/474878?s=80&v=4"},"commit":{"message":"Add Integrations section (#525)","shortMessageHtmlLink":"Add Integrations section (#525)"}},{"before":"bc992ef074d134869c1c7104b565da5d91f608a5","after":"731d4e45f1f46e45ac1d9d9fd0dc79cadd4d14ae","ref":"refs/heads/master","pushedAt":"2023-04-19T15:40:18.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"wesyoung","name":"wes","path":"/wesyoung","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/474878?s=80&v=4"},"commit":{"message":"Fix token creation deserialization due to datetime (#524)","shortMessageHtmlLink":"Fix token creation deserialization due to datetime (#524)"}},{"before":"2d4ab3219f5c093a8ed8a089cf463b674eda092f","after":"bc992ef074d134869c1c7104b565da5d91f608a5","ref":"refs/heads/master","pushedAt":"2023-04-14T18:17:54.000Z","pushType":"pr_merge","commitsCount":1,"pusher":{"login":"wesyoung","name":"wes","path":"/wesyoung","primaryAvatarUrl":"https://avatars.githubusercontent.com/u/474878?s=80&v=4"},"commit":{"message":"feature/merge changes (#523)\n\n* Check `expires` and `revoked` token fields\r\n\r\n* Implement strtobool\r\n\r\n* Standardize print statements\r\n\r\n* Implement token tweaks\r\n\r\n* add `last_edited_at` and `last_edited_by`, `created_at`, and `created_by` fields for token updates in ES store\r\n* `token_cache` dict implemented as mp.Manager proxy to share cache b/w processes\r\n* token `search()` caches token result for fast lookup\r\n* token `auth_search()` now checks cache first, possible 100x speed increase\r\n* token `edit()` changes in-mem cache and uses ES versioning to prevent stale cache stomping b/w diff router instances\r\n* token cache flushed at `TOKEN_CACHE_DELAY` interval and written using new bulk=True edit param\r\n* add pytests for the above\r\n\r\n* Add some logic and improved error handling\r\n\r\n* Eliminate hardcoded `fqdn` itype\r\n\r\nFix the assumption that a hostname for a url will always be an `fqdn` itype, e.g. `http://10.10.10.1/Malware.exe`\r\n\r\n* Skip ja3 gatherer if source is already `ja3er.com`\r\n\r\n* Add `find_relatives` and `sort` param frontend func\r\n\r\n* Enables passing a `?sort=` param to API (defaults to `?sort=-reporttime,-lasttime` which specifies reporttime DESC and lasttime DESC)\r\n * feed pulls default to (and can't be changed from) `-confidence,-reporttime`\r\n* Adds `?find_relatives=` API param which default to `False` for all searches\r\n * Enables finding related/familial indicators (super/subnets for IPs and related SSDEEP)\r\n\r\nBackend CIF searches such as allowlist searches for feed pulls or searches done during upsert checks will enforce `find_relatives=False`\r\n\r\n* change `find_relatives` frontend default to `False`\r\n\r\n* Add ssdeep as newly supported `itype`\r\n\r\n* add appropriate new fields to analyze/tokenize on store\r\n* includes new search filters for finding related ssdeep matches\r\n* add ability to pull ssdeep feeds\r\n* add new unit tests\r\n\r\n* Refactor hunters\r\n\r\n* Add spamhaus DQS support\r\n* lower fqdn hunter conf such that no further hunters run on their creations\r\n* each hunter now has a set of supported itypes and mtypes\r\n* factored out each hunter's prereq check into own func\r\n* additional info such as mtype and nolog passed to each hunter for flexibility\r\n* make http filters a set and better validate input\r\n\r\n* Indicator index tweaks\r\n\r\n* .init() will only try to update mappings if schema is diff\r\n* if there is a diff and index is open, IllegalOperation returned\r\n* if another proc already closing/reopening, catch/retry\r\n\r\n* Fix upserts matching on overlapping tags\r\n\r\n* fix issue where identical indicator w/ additional tags would previously upsert into existing indicator with fewer tags that had overlap, e.g.:\r\n * provider=example.com, confidence=5, indicator=bad.tld, tags=[old]\r\n * provider=example.com, confidence=5, indicator=badl.tld, tags[old,new]\r\n previously, the new indicator would be upserted into the old indicator and the new tag info would be lost. it now creates an entirely new indicator\r\n\r\n* URL param keys now lowercased on every frontend request for better UX in the event a human specifies a non-lowercase param (like Asn=0000 instead of asn=0000)\r\n\r\n* check for existence of `indicator` field at frontend during submission to ensure no malformed indicators are sent on\r\n\r\n* Update to TLP2\r\n\r\n* adjust `white` to `clear` to align with tlp2\r\n\r\n* Fix upsert group match\r\n\r\nUpserts weren't correctly matching diff groups in certain cases, causing an indicator w/ all other identical fields to upsert match and increase count of existing indicator rather than create a new indicator. This was due to the API only allowing the url param `groups` (plural), but several cif clients making use of `group` (singular).\r\n\r\nAlso fixed upsert matches on optional fields. e.g., if user-specified, but optional, field like `rdata` is specified in one indicator but not another, all other fields being equal, those indicators are diff and therefore shouldn't upsert match.\r\n* fix the above issue so that if a field isn't specified in the upsert search, it's specificially excluded in the ES search\r\n\r\n* misc fixes\r\n\r\n* Enable API multiparams\r\n\r\n* support multiple URL params w/ the same name\r\n\r\nThere's no web std for how to pass multiple values to a URL param like `/feed?tags=malware,exploit` and some clients (like python requests lib) takes multiple values like `{ 'tags': [ 'exploit', 'malware' ] }` and passes those in as `/feed?tags=exploit&tags=malware`. Unfortunately, by default Flask ignores all URL params of the same name after the first one, so it would receive such a request at the API and only consider the `exploit` tag. This change normalizes multiple params of the same name into a single key whose value is a comma-delimieted string.\r\n\r\nTherefore, `/feed?tags=malware&tags=exploit` is re-interpreted as `tags=malware,exploit` and therefore correctly considers all passed values to the API.\r\n\r\nAdditionally, this commit supports a SHARDS_PER_INDEX variable for ES indices.\r\n\r\n* Refactor a few gatherers\r\n\r\n* misc fixes\r\n\r\n* Move `reporttime` parsing from specific stores to main\r\n\r\n* Remove deprecated use of _request_ctx_stack\r\n\r\n* Align with tlp2\r\n\r\n* URL param filtering now happens closer to frontend\r\n\r\n* Fix some things related to sort tweaks\r\n\r\n* Add test for `reporttime`","shortMessageHtmlLink":"feature/merge changes (#523)"}}],"hasNextPage":false,"hasPreviousPage":false,"activityType":"all","actor":null,"timePeriod":"all","sort":"DESC","perPage":30,"cursor":"djE6ks8AAAADHTiiiAA","startCursor":null,"endCursor":null}},"title":"Activity ยท csirtgadgets/bearded-avenger"}