Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename event serializers #2300

Merged
merged 42 commits into from
Sep 16, 2022
Merged
Show file tree
Hide file tree
Changes from 40 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
0e2b165
Agent: Rename IEventSerializer -> IAgentEventSerializer
shreyamalviya Sep 16, 2022
619dc68
UT: Use IAgentEventSerializer in place of IEventSerializer
shreyamalviya Sep 16, 2022
3b22bf0
Common:Rename i_event_serializer.py -> i_agent_event_serializer.py
shreyamalviya Sep 16, 2022
c4ab5d9
Agent: Fix import in agent_event_forwarder.py
shreyamalviya Sep 16, 2022
6ced4af
Common: Rename EventSerializerRegistry -> AgentEventSerializerRegistry
shreyamalviya Sep 16, 2022
19a59bf
Agent: Use AgentEventSerializerRegistry in place of EventSerializerRe…
shreyamalviya Sep 16, 2022
4a23f5a
Island: Use AgentEventSerializerRegistry in place of EventSerializerR…
shreyamalviya Sep 16, 2022
2568d52
UT: Use AgentEventSerializerRegistry in place of EventSerializerRegistry
shreyamalviya Sep 16, 2022
940b479
Common: Rename event_serializer_registry.py -> agent_event_serializer…
shreyamalviya Sep 16, 2022
bf5e32a
Project: Remove irrelevant Vulture allowlist entries
shreyamalviya Sep 16, 2022
43fd978
Common: Rename PydanticEventSerializer -> PydanticAgentEventSerializer
shreyamalviya Sep 16, 2022
d729b51
UT: Use PydanticAgentEventSerializer in place of PydanticEventSerializer
shreyamalviya Sep 16, 2022
f5a0b42
Common: Rename pydantic_event_serializer.py -> pydantic_agent_event_s…
shreyamalviya Sep 16, 2022
d694591
UT: Rename test_pydantic_event_serializer.py -> test_pydantic_agent_e…
shreyamalviya Sep 16, 2022
2f74851
UT: Change pydantic_event_serializer -> pydantic_agent_event_serializ…
shreyamalviya Sep 16, 2022
d732727
Common: Rename event_serializers/ -> agent_event_serializers/
shreyamalviya Sep 16, 2022
922db9c
Agent: Fix import common.event_serializers -> common.agent_event_seri…
shreyamalviya Sep 16, 2022
91b973c
Island: Fix import common.event_serializers -> common.agent_event_ser…
shreyamalviya Sep 16, 2022
7bfc2da
UT: Fix import common.event_serializers -> common.agent_event_seriali…
shreyamalviya Sep 16, 2022
edb1c97
UT: Rename event_serializers/ -> agent_event_serializers/
shreyamalviya Sep 16, 2022
e6db64b
UT: Rename test_event_serializer_registry.py -> test_agent_event_seri…
shreyamalviya Sep 16, 2022
5ad5de9
UT: Change event_serializer_registry -> agent_event_serializer_regist…
shreyamalviya Sep 16, 2022
edce35e
Agent: Make log message more specific in agent_event_forwarder.py
shreyamalviya Sep 16, 2022
c37876c
Island: Rename resource Events -> AgentEvents
shreyamalviya Sep 16, 2022
354d662
UT: Use AgentEvents in place of Events in test_events.py
shreyamalviya Sep 16, 2022
cfc0905
Island: Rename `/api/events` to `/api/agent-events`
shreyamalviya Sep 16, 2022
66eaccb
Agent: `/api/agent-events` in place of `/api/events`
shreyamalviya Sep 16, 2022
ad30efc
Chaneglog: Add entry for endpoint rename
shreyamalviya Sep 16, 2022
f0d5545
Island: Rename cc/resources/events.py -> cc/resources/agent_events.py
shreyamalviya Sep 16, 2022
0dccbd3
Island: Rename IEventRepository -> IAgentEventRepository
shreyamalviya Sep 16, 2022
be612ab
UT: Change IEventRepository -> IAgentEventRepository
shreyamalviya Sep 16, 2022
c7fef90
Project: Fix Vulture allowlist entries
shreyamalviya Sep 16, 2022
89a8ed7
Island: Rename i_event_repository.py -> i_agent_event_repository.py
shreyamalviya Sep 16, 2022
e2fed84
Project: Fix import in Vulture allowlist
shreyamalviya Sep 16, 2022
b5736fd
Common: Rename common/events/ -> common/agent_events/
shreyamalviya Sep 16, 2022
c8d11cd
Agent: Fix imports common.events -> common.agent_events
shreyamalviya Sep 16, 2022
d93eb45
Island: Fix imports common.events -> common.agent_events
shreyamalviya Sep 16, 2022
f8134e5
UT: Fix imports common.events -> common.agent_events
shreyamalviya Sep 16, 2022
83279cb
UT: Fix CredentialsStolenEvent event import
mssalvatore Sep 16, 2022
f0e8691
UT: Rename common/events -> common/agent_events
mssalvatore Sep 16, 2022
dd2ed50
CHANGELOG: Fix `/api/agent-events` entry
mssalvatore Sep 16, 2022
998c1d7
Agent: Rename EVENTS_API_URL to AGENT_EVENTS_API_URL
VakarisZ Sep 16, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@ Changelog](https://keepachangelog.com/en/1.0.0/).
- Tunneling to relays to provide better firewall evasion, faster Island
connection times, unlimited hops, and a more resilient way for agents to call
home. #2216, #1583
- `/api/events` to `/api/agent-events`. #2300

### Removed
- VSFTPD exploiter. #1533
Expand Down
5 changes: 5 additions & 0 deletions monkey/common/agent_event_serializers/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
from .consts import EVENT_TYPE_FIELD
from .i_agent_event_serializer import IAgentEventSerializer, JSONSerializable
from .agent_event_serializer_registry import AgentEventSerializerRegistry
from .pydantic_agent_event_serializer import PydanticAgentEventSerializer
from .register import register_common_agent_event_serializers
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
from typing import Type, Union

from common.event_serializers import IEventSerializer
from common.events import AbstractAgentEvent
from common.agent_event_serializers import IAgentEventSerializer
from common.agent_events import AbstractAgentEvent


class EventSerializerRegistry:
class AgentEventSerializerRegistry:
"""
Registry for event serializers using event class.

Example:
event_serializer_registry = EventSerializerRegistry()
event_serializer_registry = AgentEventSerializerRegistry()
event_serializer_registry[MyEvent] = MyEventSerializer()

my_event_dict = {"type": "MyEvent", "data": "123"}
Expand All @@ -22,18 +22,20 @@ def __init__(self):
self._registry = {}

def __setitem__(
self, event_class: Type[AbstractAgentEvent], event_serializer: IEventSerializer
self, event_class: Type[AbstractAgentEvent], event_serializer: IAgentEventSerializer
):
if not issubclass(event_class, AbstractAgentEvent):
raise TypeError(f"Event class must be of type: {AbstractAgentEvent.__name__}")

if not isinstance(event_serializer, IEventSerializer):
raise TypeError(f"Event serializer must be of type: {IEventSerializer.__name__}")
if not isinstance(event_serializer, IAgentEventSerializer):
raise TypeError(f"Event serializer must be of type: {IAgentEventSerializer.__name__}")

self._registry[event_class] = event_serializer
self._registry[event_class.__name__] = event_serializer

def __getitem__(self, event_class: Union[str, Type[AbstractAgentEvent]]) -> IEventSerializer:
def __getitem__(
self, event_class: Union[str, Type[AbstractAgentEvent]]
) -> IAgentEventSerializer:
if not (isinstance(event_class, str) or issubclass(event_class, AbstractAgentEvent)):
raise TypeError(
f"Registry get key {event_class} must be of type: {AbstractAgentEvent.__name__} or "
Expand Down
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
from abc import ABC, abstractmethod
from typing import Dict, List, Union

from common.events import AbstractAgentEvent
from common.agent_events import AbstractAgentEvent

JSONSerializable = Union[ # type: ignore[misc]
Dict[str, "JSONSerializable"], # type: ignore[misc]
Expand All @@ -14,7 +14,7 @@
]


class IEventSerializer(ABC):
class IAgentEventSerializer(ABC):
"""
Manages serialization and deserialization of events
"""
Expand Down
Original file line number Diff line number Diff line change
@@ -1,17 +1,17 @@
import logging
from typing import Generic, Type, TypeVar

from common.events import AbstractAgentEvent
from common.agent_events import AbstractAgentEvent
from common.utils.code_utils import del_key

from . import EVENT_TYPE_FIELD, IEventSerializer, JSONSerializable
from . import EVENT_TYPE_FIELD, IAgentEventSerializer, JSONSerializable

logger = logging.getLogger(__name__)

T = TypeVar("T", bound=AbstractAgentEvent)


class PydanticEventSerializer(IEventSerializer, Generic[T]):
class PydanticAgentEventSerializer(IAgentEventSerializer, Generic[T]):
def __init__(self, event_class: Type[T]):
self._event_class = event_class

Expand Down
11 changes: 11 additions & 0 deletions monkey/common/agent_event_serializers/register.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
from common.agent_events import CredentialsStolenEvent

from . import AgentEventSerializerRegistry, PydanticAgentEventSerializer


def register_common_agent_event_serializers(
event_serializer_registry: AgentEventSerializerRegistry,
):
event_serializer_registry[CredentialsStolenEvent] = PydanticAgentEventSerializer(
CredentialsStolenEvent
)
2 changes: 1 addition & 1 deletion monkey/common/event_queue/i_agent_event_queue.py
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
from abc import ABC, abstractmethod
from typing import Type

from common.events import AbstractAgentEvent
from common.agent_events import AbstractAgentEvent

from . import AgentEventSubscriber

Expand Down
2 changes: 1 addition & 1 deletion monkey/common/event_queue/pypubsub_agent_event_queue.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@

from pubsub.core import Publisher

from common.agent_events import AbstractAgentEvent
from common.event_queue import PyPubSubPublisherWrapper
from common.events import AbstractAgentEvent

from . import AgentEventSubscriber, IAgentEventQueue

Expand Down
2 changes: 1 addition & 1 deletion monkey/common/event_queue/types.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
from typing import Callable

from common.events import AbstractAgentEvent
from common.agent_events import AbstractAgentEvent

AgentEventSubscriber = Callable[[AbstractAgentEvent], None]
5 changes: 0 additions & 5 deletions monkey/common/event_serializers/__init__.py

This file was deleted.

9 changes: 0 additions & 9 deletions monkey/common/event_serializers/register.py

This file was deleted.

11 changes: 5 additions & 6 deletions monkey/infection_monkey/agent_event_forwarder.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,17 +5,16 @@

import requests

from common.agent_event_serializers import AgentEventSerializerRegistry, JSONSerializable
from common.agent_events import AbstractAgentEvent
from common.common_consts.timeouts import MEDIUM_REQUEST_TIMEOUT
from common.event_serializers import EventSerializerRegistry
from common.event_serializers.i_event_serializer import JSONSerializable
from common.events import AbstractAgentEvent
from infection_monkey.utils.threading import create_daemon_thread

logger = logging.getLogger(__name__)


DEFAULT_TIME_PERIOD_SECONDS = 5
EVENTS_API_URL = "https://%s/api/events"
EVENTS_API_URL = "https://%s/api/agent-events"


class AgentEventForwarder:
Expand All @@ -24,7 +23,7 @@ class AgentEventForwarder:
"""

def __init__(
self, server_address: str, agent_event_serializer_registry: EventSerializerRegistry
self, server_address: str, agent_event_serializer_registry: AgentEventSerializerRegistry
):
self._server_address = server_address
self._agent_event_serializer_registry = agent_event_serializer_registry
Expand Down Expand Up @@ -85,7 +84,7 @@ def _send_events_to_island(self):
events.append(self._queue.get(block=False))

try:
logger.debug(f"Sending events to Island at {self._server_address}: {events}")
logger.debug(f"Sending Agent events to Island at {self._server_address}: {events}")
requests.post( # noqa: DUO123
EVENTS_API_URL % (self._server_address,),
json=events,
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
import logging
from typing import Sequence

from common.agent_events import CredentialsStolenEvent
from common.credentials import Credentials, LMHash, NTHash, Password, Username
from common.event_queue import IAgentEventQueue
from common.events import CredentialsStolenEvent
from infection_monkey.i_puppet import ICredentialCollector
from infection_monkey.model import USERNAME_PREFIX
from infection_monkey.utils.ids import get_agent_id
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,9 @@
import os
from typing import Dict, Iterable, Sequence

from common.agent_events import CredentialsStolenEvent
from common.credentials import Credentials, SSHKeypair, Username
from common.event_queue import IAgentEventQueue
from common.events import CredentialsStolenEvent
from common.utils.attack_utils import ScanStatus
from infection_monkey.telemetry.attack.t1005_telem import T1005Telem
from infection_monkey.telemetry.attack.t1145_telem import T1145Telem
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import logging

from common.events import CredentialsStolenEvent
from common.agent_events import CredentialsStolenEvent

from . import IPropagationCredentialsRepository

Expand Down
2 changes: 1 addition & 1 deletion monkey/infection_monkey/exploit/zerologon.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@
from impacket.dcerpc.v5 import epm, nrpc, rpcrt, transport
from impacket.dcerpc.v5.dtypes import NULL

from common.agent_events import CredentialsStolenEvent
from common.common_consts.timeouts import LONG_REQUEST_TIMEOUT
from common.credentials import Credentials, LMHash, NTHash, Username
from common.events import CredentialsStolenEvent
from infection_monkey.exploit.HostExploiter import HostExploiter
from infection_monkey.exploit.tools.wmi_tools import WmiTools
from infection_monkey.exploit.zerologon_utils.dump_secrets import DumpSecrets
Expand Down
14 changes: 7 additions & 7 deletions monkey/infection_monkey/monkey.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,12 @@

from pubsub.core import Publisher

from common.event_queue import IAgentEventQueue, PyPubSubAgentEventQueue
from common.event_serializers import (
EventSerializerRegistry,
from common.agent_event_serializers import (
AgentEventSerializerRegistry,
register_common_agent_event_serializers,
)
from common.events import CredentialsStolenEvent
from common.agent_events import CredentialsStolenEvent
from common.event_queue import IAgentEventQueue, PyPubSubAgentEventQueue
from common.network.network_utils import address_to_ip_port
from common.utils.argparse_types import positive_int
from common.utils.attack_utils import ScanStatus, UsageEnum
Expand Down Expand Up @@ -213,8 +213,8 @@ def _setup(self):
register_signal_handlers(self._master)

# TODO: This is just a placeholder for now. We will modify/integrate it with PR #2279.
def _setup_agent_event_serializers(self) -> EventSerializerRegistry:
agent_event_serializer_registry = EventSerializerRegistry()
def _setup_agent_event_serializers(self) -> AgentEventSerializerRegistry:
agent_event_serializer_registry = AgentEventSerializerRegistry()
register_common_agent_event_serializers(agent_event_serializer_registry)

return agent_event_serializer_registry
Expand Down Expand Up @@ -259,7 +259,7 @@ def _subscribe_events(
event_queue: IAgentEventQueue,
propagation_credentials_repository: IPropagationCredentialsRepository,
server_address: str,
agent_event_serializer_registry: EventSerializerRegistry,
agent_event_serializer_registry: AgentEventSerializerRegistry,
):
event_queue.subscribe_type(
CredentialsStolenEvent,
Expand Down
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
import logging

from common.events import AbstractAgentEvent
from monkey_island.cc.repository import IEventRepository, StorageError
from common.agent_events import AbstractAgentEvent
from monkey_island.cc.repository import IAgentEventRepository, StorageError

logger = logging.getLogger(__name__)


class save_event_to_event_repository:
def __init__(self, event_repository: IEventRepository):
def __init__(self, event_repository: IAgentEventRepository):
self._event_repository = event_repository

def __call__(self, event: AbstractAgentEvent):
Expand Down
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
import logging

from common.events import CredentialsStolenEvent
from common.agent_events import CredentialsStolenEvent
from monkey_island.cc.repository import ICredentialsRepository, StorageError

logger = logging.getLogger(__name__)
Expand Down
4 changes: 2 additions & 2 deletions monkey/monkey_island/cc/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,9 @@
from monkey_island.cc.resources import (
AgentBinaries,
AgentConfiguration,
AgentEvents,
Agents,
ClearSimulationData,
Events,
IPAddresses,
IslandLog,
PBAFileDownload,
Expand Down Expand Up @@ -187,7 +187,7 @@ def init_restful_endpoints(api: FlaskDIWrapper):
api.add_resource(IslandLog)
api.add_resource(IPAddresses)

api.add_resource(Events)
api.add_resource(AgentEvents)

# API Spec: These two should be the same resource, GET for download and POST for upload
api.add_resource(PBAFileDownload)
Expand Down
2 changes: 1 addition & 1 deletion monkey/monkey_island/cc/repository/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
from .i_machine_repository import IMachineRepository
from .i_agent_repository import IAgentRepository
from .i_node_repository import INodeRepository
from .i_event_repository import IEventRepository
from .i_agent_event_repository import IAgentEventRepository


from .local_storage_file_repository import LocalStorageFileRepository
Expand Down
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
from abc import ABC, abstractmethod
from typing import Sequence, Type, TypeVar

from common.events import AbstractAgentEvent
from common.agent_events import AbstractAgentEvent
from common.types import AgentID

T = TypeVar("T", bound=AbstractAgentEvent)


class IEventRepository(ABC):
class IAgentEventRepository(ABC):
"""A repository used to store and retrieve event objects"""

@abstractmethod
Expand Down
12 changes: 7 additions & 5 deletions monkey/monkey_island/cc/repository/mongo_event_repository.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,21 @@

from pymongo import MongoClient

from common.event_serializers import EVENT_TYPE_FIELD, EventSerializerRegistry
from common.events import AbstractAgentEvent
from common.agent_event_serializers import EVENT_TYPE_FIELD, AgentEventSerializerRegistry
from common.agent_events import AbstractAgentEvent
from common.types import AgentID
from monkey_island.cc.repository import IEventRepository
from monkey_island.cc.repository import IAgentEventRepository

from . import RemovalError, RetrievalError, StorageError
from .consts import MONGO_OBJECT_ID_KEY


class MongoEventRepository(IEventRepository):
class MongoEventRepository(IAgentEventRepository):
"""A repository for storing and retrieving events in MongoDB"""

def __init__(self, mongo_client: MongoClient, serializer_registry: EventSerializerRegistry):
def __init__(
self, mongo_client: MongoClient, serializer_registry: AgentEventSerializerRegistry
):
self._events_collection = mongo_client.monkey_island.events
self._serializers = serializer_registry

Expand Down
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
from typing import Sequence, Type, TypeVar

from common.events import AbstractAgentEvent
from common.agent_events import AbstractAgentEvent
from common.types import AgentID

from . import IEventRepository
from . import IAgentEventRepository

T = TypeVar("T", bound=AbstractAgentEvent)


# TODO: Remove this class after #2180 is complete
class StubbedEventRepository(IEventRepository):
class StubbedEventRepository(IAgentEventRepository):
def save_event(self, event: AbstractAgentEvent):
return

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
from monkey_island.cc.models.zero_trust.event import Event


class IEventRepository(ABC):
class IAgentEventRepository(ABC):
def get_events(self, finding_id: Optional[str] = None) -> Sequence[Event]:
pass

Expand Down
Loading