Operational Data References

This Operational Data Reference (ODR) is the key a reference for how data should be formated to be accepted and parsed by the Nemesis. Various ODRs have been defined for data that is useful to collect and perform automated post processing analysis on.

Each data submission to Nemesis must consist of an metadata block and the data block. The metadata includes information that is relavent to all submissions.

{
    "metadata": {
        ...
    }
    "data" : {
        ...
    }
}

Design Considerations

The goal of Nemesis is to store operational data that may be useful to do automated post process analysis on. To this end, several considerations are made before a new ODR is added: - Is the ODR too specific to be useful to do post process analysis on during most engagements? - If a user may already submit the data directly to the binary data endpoint (ex. PCAP data), would an ODR provide additional useful information or cause more burden for the user? - Can any parameters in the ODR by defined in a way that will cause less ambiguity for the user?

Encodings and Formats

All data must be encoded as UTF-8. Data must be formated as JSON to be parsed by Nemesis. The metadata and data sections of the submission will both contain information of different formats such as timestamps, booleans, and binary data. Conventions for each are defined here. Any information format that is unique to an ODR will be specified within that specific ODR. Any string value may include hex escape sequences. Escape sequences for UTF-8 is not needed because all submissions are already in UTF-8.

Base Information Formats	Standard and encodings
UUID	Nemesis UUID string referencing a binary data already uploaded to Nemesis
bool	True or False booleans
datetime	ISO 8601 data timestamp in UTC
int/long	Numeric data
string	String data with \x for hex escapes

* Binary data is expected to be submitted first to the Nemesis before submitted any ODR formatted data that refers to it. Nemesis will return a UUID for a binary submission which may be used as a UUID reference parameter.

Metadata Section

The metadata section include the project name and agent from which the data was collected. Each data submission will be primarily relevant to one of these two scopes which will be defined in each ODR.

The metadata must include the time the data was sent to Nemesis, type of agent from which the data was collected (ex. beacon), a unique agent identifier, and whether the data was automatically sent from the agent or manually submitted by an operator. This information helps triage issues that may occure with data ingest to Nemesis.

The metadata must include an experation date for when the data should be removed from Nemesis. This helps with data compliance policies.

Finally, the metadata must include a type name to define which ODR the body of the data submission must conform to.

Values:

Name	Format	Description
agent_id	string	Name or unique identifier for an agent
agent_type	string	Name of the type of agent (ex. beacon/apollo/etc.)
automated	bool	True if the submission was submitted automatically
data_type	string	Name of the ODR the body must conform to
expiration	datetime	Time the data should be removed from Nemesis
source	string	Info about the source of the data. See each data type for possible values
project	string	Name or unique identifier for a project
timestamp	datetime	Time the C2 platform sent the data to Nemesis

Json Example:

{
    "metadata": {
        "agent_id": "339429212",
        "agent_type": "beacon",
        "automated": 1,
        "data_type": "file_data",
        "expiration": "2023-08-01T22:51:35",
        "source": "DC",
        "project": "ASSESS-X",
        "timestamp": "2022-08-01T22:51:35"
    },
    "data": {
        ...
    }
}