packer
A Record and the information required to generate the file name where the Record will be stored in a Mothra repository.
A Record and the information required to generate the file name where the Record will be stored in a Mothra repository.
The record to write to the file.
The relative path of the file to return; does not include time prefix or filename suffix, but does include a version number
The date+time to use for time-based parts of file path
The domain used for all IPFIX Messages in the file; this is needed in case a new file must created and opened.
A method for Mothra packers to map Records to storage locations in a Mothra repository.
A method for Mothra packers to map Records to storage locations in a Mothra repository. It permits storing a Record zero, one, or multiple times, and it also allows the opportunity to modify the Record before it is stored.
PartitionerConfigurator instead. While some packing tools
(mothra-packer and mothra-repacker) still allow general
PackingLogic definitions to be used, the more structured
approach of splitting records into a partition structure is more
generally useful and safe.
To use a PackingLogic object for the packing logic configuration
of one of the Mothra packer tools, use a file containing something
like the following:
import org.cert.netsa.io.ipfix.Record import org.cert.netsa.mothra.packer.{PackableRecord, PackingLogic} new PackingLogic { override def pack(rec: Record): Iterator[PackableRecord] = ... }
Those tools will then compile this and use the partition configuration for packing records.
A method for Mothra packers to determine into which single partition (if any) on one level of a partitioning hierarchy a Record should be stored.
A method for Mothra packers to determine into which single partition (if any) on one level of a partitioning hierarchy a Record should be stored. Partitioners are used to partition (split) records into various directories in the long-term data storage location.
A single Partitioner typically splits on one attribute of a Record, such as its source IP address, IP protocol, or destination port.
A sequence of multiple Partitioners is used by PartitionerConfigurator to partition on multiple attributes in order.
the companion object for numerous ways to define simple Partitioners on various data types. These allow you to quickly specify ranges or specific values for different partitions.
A method for Mothra packers to map Records to storage locations in a Mothra repository by using multiple Partitioner definitions.
A method for Mothra packers to map Records to storage locations in a Mothra repository by using multiple Partitioner definitions. Each Partitioner is called in order to produce a single optional path element to be included in the final storage location. The combined list of path elements (excluding any which are None) is used to construct the final path.
To use a PartitionerConfigurators for the packing logic configuration of one of the Mothra packer tools, use a file containing something like the following:
import org.cert.netsa.mothra.packer.{Partitioner, PartitionerConfigurator} new PartitionerConfigurator { override def partitioners: Seq[Partitioner] = { ... } }
Those tools will then compile this and use the partition configuration for packing records.
An implementation of the PackingLogic trait.
An implementation of the PackingLogic trait. The pack() method
stores Records into files based on a sequence of Partitioners that is
passed to this class's constructor.
How records are to be partitioned.
An implementation of the PackingLogic trait.
An implementation of the PackingLogic trait. The pack() method
stores Records into files based on the record's IP Protocol and source
transport port.
In this version, a directory is created for each unique protocol and source port. For a version that groups uncommon protocols and ports into larger files, see ProtoSportPartitionerPackLogic.
ProtoSportPartitionerPackLogic for a better way to pack Records
A simple class whose partitioners() method needs to be called to
get a sequence of Partitioners.
A simple class whose partitioners() method needs to be called to
get a sequence of Partitioners. Those results are used to
configure a PartitionerPackLogic.
The sequence of Partitioners returned by this class split first by protocol and then by source port.
See the comments in the VlanPartitionerPackLogic source file for instructions on how this file needs to be edited to support loading these partitioners at run-time.
ProtoSportPackLogic for an older version of this concept.
An implementation of the PackingLogic trait.
An implementation of the PackingLogic trait. The pack() method
stores records into files based on the record's IP Protocol.
The start time of a Record.
The start time of a Record. All returned values use UTC.
The date (YEAR, MONTH, DAY) part of the start time.
The time (HOUR, MINUTE, SECOND) part of the start time.
A simple class whose partitioners() method needs to be called to
get a sequence of Partitioners.
A simple class whose partitioners() method needs to be called to
get a sequence of Partitioners. Those results are used to
configure a PartitionerPackLogic.
The sequence of Partitioners returned by this class split by vlanId.
ProtoSportPartitionerPackLogic for a different set of partitioners.
A PackableRecord factory.
An object to hold classes that extend the [Partitioner] trait.
A RecordStartTime factory.
The public traits and classes in this package are used to provide configuration for the Mothra packing tools. (
mothra-packer,mothra-invariantpacker, andmothra-repacker.)All three tools may be configured using PartitionerConfigurator objects, which provide a sequence of Partitioners to determine which single partition directory within a Mothra repository should receive each record. This is the recommended method for configuring partitioning.
For more specialized needs, outside the standard Mothra repository structure, PackingLogic objects may produce any number of output PackableRecords for each input Record, allowing for arbitrary directory structures, record re-writing, record filtering, and other possibilities. We do not recommend using this configuration method unless you have very specific needs.